EVALUATIONS OF SCIENTIFIC PRODUCTIONS: CHALLENGES AND MOTIVATIONS OF EDITORS AND REVIEWERS

ABSTRACT

Objective:

This investigation was to understand the challenges and motivations of editors and reviewers in the evaluation process of productions submitted to scientific journals.

Methodology:

To this end, this research was conducted through a methodological approach focused on the qualitative approach, using the exploratory research method and questionnaires directed to reviewers and editors of journals in Information Science and Education, to collect data to be treated through content analysis with the establishment of categories.

Results:

The results show that most of the responding editors and reviewers of this research, showed to be experienced in the perception of editorial work, and that in Brazil those involved with scientific editing are volunteers, and still little recognized for academic merits.

Conclusion:

It is concluded that, with the complexity, relevance and the controversies around the evaluation process, which involves all the actors of the scientific production (editors, reviewers and authors), perhaps these role experiences help in the understanding of the need of a well elaborated and timely opinion, but also, should contribute so that the several actions suggested by the research subjects, were really put into practice, collaborating to the development of science and the scientific community.

KEYWORDS:
Editors; Reviewers; Scientific productions; Articles; Peer review

RESUMO

Introdução:

O conhecimento científico para ser comunicado requer diversas etapas, a exemplo do processo de avaliação por pares, ou seja, por especialistas nas temáticas a serem analisadas, visando publicar produções que irão beneficiar a comunidade acadêmica.

Objetivo:

Esta investigação teve como foco compreender os desafios e motivações de editores e avaliadores, no processo de avaliação de produções submetidas a periódicos científicos.

Metodologia:

Para tal, conduziu-se esta pesquisa por meio de aporte metodológico focado na abordagem qualitativa, com uso do método de pesquisa exploratória e de questionários direcionados a avaliadores e a editores de periódicos da Ciência da Informação e da Educação, a fim de coletar dados a serem tratados por meio da análise de conteúdo com o estabelecimento de categorias.

Resultados:

Os resultados demonstram que a maioria dos editores e avaliadores, respondentes desta investigação, é experiente na percepção do trabalho editorial, e que no Brasil os envolvidos com a editoração científica são voluntários, e ainda pouco reconhecidos por méritos acadêmicos.

Conclusão:

Conclui-se que, com a complexidade, relevância e as controvérsias em torno do processo de avaliação, o qual envolve todos os atores da produção científica (editores, avaliadores e autores), talvez essas experiências de papéis ajudem na compreensão da necessidade de um parecer bem elaborado e em tempo hábil, mas também, deveriam contribuir para que as diversas ações sugeridas pelos sujeitos da pesquisa, fossem realmente colocadas em prática, colaborando para o desenvolvimento da ciência e da comunidade científica.

PALAVRAS-CHAVE:
Editores; Revisores; Produções científicas; Artigos; Revisão por pares

1 INTRODUCTION

The scientific knowledge produced by professors and researchers at educational institutions and research centers, in order to be communicated, requires several stages, such as the peer review process, which aims to publish productions that will benefit the academic community, and consequently society in general through services, techniques and processes applicable to various areas, patents, medicines, vaccines, etc.

Those involved in the world of science communication are well aware of how the process of evaluating articles submitted to scientific journals, which are peer-reviewed, works. This system of evaluation or peer review is considered to be a certification of quality, indicating that the publication of an article submitted to a journal is original, innovative, up-to-date, presents a grounded theoretical and methodological context, quality in the results and arguments, in short, if it makes effective and relevant contributions to the advancement of science.

The peer review system is also used to analyze articles submitted to scientific events, research projects submitted to calls for proposals or for master's and doctoral degrees, for example, when applying for scholarships, as well as other types of evaluations. However, for this particular study, the focus is only on the evaluation of scientific articles submitted to journals in the fields of Education and Information Science in Brazil.

The justification for producing this research is due to the fact that the authors have experience as editors of scientific journals and as evaluators of various articles in these two areas of knowledge in focus and feel the need to contribute to discussions and to the advancement of this theme. The aim of this research was to understand the challenges, motivations, delays in evaluations, the reward system and the selection of new evaluators from the point of view of editors and evaluators, using questionnaires applied to research subjects in the areas of Education and Information Science.

2 EVALUATIONS OF SCIENTIFIC PRODUCTIONS

There are various types of peer review: single-blind (the reviewer and editor know the author's identity, but the author doesn't know who reviewed their article); double-blind (only the editor knows the identity of the author and the reviewer); triple-blind (only the editor-in-chief knows the identities of the section/associate editor, the reviewer and the author) and open review (all those involved in the process have their identity revealed and the reviews should be published together with the article with the identities of the reviewers). It can be seen that in the areas of Information Science and Education in Brazil, there is still a predominant use of double-blind evaluation. Gradually, some journals are starting the process of open peer review, some of which leave it up to the author to choose whether or not to publish the opinion of their article, but without identifying the reviewers.

There are advantages and disadvantages to all the types of evaluation listed above, but it is not the purpose of this article to detail them, but rather to raise a reflection regarding the double-blind system, which allows evaluators to make critical analyses without fear of being rejected or being the target of some kind of 'academic revenge'; but at the same time this type of evaluation can provide a feeling of freedom for some evaluators, to write unfounded, aggressive, in-depth and biased evaluations, obviously in these cases the editor must intervene.

There is also criticism of this evaluation model, as it is possible for an experienced evaluator to discover the identity of an author, if they have been working in that field for years and have other writings on the subject, or if it is an article from a dissertation or thesis already deposited in an institutional repository. The right thing to do would be for the reviewer to refuse if he or she sees a conflict of interest in assessing the article. However, sometimes this doesn't happen, which further reinforces the need for training and preparation of evaluators, who perform a dual function in their work: evaluating scientific productions, helping the editor to choose what will or will not be published through their opinions, as well as guiding authors of submitted articles to optimize their writing through notes based on suggestions for improvements in structural, theoretical, epistemological, conceptual and methodological terms, among others.

In this way, this evaluation activity plays an important role in the communication of scientific knowledge and in the growth of science, and therefore requires recognition and proper preparation to perform this function, since, theoretically, a peer-reviewed article carries with it a certification of quality for having been analyzed by experts in the field.

In the relationship between evaluator and author, the editor acts as a mediator, receiving criticism from authors regarding superficial, incomprehensible, dubious, fragile, disrespectful, lengthy and inconsistent opinions. This situation of evaluations in this format can damage the image and credibility of the journal and the editor, by sending articles to evaluators who are potentially unprepared or perhaps uninterested in a job that is mostly voluntary, which in no way justifies the lack of interest. According to Stumpf (2008, p. 23), the evaluation of an article is a serious and complex task, which aims to "educate the authors, suggesting corrections to the form and presentation of the contributions submitted. The criticism should help them, above all, in the preparation or writing of subsequent works that they wish to publish". However, there are reviews that do not achieve this goal, some are poorly prepared, or telegraphic, focusing, according to Gross (2020), on extensive observations directed at grammatical errors, specific terms or works not cited, when they should focus on the theoretical, conceptual, methodological, epistemological and structural aspects of the production. Cabral (2018, p. 434) also emphasizes that some opinions are considered "unfair and useless [...] but should be seen as a result of the maturing process of academia, which is certainly better today than it was 10 or 20 years ago."

For this reason, the relationship between reviewers and editors can be described as delicate, because on the one hand there are dissatisfied authors who may feel wronged by an inconsistent review, and on the other hand there are volunteer reviewers with whom editors may not want to get involved. Of course, this is not to say that these situations occur in all journals. In a study carried out by Maia, Farias and Farias (2022), the authors analyzed the perception of knowledge sharing among reviewers from the point of view of scientific editors and, based on the data analyzed, observed that the latter perceive the relevance of knowledge sharing among reviewers, but have not yet put such initiatives into practice in the journals in which they work. It is therefore clear that there is still a long way to go in order to build and strengthen this delicate relationship. As Rigo (2017, p. 513) explains, sometimes we are authors and sometimes we are reviewers, because "the reviewer has their own article being reviewed by a colleague in another journal, or sometimes in the same journal". For this reason, the author points out that in order to remedy the various weaknesses of peer review, something simple and complex is needed at the same time: empathy, in order to "guarantee more legitimacy to the review system and also to reinforce attitudes such as impartiality, transparency and cordiality, capable of generating and consolidating trust within the scientific community".

Another point for reflection is certain attitudes in the peer review process when reviewers violate the rules, such as misconduct demonstrated in the form of plagiarism of the article being reviewed or indicating the reviewer's own work to the authors of the work in order to obtain citations. According to Stumpf (2006, p. 4), "the delay in evaluation can also be seen as facilitating plagiarism, because by prolonging the evaluation, evaluators can incorporate the results or methods of the work they are evaluating into their own research or publications.

These attitudes are referred to by Rigo (2017, p. 513) as misconduct, which "when perceived, creates a sense of injustice in the review process and calls into question the legitimacy not only of the process but also of the journal". According to this author, there are other aspects to consider in the process of evaluating scientific productions with regard to the reviewers:

In the peer review process, the reviewer becomes "powerful", as they have been assigned the task of building an opinion on the work that will guide the editor's decision-making. Their power of persuasion can either lead the editor to ruthlessly reject the work or approve it immediately. However, more often than not, reviews are constructed in such a way as to offer "salvation" to the work, leading to a more cautious decision to proceed with it to future rounds of evaluation until it reaches what the reviewer sees as suitable or ready for publication. (RIGO, 2017, p. 512).

As noted in the author's reflections, evaluators have a crucial task of deciding what they are evaluating, thus requiring not only technical skills in the act of evaluating, but also behavioral skills, acting consciously and ethically, without impregnating the opinion with personal opinions or indicating their own publications as research sources for the authors of the articles being evaluated.

3 METHODOLOGY

The methodological path that led to this research is based on the qualitative approach, which was used because it "does not intend to apply pre-existing concepts, and the research instruments and techniques are designed based on what the researcher feels when getting to know the subjects and the reality that surrounds them in the field of research", as Farias (2014, p. 112) explains. In terms of method, exploratory research was used, as it is understood that this type of research allows for the expansion of the knowledge to be studied and can also increase the researcher's experience with a specific problem, as well as provide an overview of a subject and/or concepts surrounding the topic in focus.

Two Google Forms questionnaires were used to collect data - one for editors¹ and the other for reviewers². Both questionnaires were provided with an informed consent form that explained what would be done with the data collected. The questionnaires were available from October 21, 2021, to January 5, 2022, and were distributed through various media, such as the Education discussion group list, the ANPEd Forum of Journal Editors in the Field of Education (FEPAE), and the Information Science researchers email list. The target audience for this research was editors and reviewers in the fields of Information Science and Education, chosen for convenience, as the researchers in this research had experience as editors and reviewers in these two fields of knowledge.

Before the questionnaires were made available, two pilot tests were conducted: one with an evaluator and the other with an editor, with the aim of improving this survey instrument. Both pilot test respondents did not suggest any changes to the questionnaires, stating that they were clear and objective. The responses from the pilot tests were considered in the data analysis, which was carried out using Bardin's (2009) content analysis, since, according to this author, it is a set of techniques for analyzing communication, using different tools simultaneously to achieve the objective of classifying and describing the data collected in an analytical and systematic way.

3.1 Profile of the research subjects

In order to organize and clarify the data in the light of the research objectives, categories of analysis were created called evaluators' perceptions and editors' perceptions, which are described in section 4 of this article. Before presenting the data collected and analyzed, a profile of the 41 evaluators who took part in the survey is presented. The majority of respondents (51%) work as reviewers for scientific journals in the field of Education, while 49% work for journals in Information Science. Regarding the age group of the respondents, Table 1 shows that most of the respondents (41%) are between 41 and 55 years old, followed by the age group between 58 and 68 years old (27%), then the age group between 24 and 40 years old (22%), and finally the age group between 71 and 75 years old with 10% of the evaluators subject to the survey. This data could mean that the age group between 41 and 55, which had the highest number of respondents, may also have the highest number of evaluators, as it is understood that there is an indication of stabilization in teachers' careers, but in order to make this claim, it would be necessary to have exact figures of the total number of evaluators in Brazil, in the two areas in focus.

Table 1
Age range of reviewers

Evaluators	Quantity	Percentage
Range 41 to 55	17	41%
Range 58 to 68	11	27%
Range 24 to 40	9	22%
Range 71 to 75	4	10%
Total respondents	41

• Source: research data (2021/2022)

Regarding the length of experience of the research subjects as reviewers of scientific articles, Table 2 shows that the majority of respondents (73%) have been involved in this activity for more than five years.

Table 2
Length of time working as a scientific article reviewer

Time of performance	Quantity	Percentage
l to 2 years	2	5%
2 to 3 years	1	3%
3 to 4 years	3	7%
4 to 5 years	5	12%
Over de 5 years	30	73%
Total respondents	41

• Source: research data (2021/2022)

Only 5% of the evaluators are still new to this role, with one or two years' experience, while the rest have more than two and up to five years. Regarding the number of scientific journals in which the subjects work as evaluators, Table 3 shows that the majority (61%) work in more than five journals, followed by three journals (15%), two (10%), and one and four journals tied at 7%.

Table 3
Number of journals the evaluators work for

Nº Journals	Quantity	Percentage
l	3	7%
2	4	10%
3	6	15%
4	3	7%
Over de 5	25	61%
Total respondents	41

• Source: research data (2021/2022)

With regard to the profile of the 40 editors who took part in the survey, the percentage of participants is as follows: 72.5% work in journals in the field of Education and 30% in Information Science; the total was more than 100%, as one editor answered that he worked in journals in these two fields of knowledge. Table 4 shows that the majority of respondents are aged between 41 and 49 (32%), followed by between 50 and 59 (30%), between 60 and 66 (15%), between 71 and 81 (13%) and between 33 and 40 (10%). As you can see, most of the respondents are between 41 and 59 years old, a similar age range to the evaluators. Bearing in mind that in many journals of higher education institutions (HEIs) in the field of Education and Information Science, lecturers also act as editors, it can be inferred that this is a predominant age group, as lecturers must feel more secure and active in their professions in order to accept the challenge of acting as editors.

Table 4:
Age range of editors

Editors	Quantity	Percentage
Range 33 to 40	4	10%
Range 4l to 49	13	32%
Range 50 to 59	12	30%
Range 60 to 66	6	15%
Range 71 to 8l	5	13%
Total respondents	40

• Source: research data (2021/2022)

The editors were also asked how long they had been working in this role for scientific journals. In Table 5, the data show that 63% of the editors have more than five years of experience, followed by a tie of two to three years and three to four years with 12% each, and four to five years with 8%, and the least experienced with only one to two years is only 5% of the respondents.

Table 5
Length of time as a journal editor

Time of performance	Quantity	Percentage
1 to 2 years	2	5%
2 to 3 years	5	12%
3 to 4 years	5	12%
4 to 5 years	3	8%
Over de 5 years	25	63%
Total respondents	40

• Source: research data (2021/2022)

In the end, it was observed that the editors and evaluators from both areas (Education and Information Science) who took part in this survey are considered experienced in their roles and can contribute to strengthening the process of evaluating scientific articles with their answers, which are analyzed in the following section.

4 DATA ANALYSIS AND DISCUSSION

The data collected through the questionnaires were tabulated to be analyzed in two categories: The first, reviewers' perceptions, aims to understand, from the point of view of the research subjects, what initiatives they are aware of aimed at rewarding and motivating reviewers, how they perceive the relationship between editors and reviewers, what the main challenges are in the review process, and whether they know of measures to "win over" new reviewers and to keep those already on the editorial board; As for the second category, perceptions of editors, it aims to understand, from the point of view of the editors of scientific journals, what initiatives the journal where they work has taken to reward reviewers, as well as what are the recurring problems related to reviews, what is the average time to receive reviews, and what actions the journal would take to select new reviewers and retain those already working in the journal. The respondents were named from A1 to A41, i.e. reviewer 1, reviewer 2, and so on.

The first category begins with information on whether or not there are initiatives by journals to reward the work of reviewers. According to the data collected, 27 respondents indicated that there is no reward system, twelve responded that there is (citing the review statement or certificate), and two responded that there is no financial reward, with A38 emphasizing this:

Most journals don't have financial reward initiatives. I wouldn't do it out of financial interest either. The initiative is usually implicit and not explicit, such as being a reviewer for the journal that is best rated by the indexers. An explicit initiative I always receive is to put my name on the editorial board, send a statement of evaluation and a message of thanks. But there is an initiative I receive from a foreign journal that I think is very important for the reviewer: the editor sends me messages about the progress of the review of the article, showing how my opinion was used to improve the article. An excellent initiative!

The respondent above mentioned one reward initiative that is considered common, which is the issuance of a statement or certificate of evaluation, and another that was not mentioned by any other respondent, which would be to obtain information on the progress of the article within the editorial process. Another initiative mentioned by Gross (2020) would be a list of the best reviewers of the year, with the aim of valuing the efforts of reviewers to maintain the quality of scientific productions. Some other measures to reward and motivate reviewers are mentioned in the subcategories below.

In the same questionnaire, reviewers were also asked if they knew of any measures taken by the journals for which they work to "attract" new reviewers and to motivate those already on the editorial board to continue reviewing. The majority (80.5%) of the reviewers did not know of any type of action, while 19.5% said they knew of these actions, which were detailed in the open-ended responses, which were analyzed in the same category of reviewers' perceptions, divided into subcategories, namely: a) feeling rewarded and motivated to review; b) analyzing the relationship between editors and reviewers; c) biggest challenges in the reviewing process; and d) actions that could be implemented to reward/winning/motivate reviewers.

The first subcategory begins with whether there is a reward and motivation to evaluate. There are several reasons why evaluators feel motivated and rewarded, listed below, which were similar among some of the respondents, some of whom also identified themselves as editors and were aware of the difficulties in obtaining evaluations. The motivations cited by the evaluators were: contributing to the development of the field; contributing to the dissemination of knowledge; exchanging experiences; helping journals; commitment to the collective; guaranteeing the quality of the field's content; getting to know the production of the research field; getting to know how issues are dealt with from different perspectives; solidarity with editors; and reducing the waiting time for authors and editors. Table 1 shows some of the responses to illustrate the evaluators' thoughts on this subcategory.

Chart 1
Impressions of reward and motivations for evaluation.

It's important to give this feedback to the authors. It's a commitment of those in academic careers to collaborate with research. If everyone did a bit, there wouldn't be so many problems and wear and tear. In addition, the number of evaluations could be reduced. (A15)

Yes, because it's something I enjoy doing, it's a pleasure to contribute to scientific communication. But if there were some form of reward, it would certainly increase my motivation even more. (A13)

The motivation is intellectual, in the learning that an evaluation offers the evaluator and the possibility of internalizing this learning in their research practice, in addition to the possible possibility of projection at curriculum level. (A19)

From the point of view of scientific academic altruism, promoting science and encouraging researchers, yes. But I feel that other motivators are lacking (more traffic to publish in journals, slow response from some of them, bureaucracy that prevents progress, as well as the accumulation of requests over several periods). (A22)

Yes. I understand that it's a partnership, because as an author myself, I want my articles to be carefully evaluated. The basis of science is peer review. (A26)

The invitation to an evaluation of this nature denotes acceptance of your expertise in the subjects and of the titles you hold as recognition of your academic and scientific importance to the areas for which you collaborate. (A29).

Yes and no. Yes, because it's a way of entering the educational field. No, because this is just another unpaid job that teachers do. (A30)

I always feel motivated to evaluate and collaborate with scientific communication and the journal. In addition, I value the work of the reviewer, as I am a journal editor, and I know how much we need reviewers to boost the editorial process. (A33)

First of all, it's a way of finding out about the various research projects underway in my area of research, although at no time have, I used this knowledge for my own benefit. But it is a way of keeping up to date. I like evaluating because it's something I can do to improve publications and teach authors how to improve their writing and organization of results. (A38)

• Source: research data (2021/2022).

As can be seen in chart 1, the evaluators' motivations are diverse, ranging from their commitment to science, to scientific communication, to the possibility of learning, to the return of their article, which is also submitted, and is analyzed in the same careful way as the responding evaluator, but there are those who deal with the lack of remuneration and the accumulation of duties of the teacher, who is also an evaluator. For this reason, "scientific academic altruism" is perceived, as A22 reports, in the work of the evaluators in the face of all the difficulties, such as the "lack of other motivators (more traffic to publish in journals, slow response from some of them, bureaucracies that prevent progress, in addition to the accumulation of requests in various periods)". According to Gross (2020, p. 6), one of the possibilities to encourage the "preparation of good reports would be precisely the valorization of this activity by Capes in the evaluation of postgraduate programs", but obviously, other long-term planned actions would be needed to encourage more motivated and engaged evaluators.

The survey data also shows that some respondents, such as A2, A11, A14 and A24, were categorical in their response, writing only that they "don't" feel rewarded or motivated. Others justified not being motivated or rewarded: "Not much. It depends on whether the topic is interesting for my research or that of my students" (A8); "I don't feel rewarded. But what motivates me is a certain amount of recognition for the subject of the articles" (A9); "No, I only do it to help colleagues because I used to be an editor and it's a difficult task" (A28); "No. I only get points on my CV and when I receive an evaluation statement" (A31); "I don't feel motivated, just a 'moral' obligation" (A35). As observed, some evaluators do not feel motivated, but continue to evaluate articles because they count points on their CV or because they feel they need to contribute to the communication of science, which is corroborated by Stumpf (2008, p. 23), when he states that evaluation "does not bring public recognition, much less remuneration for evaluators. But many scientists see the task as part of their obligation to science".

With regard to the subcategory analyzing the relationship between editors and evaluators, the majority of survey respondents say it is good, respectful, cordial, calm and appropriate, built on trust, as can be seen in the responses in Chart 2.

Chart 2
Relationship between editors and evaluators

I find it efficient in most cases. In many situations it's even possible to contact them outside the platform (whats, email) to speed up the revision process. And even when communicating via the platform, it works adequately, usually with a response within a fairly quick timeframe. (A1)

I think that the relationship between editors and reviewers depends on several factors, for example: the prestige of the journal, the resources available to reward reviewers, the length of the reviewers' careers. (A2)

Well, in general it's impersonal. It only changes when you have friendships. (A5)

Unprofessionally in general. Contacts are by auto-response, the better qualified the magazine, the less tolerance the editor has for delays. In my experience, therefore, I would ask you not to generalize. (A8)

I see them as two partners, each with their own role, and they need to work together to ensure the high quality of the journal's work. (A13)

I have a good professional relationship with the editors of the journals I work for. (A19)

Nowadays, evaluators are "captured" by proximity and personal affinity. (A25)

Relax, the evaluator has a certain amount of decision-making power, that's interesting!" (A28)

As an evaluator, I receive the invitation, followed by a thank you. Nothing beyond those formalities." (A29)

I think the relationship is collaborative and also requires understanding. (A33)

Positively. The editors depend on the reviewers and they know it. We're partners, but I think the editors in general should give the evaluators feedback on what happened with the articles. (A38)

• Source: research data (2021/2022).

Chart 2 shows a positive overview of the relationship between reviewers and editors from the perspective of the reviewers who participated in this research. The view of these subjects is that it is a collaborative, positive relationship, or even impersonal, as A5 states: "It only changes when you have bonds of friendship", which A25 also emphasizes by stating that "reviewers are 'captured' by proximity and personal affinity". A2 emphasizes that factors such as the prestige of the journal and the length of the reviewers' careers can influence the relationship between editors and reviewers, which leads to the perception of differences in thinking between the respondents, as it depends on the experience and length of experience of each.

In contrast, nine of the 41 respondents considered the relationship between reviewers and editors to be: distant, reasonable, cold, non-existent, weak, and lacking dialogue. According to E11, there is no relationship, and if there is one, it is distant. This could be due to various factors, such as the accumulation of tasks for editors who are professors and researchers, and who may not have a reduced workload to devote to the position of editor, causing, for example, as Cabral (2018, p. 436) explains, "neglect of their own research agenda, a reduction in the quality of their teaching, or even a reduction in the time they devote to their loved ones." No attempt is made to justify the lack of dialogue reported by the evaluators, only to point out possible causes.

Regarding the subcategory "biggest challenges in the evaluation process", 15 respondents said that the biggest problem is having the time to develop a careful analysis and a consistent opinion of the articles, because they combine several teaching activities with the role of evaluator, as A13 points out, who acts in other roles in the journals and feels pressure from both the evaluators and the editors. In addition, some reviewers said that it was difficult to review texts with grammatical errors, no content, and poor writing that took time to understand. Other responses are listed in Chart 3, which shows the evaluators' reflections on the challenges they experience in this task.

Chart 3
Challenges in the evaluation process

In some situations where the articles are based on statistical and even empirical data, I believe that incorporating the scientific data that underpins the research has increased the demands on the evaluators, since they need to check that the results are correctly represented. In this case, it requires more time and competence in the subject being assessed. (A1)

The constant updating of the themes made available. (A3)

Not using Publons and/or ReviewerCredits accreditors (A6)

Decide whether or not the article should be rejected or if it can be redone and resubmitted. Another precaution is to know how to choose your words so as not to discourage or attack the author. Knowing how to make suggestions within the theoretical and methodological framework chosen by the author. This requires empathy and study on the part of the referee. (A8)

Finding the courage to evaluate, to give a quick and quality response, knowing that our own submissions take months to be evaluated. (A11)

It takes a long time to send the evaluation voucher. It's not uncommon for me to have to ask, and that's very bad." (A12)

I'm speaking as an editor - it's a very disparate quality of evaluation. Sometimes they are opposite, which means more commitment, time and delay. Another problem is finding evaluators who agree to take part. (A16)

Aiming to evaluate without belittling the evaluated, paying attention to the characteristics of the submitted manuscript and, at the same time, looking for ways to contribute to its improvement. (A19)

Being fair and understanding that sometimes flaws can be pointed out, fixed and the writing still has merit for dealing with the subject in a way that contributes to the area of knowledge to which it was submitted. (A20)

There are still many evaluators who don't have the maturity to give complete opinions that really state the reason for acceptance or rejection. (A27)

At the moment, the biggest challenge is self-plagiarism. There are submissions that have aspects of "rehashed" work, which forces me - as a reviewer - to read work by that author in order to copyedit that work with what the author has submitted. (A29).

Evaluating an article presents the same difficulties for the evaluator as those involved in evaluating any academic-scientific work. Demanding linguistic adequacy, scientific rigor, originality and obedience to the technical standards of reference records and formatting is becoming almost impossible in the editorial space of Brazilian education. Evaluating work that fails to meet these requirements is always very difficult. (A30)

The illegibility of texts with problems of the author's mental organization; problems of plagiarism; unfinished research that presents poorly-written results in order to say that it has been completed; texts from areas of knowledge that are not in the evaluator's domain. (A38)

Lack of normative standardization in line with ABNT. There are different standards for each journal. (A40)

• Source: survey data (2021).

Chart 3 gives a glimpse of the day-to-day life experienced by evaluators with a wide range of challenges, such as self-plagiarism, illegibility of the text, lack of originality, scientific rigor, the delay on the part of editors in sending proof of the evaluation, as A12 states "It's not uncommon to have to ask, this is very bad", among other problems, as A38 explains "plagiarism; unfinished research that presents poorly written results to say it has been completed; texts from areas of knowledge that are not in the evaluator's domain." Even so, there is a concern to be careful when issuing an opinion, to be fair, so as not to discourage the author, as A8 points out, when explaining that it is necessary to "know how to make suggestions within the theoretical and methodological framework chosen by the author. This requires empathy and study on the part of the referee." Gross (2020) emphasizes this when he points out that having well-prepared opinions allows for the promotion of quality and consistency in the productions analyzed, promoting the growth of the actors involved in this process, i.e. evaluators and authors.

In the subcategory of actions that could be implemented to reward/conquer/motivate reviewers, the respondents suggested the following: sending statements/certificates to evaluators quickly or automatically through the system; opinions should be scored in postgraduate programs, in competitions, project submissions to calls for proposals from funding agencies, in internal evaluations carried out by universities (career progression); sending articles in the evaluator's area of interest; institutions should value and consider evaluations as an academic activity; adoption of tools such as Publons³ and ReviewerCredits⁴ for evaluator registration and recognition; invite the evaluator to publish in the journal in which they work, in a special issue or thematic dossier, for example; have some kind of tribute on the journal's website to the most active evaluators, publicizing their work and curriculum; try to train postgraduate students as future evaluators; keep the option of registering new evaluators on the journal's website; promote courses related to the activity; increase the deadlines for accepting and submitting evaluations; evaluators could have preference in publishing, and in proposing thematic dossiers; remunerate evaluators. Chart 4 shows more actions that editors can use, as well as those already mentioned, to implement in the journals in which they work.

Chart 4
Actions to reward/conquer/motivate evaluators

I feel there is a lack of natural incorporation as a reviewer into the Scientific Committee of journals after a considerable number of reviews. It is notorious that some journals add evaluators due to the representation of the curriculum, seeking to bring greater credibility to the journal. This model is recurrent in journals seeking internationalization and is repeated even among the names of researchers at a national level. This practice seems understandable and even necessary to me, but I think it's important for journals to adopt a balance between the quantity and quality of reviews, in order to incorporate new members of the scientific committees, i.e. regular reviewers who make a significant contribution. I don't know if any journals already do this, but I think it would be a satisfactory way of rewarding reviewers. (A1)

No. Most Brazilian journals are not professionalized, so volunteering not only increases the risk of thematic or theoretical conflicts of interest on the part of editors, it can offer nothing to technical work that could not be volunteered. What's more, Sucupira doesn't consider an opinion to be noteworthy. Technical work is worth almost nothing in terms of teaching output. (A8)

I'm in favor of training in evaluation, whether through Anped, Coneped, courses (A25)

Publication of a newsletter at the end of each editorial year with a body of evaluators; publication of the average evaluation time at the end of the cycle. Obviously, always preserving the impersonality of the journal in terms of the evaluation process. That's the basics (A31)

In addition to listing the evaluators who collaborate the most, evaluate with quality and meet deadlines, perhaps highlighting the 3 most prominent evaluators in this regard in the Official Report. (A33)

Sending messages to the evaluators to follow up the process of accepting or not accepting articles, so that they know what has been used and what has not; tutorials on evaluation style; good evaluation practices for the journal. (A38)

• Source: research data (2021/2022).

As noted in the A8 reflections, there are a number of issues related to the evaluation process that require the attention of editors, journal editorial boards, scientific publication promotion agencies, undergraduate and postgraduate regulatory bodies, the colleges and deans of the courses with which the journals are associated, editorial boards, journal portals, etc., with the aim of motivating and rewarding the work of reviewers, as well as of the editors themselves, who often carry out this activity on a voluntary basis, as seen in the next category, which deals with the editors' perception.

In this category, the editors are referred to as E1 to E40, and the questions have been divided into the following subcategories: a) initiatives to reward reviewers; b) recurring problems related to reviews; c) time taken to receive reviews; d) measures in the journal to select new reviewers and to keep those already on the editorial board motivated to review.

The first subcategory aimed to find out whether the journal where the researchers (editors) work has initiatives to reward reviewers. The data shows that 77% of the respondents answered that there are initiatives and 23% answered that there aren't any. The one most often mentioned by the editors is the issuance of a certificate/declaration of evaluation. According to E9, "Issuing a statement is the least an editor should do. I don't even consider it a reward for the reviewer. In the case of E33's journal, a list of reviewers who have contributed is published in the issues, "regardless of the editorial decision (but with the process completed during the period)". E12 says that the journal he works for uses ReviewerCredits as a reward, while E18 is paid for reviewing the journal he edits.

Looking at the responses of the reviewers to the same question of whether or not there are reward initiatives, where 27 reviewers (the majority) said that there is no reward system, and twelve said that there is (the review statement), it can be seen that there is no consensus between reviewers and editors on what rewards would be in this process, since the majority of editors responded positively to the same question. Of course, the respondents to this survey may not work for the same journals, i.e. the reviewers and editors who answered the questionnaire are from different institutions, but this can be considered a small sample of the overall picture.

With regard to the subcategory recurring problems in relation to evaluations, the editors point to the delay in evaluation times (46%) as the biggest challenge, followed by inconsistencies in opinions (28%), rudeness (9%) and conflicts of interest (3%), as can be seen in Table 6. The other problems cited by the editors were: evaluators not answering in the system whether or not they agree to carry out the evaluation; insufficient, poor quality, summarized, poorly written and poorly prepared opinions; complaints about the evaluation received; difficulty in finding evaluators for all themes/approaches; evaluators who poorly evaluate articles with opposing theoretical tendencies; evaluators who do not follow up the evaluation in all the rounds, requiring constant replacement of evaluators and delaying the process.

Table 6
Recurring problems in relation to evaluations

Problems	Quantity	Percentage
Conflict of interest	2	3%
Incoherent opinions	22	28%
Crudeness	7	9%
Delay in evaluations	36	46'%
Others	11	14 %
Total	78	100%

• Source: research data (2021/2022).

Analyzing Table 6 with the problems and percentages presented, it can be seen that the time taken for evaluations and inconsistencies in opinions are the biggest challenges in the evaluation process. According to Stumpf (2006, p. 4), a quality opinion takes time, and the evaluators, who are usually teachers and have a variety of activities, "undertake to carry out the task in good time so as not to interfere with the editor's schedule", but they may not be able to do so within the stipulated timeframe. Another fact warned by the author is that the "delay in evaluation can also be seen as facilitating plagiarism, because by prolonging the evaluation, evaluators can incorporate the results or methods of the work they evaluate into their research or publications", and furthermore, even with the delay in the time of evaluations and the possibility of plagiarism, there are opinions that may not contribute to the improvement of the article, and even contain an aggressive/disrespectful content.

Still on the subject of time, there is the subcategory time to receive evaluations. The most common timeframe for receiving evaluations was one month (43%), followed by other timeframes (39%), detailed below, and then three weeks (4%), with two and four weeks tied at 7% each.

Table 7
Time to receive evaluations

Tempo	Quantity	Percentage
Other	17	39%
One month	19	43%
Four weeks	3	7%
Three weeks	4	4%
Two weeks	3	7%
Total	46	100%

• Source: research data (2021/2022).

Other deadlines cited were two to three months, one semester, and other editors said that it depends on the situation. According to E35, the time it takes to evaluate articles also depends on whether there are technical problems, or, for example, if there are ongoing events in the area: "In general, it is possible to receive an evaluator's assessment within a week, but the minimum period is one month, which can always be changed according to need." As noted, this deadline will vary depending on issues involving the evaluation process itself, technical aspects and even the professional demands of the evaluators. However, Gross (2020, p. 7) warns that the "problems observed are not restricted to the limited availability of evaluators or the limited time and attention dedicated by some evaluators to the opinions". There are evaluations that focus on grammatical aspects and/or standardization, when they should make real contributions to the advancement of the scientific knowledge in question, dealing, for example, with the adequacy of research methods and techniques and conceptual problems.

In relation to the subcategory that reflects on whether there are actions in the journal to select new evaluators and to keep those already on the editorial board motivated to evaluate, 80% of the editors answered yes, and 20% no. Among the most cited actions are: giving more time to evaluate the manuscript (40%); sending a direct message to the evaluator convincing them to stay on the list of evaluators (32%); contacting the evaluator via WhatsApp (11%), as shown in Table 8. Having more time to evaluate scientific productions is also a demand from evaluators who carry out other activities such as teaching, research and administrative issues at the universities where they work.

Table 8
Actions in the journal to select new reviewers

Tempo	Quantity	Percentage
Others	17	17%
Allow more time to evaluate the manuscript	26	40%
Contact to reviewer via WhatsApp	7	11%
Sending a direct message to the reviewer convincing to stay on the list of reviewers	21	32%
Total	65

• Source: research data (2021/2022).

With regard to the others (17%) shown in the table above, the editors responded by suggesting various actions, which have been categorized into indicators in Chart 5 for better visualization, namely in ascending order: adoption of institutional policy and academic recognition were tied; editorial policy and creation of a system for accrediting evaluators, detailed below.

Chart 5
Indicators of the actions suggested by the editors

Ranking	Indicadores	Quantity
1°	Adoption of institutional policy	8
2°	Academic Recognition	8
3°	Editorial Policy	5
4°	Creation of a system of accreditation	3

• Source: research data (2021/2022)

The indicators listed above were used to organize the actions suggested by the research subjects (editors) with a view to selecting new evaluators, as well as keeping those already on the editorial board motivated to continue evaluating. As can be seen in Chart 6, the indicator 'Adoption of institutional policy' includes actions aimed at universities, research centers, etc. where journals are located. This indicator suggests that tools should be created to encourage researchers to become evaluators, and those who already are should be motivated to continue this work, as E39 points out when he explains that there should be "information exchange between editors in the sense of sharing information. For example: a list of reviewers with their contacts, etc. I think the IBICT could promote this integration"; or according to E18's reflections when he suggests strengthening "symbolic remuneration for evaluation"; and also as E17 points out when he ponders that "perhaps it could be better scored in evaluations such as RAAD and in Lattes itself to be considered a relevant task that contributes to the program's score, if it is a professor linked to a postgraduate program."

Chart 6
Actions to select new evaluators and to keep those already on the editorial board motivated to evaluate

Suggested actions	Indicators
Motivation policies for evaluators to be adopted by publishers. (E3)	Adoption of institutional policy
It would be important if these participations were taken into account in the institutions where the researchers work. (E4)
In the case of private HEIs, organize institutional publishing labels and charge the price of books to be published administered by the HEI treasury so that evaluations are remunerated. (E14)
It's a very complex issue because we generally rely on the kindness of professionals for this task. It is known that article evaluation is more of a demand and there is no remuneration. Perhaps it could be scored more highly in assessments such as RAAD and in the lattes themselves, being considered a relevant task that contributes to the program's score, if it is a professor linked to a postgraduate program. (E17)
Strengthen symbolic remuneration for evaluation (E18)
It was discussed at the last CAPES Mid-Term Seminar (2019) that CAPES area coordinators should sensitize colleagues to contribute to the area's sister journals. I think this should be reinforced." (E26)
Policies for valuing journals, editors and evaluators (E28)
There should be an exchange of information between editors in the sense of sharing information. For example: a list of reviewers with their contacts, etc. I think IBICT could promote this integration. (E39)
National base of evaluators (E8)	Creation of an accreditation system for assessors
Searching databases, searching for themes-researchers, requesting nominations for evaluators from the organizers of thematic sections, from well-known researchers. (E8)
Every year, we turn to the list of the Association for Research and Postgraduate Studies in Information Science - ANCIB - for new contacts and to colleagues in the area of Communication and Information, through e-mails, with a view to inviting colleagues to be referees for the journal. (E26)
I've thought about structuring a formal report for them to fill in so that they have more evaluation criteria that aren't based on their practices alone. We're still evaluating it with the team." (E31)	Editorial policy
I do almost all of the above, but I don't think it's innovative, I think it's part of the editor's role to have more direct contact with the reviewer. Thanking them for their opinions, giving them clear guidelines on how to prepare the opinion, passing on a statement. Unofficially, we give a slight precedence in the evaluation to articles by reviewers who are always collaborating. Precedence in evaluation, but a small one, due to the demands of the journal, and in publication in aheadof print AOP. When it comes to publication in AOP, i.e. after it has been evaluated and approved for publication, I always ask when the author, who is also an evaluator, needs the text to be published in SciELO. (E9)
The Journal's Editorial Board always invites new evaluators and adopts the Ad-Hoc Evaluator. (E27)
For new reviewers, we make personalized invitations to authors who have published with us and are not yet part of the reviewer team. There are also contacts made at events. (E34)
The Editorial Board is periodically re-evaluated and only people who actively contribute are retained. Unfortunately, this does little to motivate reviewers. (E35)
Evaluation training for new doctors; mini-courses at scientific events for evaluation training. (E23)	Academic recognition
We commend the work of the reviewer (who has already contributed his work to the Journal) and say that the opinion of such a reviewer is very important for improving the manuscript and for science itself. (E5)
Make the relationship more personalized; value the evaluators with personalized messages. (E12)
Invitation to organize dossiers and/or participate in thematic dossiers (E20)
Thank-you events; publicly visible lists of the most active evaluators by subject, for example; list of the best-rated evaluators (badge/star/patent system or similar). (E33)
I think it would be interesting for the journal to make its evaluators aware of the importance of their work for scientific communication and for the journal, through messages at certain times; for new evaluators, to open up spaces for new masters and doctors to work as evaluators. (E34)
Existence of CAPES and CNPq public notices that can pay for evaluations (E34)
We are always looking for new solutions to guarantee the permanence and activity of the evaluators, as well as constantly renewing the available list. However, we don't have any other options at the moment to improve the performance of the evaluators. We don't use the telephone to communicate with evaluators, as we consider it too invasive. It is an inherent activity of research, but the excess of competing publications overloads the evaluators. We are considering holding regular events to thank everyone who works on producing our publications, especially the authors and reviewers. (E33)

• Source: research data (2021/2022).

In relation to the indicator 'Creation of an accreditation system for evaluators', the editors suggested the development of a national and international base, which would facilitate the search work that is carried out, as E26 explained. An accreditation system for evaluators would be a tool to help in this search by profile, areas of knowledge and interest of evaluators, and could also be a platform for interactivity, promoting training for those interested in this activity.

Regarding the 'Editorial policy' indicator, the respondents indicated actions that they already carry out formally, such as issuing an evaluation statement/certificate, and others of an unofficial nature aimed at motivating those who issue the opinions, as E9 points out: "In an unofficial way, we give a small precedence in the evaluation to articles that come from evaluators who are always collaborating. Small precedence in evaluation, due to demand from the journal, and in publication in AOP. When it comes to publishing in AOP, i.e. after it has been evaluated and approved for publication, I always ask when the author who is also an evaluator needs the text to be published in SciELO." Other editors also reported that they frequently invite new evaluators and adopt the ad-hoc evaluator, as well as observing who is publishing in the journal and can be part of the board of evaluators.

In the last indicator called 'Academic recognition', the editors pointed to actions such as promoting events and courses to train new evaluators, and also to thank them; invitations to organize dossiers and even to take part in thematic dossiers by submitting a paper; making evaluators aware of the relevance of evaluations for scientific communication and for the journal; publishing lists of the most active and best-ranked evaluators; and the creation by funding agencies of calls for proposals that pay for evaluations. These thoughts from the editors were also pointed out by the evaluators, who expressed the need for actions to motivate and reward those who issue opinions, as it is a complex activity that requires time and experience.

5 CONCLUSION

It is understood that the task of being an editor or evaluator involves various complexities, as pointed out by the research subjects. This article presents a brief experience of having and working in both roles and participating in this scenario as authors of this investigation. The aim was to understand the aspects of evaluating scientific productions, with a focus on initiatives aimed at rewarding and motivating evaluators and highlighting the relationship between editors and evaluators, as well as proposing actions to select new evaluators and retain those who are on the editorial board, in a study based on content analysis applied to respondents in the areas of Education and Information Science.

Given the sample of data collected, it can be seen that the editors and evaluators who responded to this investigation were experienced in the perception of editorial work, focusing both on actions involving editors and on the evaluation process for evaluators, highlighting possible actions for improvement in both categories.

In general, it can be seen that the work of both the editor and the evaluator in Brazil, involved in scientific publishing in the areas studied in this research, is practically a voluntary action, with no financial gain and, in a way, little recognition for academic merits.

By categorizing the comments made by the respondents (editors and evaluators), a consensus was reached on indicators for actions suggested by the editors, but which could also be considered by the evaluators, as follows:

• Adoption of an institutional policy to give constitutionality to the work done by editors and evaluators;

• Developing and using strategies and mechanisms to reach higher levels, with the aim of dialoguing about the academic recognition of editors and evaluators in terms of aspects of technical-scientific editorial work;

• Building and implementing an editorial policy with the aim of promoting improvements in the priorities of the work of teachers who work as editors and evaluators of scientific publications;

• Create a system of evaluators to facilitate the work of the editorial process in terms of finding new evaluators to evaluate articles. A system based on the architecture of Lattes itself, but with the exclusive function of "searching for evaluators". Some prototypes are being made, but we need something more professional that covers the entire universe of editors from all areas.

In these four actions we can summarize all the work of analysis emphasized in this research, offering a model to be made available to journals in any field of knowledge, with the aim of: expanding the benefits obtained from the process of evaluating scientific work; highlighting the importance of having experienced editors and evaluators in the management of scientific journals, as well as promoting the visibility of scientific production by highlighting the quality of these peer reviews. It's worth pointing out that a movement needs to be created with the attention of editors, journal editorial boards, editorial groups, journal portals, which are booming in universities due to their exponential growth in support for editors, with the aim of implementing with agencies that promote scientific publication, scientific associations, undergraduate and postgraduate regulatory bodies, colleges and pro-rectorates of the courses to which the journals are linked, a compensation system for both the role of editor and reviewer, and to make their journals academically recognized in the publishing world. It's worth pointing out that a movement needs to be coined with the attention of editors, journal editorial boards, editorial groups, journal portals that are booming in universities due to their exponential growth in support for editors, with the aim of implementing with agencies that promote scientific publication, scientific associations, undergraduate and postgraduate regulatory bodies, collegiate and pro-rectorates of the courses to which the journals are linked, to implement a compensation system for both the editor's role and the evaluators, and to make their journals academically recognized in the publishing universe.

In conclusion, it has been possible to present the perceptions of both reviewers and editors, with the aim of providing a reference for work in the field of scientific communication that deals with the trajectory of the editorial process, scientific peer review, and promoting the creation and adoption of institutional editorial policies.

The analysis of the data collected reveals the complexity, relevance and controversies surrounding the evaluation process, which involves all actors in scientific production, who may be in the role of reviewers, editors or authors. Perhaps these roles will help them to understand the need for a well-prepared and timely opinion, but they should also help to ensure that the various actions proposed by the research subjects are actually put into practice, contributing to the development of science and the scientific community.

Acknowledgements

To the research subjects who took part in this investigation.

To the National Institute for Scientific and Technological Development Council (CNPq - Brazil) for the research productivity grant (PQ2), Process No. 308358/2021-0..

REFERÊNCIAS

• BARDIN, L. Análise de conteúdo. Tradução: Luís Antero reto e Augusto Pinheiro. Edição e revista atualizada. Lisboa: Edições 70, 2009.
• CABRAL, S. Autores, pareceristas e editores: tripé do processo de revisão de artigos científicos. Revista de Administração de Empresas, São Paulo, SP, v. 58, n. 4, p. 433-437, 2018. Disponível em: https://doi.org/10.1590/S0034-759020180408 Acesso em: 20 dez. 2022.
  » https://doi.org/10.1590/S0034-759020180408
• FARIAS, M. G. G. Análise da produção, implementação e avaliação de um modelo de mediação da informação no contexto de uma comunidade urbana. 2014. (Tese de Doutorado em Ciência da Informação). Universidade Federal da Bahia, Instituto de Ciência da Informação, Salvador.
• GROSS, C. P.O. Parecerista: protagonista anônimo da qualidade. Revista Direito GV, São Paulo, v.16, n. 2, e1957, 2020. Disponível em: https://doi.org/10.1590/2317-6172201957 Acesso em: 20 dez. 2022.
  » https://doi.org/10.1590/2317-6172201957
• MAIA, F. C. de A.; FARIAS, G. B. de; FARIAS, M. G. G. Percepção sobre o compartilhamento de conhecimento entre avaliadores sob a ótica dos editores científicos. RDBCI: Revista Digital de Biblioteconomia e Ciência da Informação, Campinas, SP, v. 20, e022003, 2022. Disponível em: https://periodicos.sbu.unicamp.br/ojs/index.php/rdbci/ article/view/8667456 Acesso em: 20 dez. 2022.
  » https://periodicos.sbu.unicamp.br/ojs/index.php/rdbci/ article/view/8667456
• RIGO, A. S. Comunidade acadêmica, produtivismo e avaliação por pares. Perspectivas: Revista de Administração de Empresas, São Paulo, SP, v. 57, n. 5, set./out., 2017.Disponível em: https://doi.org/10.1590/S0034-759020170508 Acesso em: 20 dez. 2022.
  » https://doi.org/10.1590/S0034-759020170508
• STUMPF, I. R. C. Avaliação pelos pares nas revistas de comunicação: visão dos editores, autores e avaliadores. Perspectivas em Ciência da Informação, Belo Horizonte, v. 13, n. 1, p. 18-32, 2008. Disponível em: https://doi.org/10.1590/S1413-99362008000100003 Acesso em: 20 dez. 2022.
  » https://doi.org/10.1590/S1413-99362008000100003
• STUMPF, I. R. C. Revisão pelos pares: do tradicional ao inovador. In: CONFERÊNCIA IBEROAMERICANA DE PUBLICAÇÕES ELETRÔNICAS NO CONTEXTO DA COMUNICAÇÃO CIENTÍFICA,1., 2006, Brasília, DF. Anais [...]. [S.l.]: [s.n.], 2006. Disponível em: https://bit.ly/3xw9v0a Acesso em: 20 dez. 2022.
  » https://bit.ly/3xw9v0a

• FINANCING

  Not applicable.

• CONSENT TO USE IMAGE

  Not applicable

• RESEARCH ETHICS COMMITTEE APPROVAL

  not applicable

• 1
  Available at: https://doi.org/10.6084/m9.figshare.23264150.v1
• 2
  Available at: https://doi.org/10.6084/m9.figshare.23264078.v2
• 3
  Publons is a free resource for the global multidisciplinary academic research community, powered by Clarivate Analytics. Publons links directly to the Web of Science (WoS), allowing researchers to claim and manage their publication history in WoS and display their history alongside their peer review and journal editing history (CLARIVATE, 2023).
• 4
  ReviewerCredits is a system for obtaining academic recognition and rewarding peer review efforts. ReviewerCredits allows you to redeem credits for publication discounts, editorial and translation services, subscriptions, training courses, conference attendance and much more (REVIEWERCREDITS, 2023).

• LICENSE OF USE

  The authors grant Encontros Bibli exclusive rights of first publication, with the work simultaneously licensed under the Licença Creative Commons Attribution (CC BY) 4.0 International. This license allows third parties to remix, adapt and create from the published work, giving due credit for authorship and initial publication in this journal. Authors are authorized to enter into additional contracts separately, for non-exclusive distribution of the version of the work published in this journal (e.g. publishing in an institutional repository, on a personal website, publishing a translation, or as a book chapter), with acknowledgement of authorship and initial publication in this journal.

• PUBLISHER

  Federal University of Santa Catarina. Postgraduate Program in Information Science. Publication in the Portal de Periódicos UFSC. The ideas expressed in this article are those of the authors and do not necessarily represent the opinion of the editors or the university.

Edited by

EDITORS

Edgar Bisset Alvarez, Ana Clara Cândido, Patrícia Neubert, Genilson Geraldo, Mayara Madeira Trevisol, Jônatas Edison da Silva, Camila Letícia Melo Furtado and Beatriz Tarré Alonso.

Publication Dates

• Publication in this collection
  20 Oct 2023
• Date of issue
  2023

History

• Received
  19 Feb 2023
• Accepted
  13 July 2023
• Published
  04 Aug 2023