Agronomy researchers and research scholars in Brazil: Gender, scientific age, scientific production and impact, and training of human resources

ABSTRACT

Transparency of evaluation criteria and monitoring recommendations for research grants require careful judgment and frequent reassessment of guiding parameters. The aim of this study is to inform the scientific community and funding agencies about the applicants profile for research productivity grants (PQ) in the field of Agronomy of the National Council for Scientific and Technological Development (CNPq), and to contribute to the analysis of grant distribution using the grant applicants of the 2018 call as a study sample. The data registered in the Lattes Curriculum platform were used to quantify the scientific production index. This index considers the number of published articles with different weights for the different segments of the journal’s impact factor values, in addition to the number of patents and books, and human resources training index that considers the number of supervisions and their level (scientific initiation, master’s, doctorate and post-doctorate) completed as principal advisor. The H index (ISI-Web of Science base and Scopus base), scientific age (equal to the number of years after doctorate thesis defense), and the m index (H index divided by scientific age) were also considered, as well as the gender of the fellows. The results show that more than three quarters (75.8 %) of Agronomy PQ fellows are male. At the Category 1 levels and on the Agronomy Committee itself, the relative participation of female researchers is even lower. Women are more involved in human resource training, publish more in non-JCR journals, and are older (scientific age) at the lower level of fellow and among candidates, while men have greater scientific production, H and m indices, and m increase as scientific age advances. The indices of scientific production and human resources training, and national/international insertion (H index) are not homogeneous within the same level/category, despite the search for more transparent and verifiable evaluation indices/indicators. There are fewer opportunities for success and advancement for women, which characterizes a space for achieving gender equity. Objective, easily calculated indices/indicators are absolutely necessary for a large number of researchers, while the critical evaluation and the search for more such parameters must move forward and be compatible with the peer-review process. We suggest that CNPq collect data on citations to Google Scholar and, in particular, share information on gender and interruptions due to illness, maternity and paternity, and care for the sick, elderly or dependent.

agro/soil scientometrics; research productivity grant; scientific quality; science evaluation; female scientists

INTRODUCTION

The evaluation of research and fellowship proposals, submitted to the Brazilian Council for Scientific and Technological Development (CNPq), is perhaps the most important activity carried out by the advisory committees (CA) of the aforementioned Council. These committees are formed by researchers who are recognized in the scientific community for their contribution to science and student advising. When evaluating proposals for the Research Grant (PQ) in Agronomy, the CA-Agronomy (CA-AG) analyzes the merits of the proposal, supported by ad hoc evaluations from the scientific community that assesses the research project. The evaluation is based on the requirements established by the CNPq and the qualification criteria defined by the committee, always in comparison with scientific peers. As the fellowship is divided into ascending levels: 2, 1D, 1C, 1B and 1A, the fellows are divided into levels. Each level provides an ascending salary supplement (“scholarship”), and level-1 fellows receive a bench fee to conduct scientific research.

In addition to the research project, numerical productivity indices from the researchers’ Curriculum Lattes (CNPq, 2021) are also considered in the evaluation. These indices include the scientific production index, which includes the impact factor (JCR - Journal of Citation Records) of scientific journals, the index of resource formation of individuals at different levels, from scientific initiation to postdoc, and the national and international engagement, which includes the H index (CA-AG, 2018).

The quality of scientific articles needs to be assessed to determine their contribution to science and help funders make their decisions (Simons, 2008). However, the quality of the journals in which the researcher publishes is not synonymous with the quality of the researcher. Adler (2009) mentions several reasons for this disparity, such as the impact factor, which is determined by a small number of manuscripts and researchers, and the frequency of citations, which does not necessarily correspond to quality. The impact factor of a journal provides information about the journal’s overall impact, but not about the impact of a particular published article, unless it is more difficult to get an article published in that journal and it is, therefore, considered better (Hartemink, 2009). In addition, the JCR impact factor can be manipulated by the journal (Simons, 2008; Adler, 2009) or as a strategy by the researchers themselves (Katchburian, 2008; Pinto and Andrade, 1999). A dramatic example is the journal Land Degradation & Development. After the journal was accused of citation irregularities, with a more than double increase in the impact factor from the years 2014 to 2015, its editor was asked to resign (McCook, 2017). Therefore, the JCR should be used sparingly in evaluating researchers’ scientific production, even though it is considered one of the best tools to measure publication quality (Garfield, 2006).

The H index would have advantages over productivity indicators (Hirsch, 2005), but not without reservations from proponents and others (e.g., Hartemink, 2009), because it makes comparison across fields difficult (Kelly and Jennions, 2006) and the actual contribution of each author is unknown for articles with multiple authors (Hartemink, 2009). This can inflate the index with frequently cited articles, and ultimately not represent the researcher’s career if she/he is not the primary author (Hirsch, 2005). The noted weaknesses of the H index have not diminished its popularity; on the contrary, its use and the literature on the subject are in full expansion (Leydesdorff et al., 2016).

The m index, which corresponds to H/t_1p, was also proposed by Hirsch (2005), where t_1p is the time elapsed since the publication of the first scientific article, and the author suggests that m is an indicator of the success of the researcher’s scientific career. However, the indices H and m can be inflated by self-citation of articles, which represents 12 % in soil science, while the number of citations varies considerably between databases (Web of Science, Scopus and Google Scholar), especially for journals of national scientific societies (Minasny et al., 2013).

Regional and gender distributions are important aspects when minimizing differences in the access and advancement of Brazilian scientists in their scientific careers, although these distributions are not part of the criteria adopted by CA-AG. In the field of Agronomy, there are not yet published data on these issues, but Capes (Coordination for the Improvement of Higher Education Personnel) data for 2017, show that women were 38 % of productive scientists at the lowest career level (PQ-2) and only 23 % at the highest career level (PQ-1A) (Barros and Silva, 2019). These results show that women’s opportunities for success and career advancement are reduced and disadvantaged (Rodrigues and Guimarães, 2016), with tentative improvements over time (Weber et al., 2015). Level 1 of the productivity stipend includes access not only to material goods (the bench fee that accompanies the stipend, as well as access to specific funding calls), but also to symbolic goods, such as the opportunity to serve on CNPq advisory committees (Wainer and Vieira, 2013; Oliveira et al., 2021). Analyzing international gender equity in soil science, Dawson et al. (2021) emphasize that it is critical that individual countries conduct detailed studies of gender equity so that we have a more complete picture of the extent of gender issues nationally and internationally.

Since these grants are highly competitive and of prestigious nature, the scientific community needs to know not only the general evaluation criteria, which are public (CNPq, 2018), but also the profile of the peers (grantees and potential grantees). Transparency of the evaluation criteria and tracking of scholarship recommendations require a critical look and frequent re-evaluation of the distinction parameters between non-scholars and scholars, as well as PQ scholarship categories/levels. Therefore, the aim of this study is to inform the scientific community and development agencies about the profile of Agronomy fellows and contribute to the analysis of the distribution of fellowships using the applicants of the 2018 CNPq call as a sample.

MATERIALS AND METHODS

The universe of researchers considered in this study were all PQ grant applicants in the Agronomy Committee (CA-AG), in the 2018 CNPq call. The CA-AG includes the subfields of Agrometeorology, Soil Science, Rural Extension, Plant Protection, and Crop Science. The scientific production data registered in Lattes Curriculum were provided to the committee by CNPq, and faithfully reflect the recorded data. All data, which are also publicly available, cover the last five and ten years, but only data from the last five years were used in this study, while the curriculum is frozen at the application deadline. Applicants comprise both researchers who have been fellows and are seeking renewal of the fellowship, and non-fellows seeking the fellowship.

Researcher productivity is assessed using indices of scientific production and human resources training as well as the merit of the research project being developed. Contribution to innovation in science and technology, coordination or participation in relevant scientific research projects, participation in editorial activities (scientific journals and books) and scientific and academic management will also be considered. To determine the scientific production index, the number of published articles and their quality are taken into account, with “quality” measured by the JCR impact factor of the respective scientific journals.

Scientific production takes into account the number of articles published, with different weights for each JCR segment, as well as the number of patents and the number of books published. The human resources training index takes into account the number of supervised students (scientific initiation, master’s, doctoral, and postdoctoral) as the main advisor. For the evaluation of the research project, the evaluations carried out by ad hoc evaluators are taken into account, according to the requirements in the call for PQ grants.

The scientific production index is calculated by adding the number of articles per class of JCR impact factor, patents and books multiplied by their respective weights: Articles with JCR >1.5 (2.5), JCR between 1.1 - 1.50 (1.5), JCR between 0.51 - 1.0 (1.0), JCR between 0.1 - 0.5 (0.5), JCR between 0 - 0.5 (0.15) and without JCR (0.15); number of patents (2.5); and published books (2.5). The human resources training index is calculated by summing the number of students in each training class multiplied by their respective weights, i.e., scientific initiation (0.2), master’s (1.0), doctoral (2.0) and post-doctoral (1.0).

In addition to the above indices used by the committee, we also considered the H index (ISI-Web of Science base and Scopus base), the scientific age, and the m index. Hirsch (2005) defines scientific age as the number of years since the publication of the first article in a journal indexed in the ISI Web of Science, and the m index as H index/scientific age in years. However, in this study, we use scientific age as the equivalent of the number of years after doctorate thesis defense (t_Dr), since this information is available from CNPq.

RESULTS AND DISCUSSION

Male and female researcher scholars: Is agricultural science a male-dominated science?

Regardless of gender, of the total 488 researchers who applied for a grant in Agronomy in 2018, only 47 % had the opportunity to receive the grant, i.e., more than half of the researchers did not have access to research productivity incentives because of CNPq’s limited financial resources. Out of the total 231 fellows, the majority (160) remained at the same level, 8 were downgraded and 43 were upgraded, while 25 did not have the opportunity to renew the fellowship (CA-AG, 2018). Due to non-renewal based on merit or lack of renewal applications from fellows, a significant number of new PQ fellowships were approved. When there is no increase in quotas, the addition of new fellows only occurs if previous fellows leave or are not renewed, increasing the pressure on the PQ system as more researchers meet the minimum requirements to access to the productivity fellowship. Data in table 1 show that the ranking order, 1A = 1B < 1C < 1D <<< 2 <<< non-scholars, corresponds to a very sharp pyramid of difficult progression for researchers.

Table 1
Number and percentage of fellowship holders who requested renewal or new fellowship in CNPq Call 2018, classified by gender and class of fellowship PQ

Fellowship category/level	Female		Male		Total
	Number	%	Number	%	Number	%
1A	0	0	12	100	12	2
1B	2	17	10	83	12	2
1C	2	12	15	88	17	3
1D	5	17	25	83	30	6
2	47	29	113	71	160	33
Non-fellows (SB)	76	30	181	70	257	53
Total applicants (SB + fellows)	132	27	356	73	488	100

Among fellows and non-fellows, women represent 27 % of the total and hold 23 % of fellowships (Table 1). Relatively, women have less access to the PQ fellowship, with a difference of less than 4 percentage points. However, analysis of the distribution of PQ-1 fellowships reveals a clear disadvantage for women, as of the total number of female fellows, 16 % are PQ-1 and none are PQ-1A, while 35 % of male fellows are PQ-1, including 12 PQ-1A researchers. These gender-specific results are worse than those in other fields of knowledge. In Chemistry, for example, an analysis of scholarship by gender in 2009 shows that 32.8 % of scholarship was held by women, mainly PQ-2 fellowships (40.2 %) and only 4.8 % of PQ-1A fellowship holders. For all major areas of knowledge in 2008, women held 33.8 % of the fellowships, with 23.6 % of PQ-1A (Santos et al., 2010). Among the fourteen members of the CA-AG in 2018, three were women, further indicating a clear male predominance (79 %).

Despite the growing participation of women in science and technology activities, chances of success and career advancement are reduced and at a disadvantage in a system predominantly controlled by men (Leta, 2003; Rodrigues and Guimarães, 2016; Barros and Mourão, 2018). According to data from Capes (Coordination for the Improvement of Higher Education Personnel) for 2017, men correspond for 62 % of productivity fellowship holders at the first career level (PQ-2), but reach 77 % at the last career level (PQ-1A) (Barros and Silva, 2019). Thus, with the advancing scientific age, the number of women in the fellowship system decreases (Aguinis et al., 2018). In a ranking of 160 countries using the Gender Inequality Index - an indicator that includes education and labor - Brazil occupies the 94th position (PNUD, 2015). Interestingly, women winners of important scientific awards have lower marriage rates, and tend to have fewer children than scientific-distinguished men (Charyton et al., 2011).

The timid improvement and the expectation of reduction in gender disparities in science over time (Barros and Mourão, 2020) does not prevent women’s difficulty in joining the CNPq fellowship system and in progressing in their careers (Weber et al., 2015). Women graduate later and take longer to enter the fellowship system (Barros and Silva, 2019). These authors suggest that, although fellowship evaluation criteria are clear and academically justifiable, the disparity in gender distribution calls to a broader discussion of the mechanisms for progressing in the scientific career.

Diversity in research is paramount, and affirmative actions should be implemented to include groups traditionally under-represented or even excluded. One possibility is by considering a curriculum summa, where each researcher describes relevant issues besides her/his scientific trajectory, including interruption resulting from illness, maternity and fatherhood, and care for ill, elder or special-needs person (Fapesp, 2021), and how these issues affect her/his scientific performance and professional carrier.

During the COVID-19 pandemics, women’s publication rates decreased in all disciplines compared with men’s rates. Women have consistently authored about 20 % of working papers on pandemic-related research questions since 2015, but account for only 12 % of authors of new COVID-19 research (Viglione, 2020). Even in households with two academics, female academics are shown to do more household chores and this may also be true for childcare. In addition to their childcare responsibilities, early- and mid-career women researchers may be more averse to undertake risk and, therefore, less likely to begin studies in a new field (Viglione, 2020).

Research productivity fellowships (PQ) in Agronomy: is there equality/equity fairness in distribution?

The regional distribution of research fellows (Table 2) shows that most of the fellows come from the Southeast, especially at the PQ-1 level. This region holds 61 % of the most prestigious fellowships, while all other regions hold only 39 %. The ratio of the number of fellowship holders and the total of applicant researchers is higher in the South (0.57) and Southeast (0.52) regions than the national average of 0.47, while the other regions have a lower value than the national one, with the highlight for the Midwest region, with a value of only 0.21.

Table 2
Regional distribution of researchers, by fellowship level and number of fellowship holders/total number of researchers in CNPq Call 2018

Fellowship category/level	Regional distribution
	Mid-West	North	North-East	South	South-East	Total
1A	1	0	2	3	6	12
1B	0	0	0	1	11	12
1C	1	1	1	1	13	17
1D	2	2	5	8	13	30
2	9	11	28	51	61	160
Non-fellows (SB)	48	15	49	48	97	257
Total applicants (SB + fellows)	61	29	85	112	201	488
Fellows/total applicants	0.21	0.48	0.42	0.57	0.52	0.47
PQ-1/PQ-2	0.44	0.27	0.29	0.25	0.70	0.44

For levels 1 and 2 of the PQ fellowships (Table 2), the national average is 0.44. The Southeast region has a high concentration of PQ-1 fellowship holders (0.70), while the South region has the lowest ratio (0.25). These two extremes, far from the national average, demonstrate an imbalance in the distribution of fellowship among levels in Brazilian regions. For regions with a lower number of grantees, these results are very dynamic and depend on the evaluation year. For example, data from the 2019 Call (not shown) show that the Midwest region almost doubled this ratio, while the North region received no PQ-1A fellowship.

Scientific age of fellows and non-fellows

The average number of years after doctorate thesis defense (t_Dr; Figure 1a) of PQ-2 grantees is 12, while for PQ-1 grantees t_Dr is 21. The PQ-1A grantees have an average scientific age of 26 years, with the shortest time equivalent to 20 years. The fellowship candidates completed their doctorate, on average, 12 years before applying for the fellowships. The standard deviation, ranged from 5 to 9 years, for all classes of fellows and fellowship candidates, resulting in high coefficients of variation (26 to 58 %), with the highest value for the aspirant groups.

Figure 1
Scientific age (number of years after completion of the doctoral degree), by class of PQ fellowship and non-fellows (SB), for all researchers, regardless of gender (a) and by gender women (b) and men (c), in CNPq Call 2018. Vertical bars depict standard deviation.

Broken down by gender, women are at a higher scientific age for at lower fellowship levels (PQ-2, PQ-1D and PQ-1C) and without fellowship, but this behavior reverses at the PQ-1B level, where men have a higher scientific age (Figures 1b and 1c).

H and m indices of researchers

Most researchers (86 %) have a Web of Science H index (H-WoS) (Figure 2a) less than or equal to 10. This group includes virtually all researchers with less than 10 years since their doctoral degree, many with 20 to 30 years of t_Dr and also some with more than 30 years since their doctorate. The group with H-WoS >15 obtained their doctorate more than 10 years ago, and the highest H-WoS of 24 is at a scientific age of 30 years. It is worth mentioning that three researchers have H-WoS index of zero, one of them is a senior researcher (doctorate completed 44 years ago). On the other hand, there are some young researchers, whose doctorate was completed 5 years ago or less and whose H-WoS is equal to or greater than 5.

Figure 2
H-index of the Web of Science (H-WoS) of researchers, as a function of the number of years after the end of the doctoral degree, for all researchers, regardless of gender (a) and by gender women (b) and men (c), in CNPq Call 2018.

These results show that there is great pressure on category 2 researchers to move up to category 1, and that there are differences between the typical H indices of the different subfields of Agronomy. Despite the large scatter in the data, the relationship between H-WoS and t_Dr was significant (R² = 0.74) (Figure 2a). The equation shows that, for each year after the doctorate thesis defense, Agronomy researcher’s H-WoS of Agronomy researchers should increase by 0.41. When analyzed by gender, the maximum H-WoS for females (H = 16) is almost half of males (H = 30), with many male researchers having an H index greater than 16 (Figures 2b and 1c). The increase in H index with increasing scientific age is 1.47 times greater for men (0.45) than for women (0.30).

The mean H-WoS index (Figure 3a) of PQ-2 scholars is 7, while it is 10 for PQ-1D scholars and 18 for PQ-1A, while scholarships candidates have a mean H-WoS of 5. The standard deviation for the H-WoS index ranged from 3 to 5 for all scholarship classes and scholarship applicants, resulting in coefficients of variation of 67 % for the applicant group of and 15 % for PQ-1 scholarship applicants (Figure 3). Across all groups (scholarship and non-scholarship applicants), women have lower average H scores than men (Figures 3b and 3c).

Figure 3
Web of Science H-Index (H-WoS), by class of PQ fellowship and non-fellows (SB), for all researchers regardless of gender (a) and by gender women (b) and men (c), in CNPq Call 2018. Vertical bars depict standard deviation.

The H-index does not account for the role of individual authors in papers with multiple authors, where a researcher can have a high H value but without being the first author of any article (Hartemink, 2009). In addition, the H index, and therefore the m index can be inflated by self-citations of articles. Although there are no published data for the field of Agronomy as a whole, self-citations correspond to 12 %, but there are differences among subdisciplines in soil science journals, where the H index without self-citations corresponds to 0.88 × H index (R² = 0.97) (Minasny et al., 2013), with a weak relationship between the percentage of self-citations and scientific age (Minasny et al., 2010).

The relationship between Scopus H (H-Sco) and H-WoS scores is close (Figure 4a), with a coefficient of determination of 0.91, but H-Sco scores are higher (angular coefficient of 1.046) than H-WoS. In only five cases of the total number of researchers was the H-Sco score lower than that of H-WoS, and in only one case was the former much higher (more than twice as high) than the later. Thus, both bases can be used to evaluate the influence of the researcher’s scientific publications. When analyzed by gender, the angular and coefficients of determination are slightly lower for women than for men (Figures 4b and 4c).

Figure 4
Relationship between H indices from Scopus (H-Sco) and Web of Science (H-WoS), for researchers in Agronomy, for all researchers regardless of sex (a) and by gender women (b) and men (c), in CNPq Call 2018.

As far as we know, there is no study that assesses publication characteristics in Agronomy, while there is one article for Soil Science that is very representative of the universe of soil scientists, published by Minasny et al. (2013). For 340 soil scientists, the number of articles in Google Scholar (GS) was, on average, 2.3 times higher, and the number of citations 1.9 times higher than for the data in WoS. Scopus metrics were slightly superior to WoS. The H index in GS was, on average, 1.4 times greater than that of WoS, and the H index in Scopus was, on average, 1.1 times greater than that of WoS. Over time, metrics increase in all three databases, but more rapidly in GS (Minasny et al., 2013).

Although GS contains more “gray” literature (informally published material) and citations may contain errors (Harzing and van der Wal, 2009), the H index appears to be quite robust and comparable to WoS and Scopus, with excellent correlation for H indices (Minasny et al., 2013). Another interesting aspect is that four soil science journals published by national soil science societies rank significantly higher in Google Scholar than the ISI impact factor (≥4 rank difference). This group includes the Brazilian Journal of Soil Science, which ranks 12th in GS and 25th in JCR, because Google Scholar contains more citations of non-English language articles (Minasny et al., 2013). This observation calls for a closer look at the importance of Brazilian journals in the publishing behavior of the scientific community and the impact on the evaluation of scientist productivity. However, such an assessment is only possible if CNPq starts collecting data from GS as well, not only from WoS and Scopus.

The m index (Figure 5a) calculated as the ratio h/scientific age in years, where scientific age corresponds to the number of years after completing the doctorate thesis defense (t_Dr), ranged from 0 to 5, with a mean of 0.52. By scholarship class, the mean is 0.72 for PQ-1A and 0.52 for non-scholars, with a high standard deviation across all scholarship classes. Analysis by gender shows that women always have a lower m index than men (Figures 5b and 5c). An index of m~1 indicates a successful researcher, m~2 are unusual researchers found only at the best universities, and m~3 or higher indicates individuals with outstanding publications (Hirsch, 2005).

Figure 5
Index m calculated as the H-WoS index/scientific age ratio in years, in which the scientific age is equivalent to the number of years after doctorate thesis defense, for all researchers regardless of sex (a) and by gender women (b) and men (c), in CNPq Call 2018. Vertical bars depict standard deviation.

Thus, assuming that t_1p does not deviate much from t_Dr, the group of rated researchers is usually smaller than 1, especially considering that agricultural science is not a top science. In addition, the values for the m index described by Hirsch take into account the mean values of the H index observed in developed countries where the number of publications is higher than in Brazil (Hermes-Lima et al., 2007a, b), which raises the values of the H and m indices to atypical values for developing countries. Despite these difficulties, there are some outstanding researchers, regardless of the level of scholarship, i.e., there are also young researchers with a high m index.

In Soil Science, Minasny et al. (2007) found for a group of 228 soil scientists, that the H index of a soil scientist’s is about 0.7 times the number of years since her/his first publication. This relationship also holds for a larger group of 340 international soil scientists (Minasny et al., 2013): H index = 0.73 × scientific age (R² = 0.72). The mean m of 0.73 is within the values of 0.7 for Scopus and 0.8 for Google Scholar (Minasny et al., 2013), as well as the disciplines of water (0.47) and biochemistry (0.83) (McCarty and Jawitz, 2013). Among the various soil science disciplines, soil biology, biogeochemistry and ecology have the highest m values (0.8), while pedology has the lowest (0.5).

Scientific production index

The scientific production index (Figure 6a), which includes scientific articles, books and patents, shows that the range from 0 to 30 consists mainly of young researchers with less than 10 years of t_Dr, but also of senior(older) researchers with more 20 years of t_Dr. The highest scores (over 120) for this index correspond to middle-aged researchers (12-24 years of t_Dr).

Figure 6
Scientific production index in the last five years (January 2013 to October 2018), as a function of the number of years after doctorate thesis defense, for all researchers regardless of gender (a) and by gender women (b) and men (c), in CNPq Call 2018.

There is a direct relationship between the index of scientific production and t_Dr, with a large dispersion of scores and a low (0.56) but significant R² (Figure 6a). There is also a difference between genders: as scientific age increases, the scientific production rate increases 1.4 times more for men than for women (Figures 6b and 1c). Nonetheless, this equation should not be used to estimate productivity because of data scatter dispersion, the gender problem issue, and the lower scientific production of researchers with 35 years or more of t_Dr. The average index of scholar production is 33 for PQ-2, 47 for PQ-1D, 83 for PQ-1A, and 23 for fellows, with standard deviations ranging from 17 to 43, for all fellowship levels/categories (Figure 7). Although Category 1 fellows generally published more articles than Category 2 fellows, there are exceptional cases in all levels/categories. In addition, the index of scientific production is lower for women than for men, in all categories of fellows and non-fellows, with a larger difference for PQ-1B grantees (Figures 7b and 7c).

Figure 7
Scientific production index in the last five years (January 2013 to October 2018), by PQ fellowship class and non-fellows (SB), for all researchers regardless of sex (a) and by gender, women (b) and men ( c), in CNPq Call 2018.

In the 2019 call (data not shown), the average JCR for a researcher to be eligible to apply for a fellowship would be 1.9 over the five-year period. This means that, someone publishing only in Brazilian agronomic journals (Soil Science) would likely not receive a PQ grant. The study by Wainer and Vieira (2013) using data from the large field of agricultural sciences, which includes Agronomy and six other subfields (or CAs), showed a preference for rewarding scholars at the beginning of their scientific career (levels 2 and 1D) with the highest total number of citations in WoS, moving to an incentive direction at the next higher levels that takes into account recent publications in WoS. This policy allows for entry into the grant system (levels 2 and 1D) while considering the different paces of early career researchers.

The number of articles published in journals without JCR (Figure 8a) is high regardless of the level/category of scholarship. Values range from 28 % (PQ-1A) to 42 % (PQ-1B), and the overall average is 35 %. By comparison, in the 2019 call (data not shown), these values ranged from 20 % (PQ-1C) to 35 % (PQ-2), and the overall average was 30. Compared to men, women published more in journals without JCR (Figures 8b and 8c). Although these percentages are high, they are lower than the 55 % found by Minasny et al. (2013) for international soil scientists, who publish an average of 5.5 articles per year and 2.5 such articles in referenced international journals. Even though the relative value of articles without JCR is low, the high number of articles significantly affects the value of the scientific production index. The high standard deviation and the resulting high coefficients of variation are due to the differences between the various subfields of Agronomy. For example, the average values for production in soil science are higher than for the entire field of Agronomy, namely 37, 75 and 122 for the grantees of PQ-2, PQ-1D and PQ-1A, respectively. Journals from different fields and sub-fields have characteristic impact factors and, thus, a comparison of researchers regarding this criterion must take this aspect into account.

Figure 8
Percentage of articles published in journals without JCR in relation to the total of articles published in the last five years (January 2013 to October 2018), by PQ fellowship class and non-fellows (SB), for all researchers regardless of gender (a) and by gender women (b) and men (c), in CNPq Call 2018.

The impact factor calculated for individual journals should not be used to judge the significance of an individual researcher past performance or scientific potential (Adler, 2009). There are no numerical shortcuts to assessing research quality. What matters is the quality of a scientist’s work, regardless of where it is published and ultimately judged by peers (Simons, 2008; Adler, 2009), and the intrinsic value of the work must be considered in research funding decisions (Wellcome Trust, 2008). Thus, if, on the one hand, objective and easily calculated indicators/indices are needed for a large number of researchers, as in the present case of the PQ call, on the other hand, the critical evaluation of these indicators and the search for others must be consistent with peer review. Moreover, the evaluation must not focus too much on scientific production, otherwise Brazilian Agronomy will succumb to the “publish or perish” motto.

Of the total number of researchers, 89 of them (18 %) have at least one registered patent (data not shown) within a 5-year period. Of these, 48 researchers have 1 to 2 registered patents and nine researchers apply for 10 or more patents, mostly in plant breeding science. The highest number of patents for a researcher is 74. There is no correlation between the number of patents and the amount of the PQ grant, nor with being a grantee or not.

Human resources training index

Researchers between the ages of 13 and 22 have the highest percentage (over 45) of human resources training (Figure 9a) and devote a significant amount of their time to advising students, regardless of the category/level of scholarship. This index is directly related to the time spent postdoctoral defense, but with a high dispersion of points and R² of 0.64 (Figure 9a), and it depends on the researcher’s gender (Figures 9a and 9b). Therefore, because of the scatter of the data, gender issues, and lower human resource training among researchers who have completed 30 years or more of doctorate, this equation should not be used a priori to estimate the index. In the Chemistry Committee, most researchers are more involved in human resources training between 10 and 30 years after their doctorate (Santos et al., 2009), similar to the observed in our study. After this period, the decline in training activities could be related to retirement, withdrawal from student advising (Santos et al., 2009), or the reduced focus of the older generation of researchers on scientific production.

Figure 9
Human resources training index in the last five years (January 2013 to October 2018), as a function of the number of years after the end of the doctoral course, for all researchers, regardless of gender (a) and by gender women (b ) and men (c), in CNPq Call 2018.

The average human resources training index (Figure 10a) is 15 for PQ-2, 18 for PQ-1D, 29 for PQ-1A, and 8 for non-fellows. The standard deviation of the H-WoS index ranged from 7 to 11 for all classes of fellowship and applicants (Figure 10a), with high coefficients of variation of up to 86 % for the fellowship applicant group. Women have a higher training index than men, especially the PQ-1B and PQ-1C fellows.

Figure 10
Human resources training index in the last five years (January 2013 to October 2018), by class of PQ fellowship and non-fellows (SB), for all researchers regardless of gender (a) and by gender women (b) and men (c), in CNPq Call 2018.

CONCLUSION

When analyzing the profile of CNPq Agronomy research fellowship applicants, it appears that less than a quarter of Agronomy research fellows are women. At the upper level of grants, the relative proportion of women researchers is even lower. Women work more in training human resources, publish more in journals without JCR, and have higher scientific age in the lower-grants and fellowship candidate groups, while men have higher scientific production, H and m indices, and these indices increase with increasing scientific age. Despite the search for more transparent and verifiable evaluation indicators, the indices of scientific production, human resources training and national/international insertion (H index) are not homogeneous within the same level/category apart from the obvious gender issue. Moreover, there are regional differences in access to fellowships and their distribution among the different classification levels.

The lower access and career advancement of women is a gap that needs to be addressed to achieve gender equity. We propose that CNPq share information on gender and interruptions due to illness, maternity, and care of the sick, elderly or dependent, to improve gender equity. In addition, objective, easy-to-calculate indicators/indices are essential for many researchers, and the critical evaluation and the search for other indicators/indices must move forward and be consistent with peer review. This is especially important given the pressures on grants, where more than half of qualified applicants do not become CNPq research fellows. A further look at citations is possible when CNPq begins to collect data from Google Scholar.

ACKNOWLEDGMENTS

We thank CNPq for the research productivity fellowships for the first two authors and for the opportunity to analyze the data presented in this article, Capes – Finance code 001, and professor Rômulo Simões Cézar Menezes for reviewing the manuscript.

REFERENCES

• Adler KB. Impact factor and its role in academic promotion. Am J Respir Cell Mol Biol. 2009;41:127-8. https://doi.org/10.1165/rcmb.R09ED1
  » https://doi.org/10.1165/rcmb.R09ED1
• Aguinis H, Ji YH, Joo H. Gender productivity gap among star performers in STEM and other scientific fields. J Appl Psychol. 2018;103:1283-306. https://doi.org/10.1037/apl0000331
  » https://doi.org/10.1037/apl0000331
• Barros SCV, Mourão L. Gender and science: An analysis of Brazilian postgraduation. Estud Psicol. 2020;37:e180108. https://doi.org/10.1590/1982-0275202037e180108
  » https://doi.org/10.1590/1982-0275202037e180108
• Barros SCV, Mourão L. Panorama da participação feminina na educação superior, no mercado de trabalho e na sociedade. Psicol Soc. 2018;30:e174090. https://doi.org/10.1590/1807-0310/2018v30174090
  » https://doi.org/10.1590/1807-0310/2018v30174090
• Barros SCV, Silva LMC. Desenvolvimento na carreira de bolsistas produtividade: Uma análise de gênero. Arq Bras Psicol. 2019;71:68-83. https://doi.org/10.36482/1809-5267.ARBP2019v71i2p.68-83
  » https://doi.org/10.36482/1809-5267.ARBP2019v71i2p.68-83
• Charyton C, Elliott JO, Rahman M, Woodard JL, Dedios S. Gender and science: Women nobel laureates. J Creat Behav. 2011;45:203-14. https://doi.org/10.1002/j.2162-6057.2011.tb01427.x
  » https://doi.org/10.1002/j.2162-6057.2011.tb01427.x
• Comitê Assessor de Agronomia - CA-AG. Relatório de julgamento das Bolsas de Produtividade em Pesquisa (Chamada CNPq Nº 09/2018) e Bolsas Especiais. Brasília, DF: CNPq; 2018.
• Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq. Anexo I - Critérios dos comitês de assessoramento. Vigência 2018 a 2020. Brasília, DF: CNPq; 2018.
• Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq. Plataforma Lattes. Brasília, DF: CNPq; 2021 [cited 2021 Aug 18]. Available from: https://lattes.cnpq.br/
  » https://lattes.cnpq.br/
• Dawson L, Brevik EC, Reyes-Sánchez LB. International gender equity in soil science. Eur J Soil Sci. 2021;72:1929-39. https://doi.org/10.1111/ejss.13118
  » https://doi.org/10.1111/ejss.13118
• Fundação de Amparo à Pesquisa do Estado de São Paulo - Fapesp. Financiamento à pesquisa: Roteiro para elaboração da Súmula Curricular. São Paulo: Fapesp; 2021 [cited 2021 Aug 18]. Available from: https://fapesp.br/sumula
  » https://fapesp.br/sumula
• Garfield E. The history and meaning of the Journal Impact Factor. J Am Med Assoc. 2006;295:90-3. https://doi.org/10.1001/jama.295.1.90
  » https://doi.org/10.1001/jama.295.1.90
• Hartemink AE. Publications for evaluations: The impact of soil science and soil scientists. J Soil Water Conserv. 2009;64:18-9. https://doi.org/10.2489/jswc.64.1.18A
  » https://doi.org/10.2489/jswc.64.1.18A
• Harzing AW, van der Wal R. A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. J Am Soc Inf Sci Technol. 2009;60:41-6. https://doi.org/10.1002/asi.20953
  » https://doi.org/10.1002/asi.20953
• Hermes-Lima M, Alencastro ACR, Santos NCF, Navas CA, Beleboni RO. The relevance and recognition of Latin American science. Introduction to the fourth issue of CBP-Latin America. Comp Biochem Phys C: Toxicol Pharmacol. 2007a;146:1-9. https://doi.org/10.1016/j.cbpc.2007.05.005
  » https://doi.org/10.1016/j.cbpc.2007.05.005
• Hermes-Lima M, Santos NCF, Alencastro ACR, Ferreira ST. Whither Latin America? Trends and challenges of science in Latin America. IUBMB Life. 2007b;59:199-210. https://doi.org/10.1080/15216540701258751
  » https://doi.org/10.1080/15216540701258751
• Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA. 2005;102:16569-72. https://doi.org/10.1073/pnas.0507655102
  » https://doi.org/10.1073/pnas.0507655102
• Katchburian E. Publish or perish: A provocation. Sao Paulo Med J. 2008;126:200-3. https://doi.org/10.1590/S1516-31802008000300013
  » https://doi.org/10.1590/S1516-31802008000300013
• Kelly CD, Jennions MD. The h index and career assessment by numbers. Trends Ecol Evol. 2006;21:167-70. https://doi.org/10.1016/j.tree.2006.01.005
  » https://doi.org/10.1016/j.tree.2006.01.005
• Leta J. As mulheres na ciência brasileira: Crescimento, contrastes e um perfil de sucesso. Estud Avançados. 2003;17:271-84. https://doi.org/10.1590/S0103-40142003000300016
  » https://doi.org/10.1590/S0103-40142003000300016
• Leydesdorff L, Wouters P, Bornmann L. Professional and citizen bibliometrics: Complementarities and ambivalences in the development and use of indicators – A state-of-the-art report. Scientometrics. 2016;109:2129-50. https://doi.org/10.1007/s11192-016-2150-8
  » https://doi.org/10.1007/s11192-016-2150-8
• McCarty C, Jawitz JW. Attitudes about publishing and normal science advancement. J Informetr. 2013;7:850-8. https://doi.org/10.1016/j.joi.2013.08.001
  » https://doi.org/10.1016/j.joi.2013.08.001
• McCook A. Journal hit by citation scandal named among top in field. Retraction Watch [Internet]; 2017 [cited 2022 Jun 22]. Available at: https://retractionwatch.com/2017/06/19/journal-hit-citation-scandal-named-among-top-field/
  » https://retractionwatch.com/2017/06/19/journal-hit-citation-scandal-named-among-top-field/
• Minasny B, Hartemink AE, McBratney A. Individual, country, and journal self-citation in soil science. Geoderma. 2010;155:434-8. https://doi.org/10.1016/j.geoderma.2009.12.003
  » https://doi.org/10.1016/j.geoderma.2009.12.003
• Minasny B, Hartemink AE, McBratney A. Soil science and the h-index. Scientometrics. 2007;73:257-64. https://doi.org/10.1007/s11192-007-1811-z
  » https://doi.org/10.1007/s11192-007-1811-z
• Minasny B, Hartemink AE, McBratney A, Jang H. Citations and the h index of soil researchers and journals in the Web of Science, Scopus, and Google Scholar. PeerJ. 2013;1:e183. https://doi.org/10.7717/peerj.183
  » https://doi.org/10.7717/peerj.183
• Oliveira A, Melo MF, Rodrigues QB, Pequeno M. Gênero e desigualdade na academia brasileira: Uma análise a partir dos bolsistas de produtividade em pesquisa do CNPq. Configurações. 2021;27:75-93. https://doi.org/10.4000/configuracoes.11979
  » https://doi.org/10.4000/configuracoes.11979
• Pinto AC, Andrade JB. Fator de impacto de revistas científicas: Qual o significado deste parâmetro? Quím Nova. 1999;22:448-53. https://doi.org/10.1590/S0100-40421999000300026
  » https://doi.org/10.1590/S0100-40421999000300026
• Programa das Nações Unidas para o Desenvolvimento - PNUD. Relatório de desenvolvimento humano 2015: O trabalho como motor do desenvolvimento humano. Nova York: PNUD; 2015 [cited 2021 Aug 12]. Available from: http://hdr.undp.org/sites/default/files/hdr15_overview_pt.pdf
  » http://hdr.undp.org/sites/default/files/hdr15_overview_pt.pdf
• Rodrigues JG, Guimarães MCS. A fundação Oswaldro Cruz e a ciência do feminino: A participação feminina na prática e na gestão da pesquisa em uma instituição de ensino e pesquisa. Cad Pagu. 2016;46:197-222. https://doi.org/10.1590/18094449201600460197
  » https://doi.org/10.1590/18094449201600460197
• Santos N, Candido L, Kuppens C. Produtividade em pesquisa do CNPq: análise do perfil dos pesquisadores da química. Quim Nova. 2010;33(2):489-95. https://doi.org/10.1590/S0100-40422010000200044
  » https://doi.org/10.1590/S0100-40422010000200044
• Simons K. The misused impact factor. Science. 2008;322:165. https://doi.org/10.1126/science.1165316
  » https://doi.org/10.1126/science.1165316
• Viglione G. Are women publishing less during the pandemic? Here’s what the data say. Nature. 2020;581:365-6. https://doi.org/10.1038/d41586-020-01294-9
  » https://doi.org/10.1038/d41586-020-01294-9
• Wainer J, Vieira P. Avaliação de bolsas de produtividade em pesquisa do CNPq e medidas bibliométricas: Correlações para todas as grandes áreas. Perspect Cienc Inf. 2013;18:60-78. https://doi.org/10.1590/S1413-99362013000200005
  » https://doi.org/10.1590/S1413-99362013000200005
• Weber JLA, Ramos CC, Mester A, Lindern D, Horlle KR, Souza CS, Pizzinato A, Rocha KB. Perfil dos pesquisadores bolsistas de produtividade científica em Psicologia no Conselho Nacional de Desenvolvimento Científico e Tecnológico. Estud Psicol. 2015;32:1-11. https://doi.org/10.1590/0103-166X2015000100001
  » https://doi.org/10.1590/0103-166X2015000100001
• Wellcome Trust. Position statement in support of open and unrestricted access to published research. London: Wellcome Trust; 2008 [cited 2020 Sep 19]. https://www.southampton.ac.uk/~harnad/Hypermail/Amsci/3031.html
  » https://www.southampton.ac.uk/~harnad/Hypermail/Amsci/3031.html

Edited by

Editors: Reinaldo Bertola Cantarutti and Déborah de Oliveira.

Publication Dates

• Publication in this collection
  22 July 2022
• Date of issue
  2022

History

• Received
  10 Dec 2021
• Accepted
  01 June 2022