Profile and scientific output of researchers recipients of CNPq productivity grant in the field of medicine

SUMMARY

BACKGROUND.

This study aimed to evaluate the scientific production of researchers in the field of Medicine who receive a productivity grant from the CNPq.

METHODS:

The curriculum Lattes of 542 researchers with active grants from 2012 to 2014 were included in the analysis. Grants categories/levels were stratified into three groups according to the CNPq database (1A-B, 1C-D, and 2).

RESULTS.

There was a predominance of grants in category 2. During their academic career, Medicine researchers published 76512 articles, with a median of 119 articles per researcher (IQ, interquartile range, 77 to 174). Among the 76512 articles, 36584 (47.8%) were indexed in the Web of Science (WoS database). Researchers in Medicine were cited 643159 times in the WoS database, with a median of 754 citations (IQ, 356 to 1447). There were significant differences among the categories of grants concerning the number of citations in WoS (P <0.001). There was a significant difference in the number of times researchers were cited according to the specialty included in Medicine area. (P < 0.001).

CONCLUSION.

Strategies to improve the scientific output qualitatively possibly can be enhanced by the knowledge of the profile of researchers in the field of Medicine.

KEYWORDS:
Research Personnel; Work Performance; Medicine

RESUMO

OBJETIVO:

O objetivo deste estudo foi avaliar a produção científica de pesquisadores da área de Medicina que recebem bolsa de produtividade do CNPq.

MÉTODOS:

Os currículos Lattes de 542 pesquisadores com bolsas ativas de 2012 a 2014 foram incluídos na análise. As categorias/níveis das bolsas de produtividade foram estratificadas em três grupos de acordo com a classificação do CNPq (1A-B, 1C-D e 2).

RESULTADOS:

Houve predomínio de bolsas na categoria 2. Durante a carreira acadêmica, pesquisadores da Medicina publicaram 76.512 artigos, com mediana de 119 artigos por pesquisador (Intervalo Interquartil, IQ, 77 a 174). Entre os 76.512 artigos, 36.584 (47,8%) foram indexados no banco de dados da Web of Science (WoS). Pesquisadores em Medicina receberam 643.159 citações no banco de dados de WoS, com uma mediana de 754 citações (IQ, 356 a 1.447). Houve diferenças significativas entre as categorias de bolsas em relação ao número de citações em WoS (P < 0,001). Houve uma diferença significativa no número de citações recebidas pelos pesquisadores de acordo com a especialidade incluída na área de Medicina (P < 0,001).

CONCLUSÃO:

Estratégias para melhorar qualitativamente a produção científica possivelmente podem ser aprimoradas pelo conhecimento do perfil dos pesquisadores no campo da Medicina.

PALAVRAS-CHAVE:
Pesquisadores; Desempenho profissional; Medicina

INTRODUCTION

Many evaluations of tenure promotions and grants take into account appraising metrics in order to assess the performance of individual scientists and eventually rank those researchers². Thanks in part to the easier access to interdisciplinary publication and citation databases (such as Web of Science and Scopus), quantitative measuring of the performance of researchers has become even more prevalent, controversial, and influential^2–6.

Every organization that funds research wants to support science that makes a difference for the community. Therefore, quantifying the performance of individual scholars, departments, and institutions has become an essential part of decision-making in research policy, funding allocations, awarding of grants, faculty hiring, and claims for promotion^7,8. In this context, the rationale of performance evaluation is that scientific budget should flow to individuals and institutions with a qualitatively high scientific output. Nevertheless, there is a difficulty in using measurements to evaluate research since there is no simple formula for identifying truly important research. It is clear that a fair and reliable quantification of the ‘level of excellence’ of individual scientists is a challenging task^9–11. Thus, to measure the performance of a researcher using objective tools has become one of the major challenges in science. Evaluating individual research performance is a complex task that ideally examines productivity, scientific impact, and research quality ¹². At present, the impact of scientific work is traditionally measured by the balance between the number of papers and the number of times that these publications are cited^13–16. However, the evaluation system that has been built upon bibliometric indices is very complex and its results are often inconsistent^17–19.

In Brazil, the National Council for Scientific and Technological Development (CNPq) was created in 1951 and, since then, it has been the most important Brazilian agency supporting science and technology²⁰. The scientific divisions of this entity are categorized into nine major areas subdivided into several subareas of knowledge. Medicine is one of these major areas and is subdivided into diverse subareas of specialization²¹. In the 1970s, the research productivity grant was implemented as a stimulus to researchers with notable scientific contributions in their respective areas²². The area of Medicine had 542 researchers (3.8%) among 14,077 productivity grants in 2014. In order to apply for a grant as a CNPq investigator in Medicine, researchers must fulfill some requirements, and they are ranked according to the criteria established by CNPq. According to the Advisory Committee of CNPq, the criteria for the selection and classification of researchers in Medicine include, among several indicators, scientific production, training of human resources, and contribution to innovation²³. Therefore, in order to be classified as a CNPq investigator in Medicine, the researcher must meet the following minimum requirements over the previous decade: a) to have published at least 20 papers in scientific journals with IF greater than or equal to 1; b) to have completed the orientation of at least one Ph.D.; and c) to have a defined line of research and a present research project of scientific merit, covering the area of medicine. These researchers are currently classified into two main categories (1 and 2). Category 1 was subdivided into four levels: 1A, 1B, 1C, and 1D, and the first level of category 1 is attributed only to researchers with outstanding scientific productivity ²⁴.

Several studies have examined the profile and the scientific production of CNPq researchers in various areas of knowledge, including pharmacy²⁰, chemistry²², computer science²⁴, neurosciences^25,26, cardiology²⁷, nephrology^28,29 and clinical medicine^30,31. However, there is a deficiency of studies concerning the entire area of Medicine with its diverse subareas and specialties. In this regard, the present study aims to describe the profile and the scientific production of recipients of CNPq research productivity grants in Medicine. In addition, we evaluated the ability of the Assessor Committee in Medicine to rank researchers that outstand among their peers as a result of the scientific-technical production developed.

METHODS

Participants

This was a cross-sectional study conducted on 542 investigators registered as recipients of CNPq research productivity grants in Medicine according to a list provided by the agency in February 2013.

Data collection

For this investigation, we used the list of researchers in Medicine from CNPq, with active research productivity grants during the triennium 2012-2014. Using the openly available Lattes curriculum in the CNPq Lattes Platform (http://lattes.cnpq.br/) we elaborated a database with information on each researcher in terms of geographic distribution, institution, time since obtaining the Ph.D., scientific production (published papers), and training of human resources (supervision of undergraduate, master and Ph.D. students). For this data collection, we considered all papers and all student tutoring during the scientific career span of the researcher. We also analyzed the same content for the last five years, considering the period 2008-2012.

Additionally, we searched the database of Web of Science, Thomson Reuters (WoS) at http://apps.isiknowledge.com/. The database was consulted through the CAPES website (http://novo.periodicos.capes.gov.br/). This database was surveyed with the aim to get the number of times the researchers were cited. The main problem in processing our data was to properly identify authors since the same author can provide his/her name or can be registered in diverse ways^9–10. Therefore, the scientific name of the researcher primarily used in this investigation was that provided in the Lattes curriculum. Furthermore, there was an intense search for possible variations of researchers' names. Additionally, data were checked with the following filters available on the WoS database: (i) institution; (ii) subject area; (iii) year of publication, and (iv) source titles. We also used the filter called “Document type” and we excluded from the analysis abstracts presented at meetings.

Area of knowledge and medical specialties

For this variable, we considered the expertise area specifically assigned by the investigator in the Lattes curriculum. When this information was missing, we analyzed the researchers' scientific production over the past five years and allocated them to the area of knowledge in which there was a predominance of issues published.

Variables of interest

The following variables, divided into three categories, were analyzed: (1) Demographic variables, including gender and regional distribution; (2) Background information, including affiliation, post-graduate features, and time since obtaining the Ph.D., and (3) Scientific productivity, including mentoring undergraduate, master's, and Ph.D. students, number of articles, and number of papers indexed in the WoS databases. Concerning publications and student supervision, we analyzed all production during the entire scientific career as well as in the last five-year period. Regarding both student supervision and scientific publications, all data were adjusted per time after the researcher's Ph.D., i.e., the average of production and citation per year. Research performance indicators were also included in the analysis: adjusted number of citations, and the number of citations per paper^15,32–34. Research productivity grants categories/levels were classified in the CNPq database as two main categories (1 and 2), with four levels in category 1 (1A, 1B, 1C and 1D). These categories/levels were taken into account for analysis purposes.

Statistical analysis

The SPSS (Statistical Package for Social Science for Windows, Inc., USA) version 18.0 for Windows was used to create the database and to perform the statistical analysis. Continuous data were reported as medians and interquartile range (IQ) or means and standard deviation (SD), when appropriate. The non-parametric Mann-Whitney and Kruskal-Wallis (KW) tests were used for comparison of medians and ANOVA was used for comparison of means among groups. Dichotomous or nominal variables were compared using the chi-square test.

RESULTS

The demographic characteristics and the area of knowledge of the researchers are summarized in Table 1. There was an overall predominance of males (63.5%) and grants in category 2 (54.1%). There was a significant difference in the distribution of categories between genders. Among males, 52% had grants in categories 1 (1A-1D), while only 36% of females were included in these categories (P<0.001). Five states of the Federation were responsible for approximately 90% of the researchers, and the State of São Paulo accounted for about 60% of them. Six institutions were responsible for approximately 70% of researchers, with a prominence of USP with about a third of them. Regarding Ph.D., most of the researchers (n=504, 93%) obtained the degree in Brazil and 38 (7%) in institutions abroad. The main institution from which the researchers obtained their Ph.D. was USP (35%). Indeed, six universities accounted for about 80% of the Ph.D. (USP, UNIFESP, UFRGS, UNICAMP, UFRJ, and UFMG).

TABLE 1
DEMOGRAPHIC, BACKGROUND, AND POST-GRADUATE FEATURES OF CNPQ RESEARCHERS IN CLINICAL MEDICINE (N = 542)

Gender		n (%)
	Male	344 (63.5)
	Female	198 (36.5)
Scholarship categories
	1A	60 (11.1)
	1B	52 (9.6)
	1C	50 (9.2)
	1D	87 (16.1)
	2	293 (54.1)
Federation State
	SP	318 (58.7)
	RS	62 (11.4)
	RJ	55 (10.1)
	MG	36 (6.6)
	BA	19 (3.5)
	Others	52 (9.6)
Researcher Institution
	USP	164 (30.3)
	UNIFESP	72 (13.3)
	UFRGS	44 (8.1)
	UNICAMP	41 (7.6)
	UFMG	32 (5.9)
	UFRJ	30 (5.5)
	Others	159 (29.3)
Area of knowledge
	Psychiatry/Neurosciences	73 (13.5)
	Cardiology	47 (8.7)
	Gynecology/Obstetrics	44 (8.1)
	Endocrinology	39 (7.2)
	Pathology	37 (6.8)
	Pediatrics	33 (6.1)
	Nephrology/Urology	33 (6.1)
	Infectious Diseases	27 (5.0)
	Hematology/Oncology	26 (4.8)
	Medicine/Intensive Medicine	24 (4.4)
	Pneumology	20 (3.7)
	Ophthalmology	15 (2.8)
	Immunology	15 (2.8)
	Gastroenterology/Hepatology	14 (2.6)
	Rheumatology	13 (2.4)
	Genetics/Molecular Biology	13 (2.4)
	Others	69 (12.7)
Doctorate country*
	Brazil	502 (92.3)
	United Kingdom	12 (2.2)
	Germany	10 (1.8)
	Canada	6 (0.9)
	Japan	3 (0.6)
	Netherlands	3 (0.6)
	Others	4 (0.8)
Doctorate Institution*
	USP	191 (35.2)
	UNIFESP	103 (19.0)
	UFRGS	49 (9.0)
	UNICAMP	37 (6.8)
	UFRJ	35 (6.5)
	UFMG	27 (5.0)
	Others	100 (18.1)
Time since Doctorate (years)
	Median (IQ range)	18 (13 – 24)
Post Doctorate
	Yes	305 (56.3)
	No	237 (43.7)
Post Doctorate country
	USA	144 (47.2)
	Brazil	55 (18.0)
	United Kingdom	34 (11.1)
	Canada	19 (6.2)
	France	16 (5.2)
	Others	37 (12.1)

• ^*
  Two researchers without a doctorate

A total of 35 areas of research were identified with a large dispersion of investigators among the several areas of interest. As shown in Table 1, the five areas of research more frequently assigned as the main area of interest of the researchers include about 45% of the total of the investigators.

The overall median time since receiving the doctoral degree was of 18 years (Interquartile range, 13 – 24 years). Female researchers had a median of 17 years of Ph.D. time (13 – 21), while males had a median of 18.5 (13 – 25) (P=0.013). The majority of the researchers (56.3%) had post-doctoral training, predominantly at USA institutions

Scientific productivity: human resource training

During their scientific career, CNPq researchers in Medicine have trained 7336 undergraduate research fellows (a program known as scientific initiation at Brazilian universities), with a median of 8 (IQ, 3 – 18) per investigator, 6962 master's students (median of 10, IQ = 5 - 18), and 4962 Ph.D. students (median of 7, IQ = 3 - 13). Table 2 shows the absolute and adjusted (per year of Ph.D.) values considering the mentorship stratified by the grant categories. Concerning the absolute and adjusted values, there was a significant difference among grant categories regarding the number of masters and Ph.D. students, but not regarding undergraduate students (Table 2).

TABLE 2
MENTORING AND SCIENTIFIC PUBLICATION BY CNPQ RESEARCHERS IN MEDICINE ACCORDING TO THE SCHOLARSHIP CATEGORIES (N = 542)

Variables		1A – 1B n = 114	1C-1D n = 136	Level 2 n = 292	p-value*
Undergraduates
	Median (IQ range)	12.5 (3.0 – 26.0)	9.0 (3.0 – 17.0)	7.0 (2.0 – 17.0)	0.03
Undergraduates/year
	Median (IQ range)	0.54 (0.15 – 1.2)	0.48 (0.16 – 1.0)	0.5 (0.16 – 1.0)	0.63
Master degree
	Median (IQ range)	16.5 (9.2 – 17.7)	10.0 (7.0 – 17.7)	3.2 (8.0 – 14.8)	<0.001
Master degree/year
	Median (IQ range)	0.73 (0.42 – 1.2)	0.61 (0.3 – 0.91)	0.57 (0.3 – 0.8)	0.01
Doctorate
	Median (IQ range)	17.0 (10.0 – 23.8)	9.0 (5.0 – 12.0)	4.0 (1.0 – 8.0)	<0.001
Doctorate/year
	Median (IQ range)	0.73 (0.54 – 0.94)	0.43 (0.3 – 0.63)	0.26 (0.12 – 0.4)	<0.001
Mentorship total
	Median (IQ range)	53.5 (36.0 – 75.0)	30.5 (21.0 – 44.8)	21.5 (11.0 – 39.0)	<0.001
Mentorship total/year
	Median (IQ range)	2.3 (1.5 – 3.4)	1.6 (1.2 – 2.4)	1.5 (0.83 – 2.3)	<0.001
Total articles
	Median (IQ range)	193 (155 – 282)	129 (101 – 176)	88 (60 – 126)	<0.001
Total articles/year
	Median (IQ range)	9.4 (6.5 – 12.2)	6.9 (5.3 – 9.4)	5.8 (4.0 – 7.9)	<0.001
Total articles WoS
	Median (IQ range)	108 (87 – 148)	66 (51 – 86)	33 (25 – 49)	<0.001
Total articles WoS/year
	Median (IQ range)	4.7 (3.5 – 7.0)	3.6 (2.4 – 5.2)	2.2 (1.5 – 3.3)	<0.001
Articles last 5 years
	Median (IQ range)	70 (49 - 105)	50 (34 – 66)	36 (26 – 50)	<0.001
Articles last 5 years WoS
	Median (IQ range)	36.5 (26 - 55)	26 (18 – 32)	14 (10 – 21)	<0.001
Increment of Scientific Output (%)
	Median (IQ range)	63.7 (28 – 108)	41.0 (12 – 64)	25.4 (3 – 63)	<0.001

Scientific output: publications

During the entire period of their academic career, CNPq medicine researchers published 76512 articles, with a median of 119 articles per researcher (IQ, 77 to 174), ranging from 8 to 992 articles.

There was a significant difference of the median number of publications among the grant categories considering both the absolute and the adjusted number of articles (total of articles, in the last 5 years, and articles indexed on WoS). There was also a significant difference among grant categories in relation to the increase of scientific publication (Table 2).

During the last five years, the total of articles was 29618 with a median of 44 per researcher (IQ, 29 – 63), ranging from 7 to 546. Among the 76512 articles, 36584 (47.8%) were indexed in the WoS database (median per researcher of 53, IQ, 32 - 85). Considering the average number of articles published annually, the majority of researchers (438, 81%) increased their scientific output over the past five years. This increase ranged from 2% to 326% with a median increment of 49.8% in overall scientific production (IQ, 23 to 79).

Regarding the areas of knowledge, there was a significant difference in the increment of the scientific output in the last 5 years (F = 2.6, P < 0.001). The area of knowledge with the greatest mean percentage of increment was Immunology (64.9% ± 42), followed by Endocrinology (57.5% ± 42.0), Hematology/Oncology (54.7% ±52), Neurosciences/Psychiatry (50.3% ± 51), Pneumology (46.8% ± 51), Surgery (43.8% ± 61), Pediatrics (43.3% ± 51), and Nephrology/Urology (42.5% ± 51). Figure 1 illustrates the increment of scientific output by comparing the mean number of papers/years in the last five years and in the whole scientific career among the researchers in the diverse areas of knowledge.

FIGURE 1

Scientific production: citations

During their academic career, CNPq researchers in Medicine received 643159 citations in the WoS database, with a median of 754 citations per researcher (IQ, 356 to 1447, ranging from 29 to 12741). The median number of citations per article was 13.9 (IQ, 9.5 – 17.5). There were significant differences among the grant categories considering the number of citations in WoS (KW = 218.5, P <0.001). CNPq researchers of level 1A-B had a median of 1664 citations (IQ, 1129 – 2292); level 1C-D, 992 citations (IQ, 730 – 1645); and level 2, 425 citations (232 -752). The difference remained significant even after the adjustment for the time of obtaining the Ph.D. The mean number of citations per year was 105 (SD, 89), 78.4 (SD, 60), and 47 (SD, 83) for levels 1A-B, 1C-D, and 2, respectively (P<0.001). The mean of citations per article was 18 (SD, 9.8) for level 1A-B, 19.6 (SD, 13.4) for level 1C-D, and 15.7 (SD, 14.5) for level 2. There was also a significant difference among grant levels regarding the mean citation per article (P=0.016). Interestingly, there was a moderate positive correlation between the number of papers published in journals indexed on WoS and the number of citations received by the researchers (r=0.66, P < 0.001).

Considering the several areas of knowledge, there was a significant difference in the mean of citations per paper received by the CNPq researchers (F = 5.4, P < 0.001). The area of knowledge with the greatest mean of citations per paper was Cardiology (29.5 ± 29.1), followed by Immunology (22.7 ± 16.9), Pneumology (21.7 ± 13.0), Infectious Diseases (20.3 ± 11.2), Neurosciences/Psychiatry (19.7 ± 12.5), Nephrology/ Urology (18.6 ± 13.2), Medicine/Intensive Medicine (16.6 ± 6.5), Hematology/Oncology (16.2 ± 8.8), and Endocrinology (15.9 ± 8.0).

DISCUSSION

A relevant finding that emerges from our cross-sectional study on CNPq researchers in the field of Medicine is the concentration of scientific output in a few Brazilian states. Five of them account for approximately 90% of the researchers, and a single state (São Paulo) for remarkably 60% of the CNPq researchers in Medicine. Notably, this figure is even disproportionately higher than the participation of the State of São Paulo in the Brazilian Gross Domestic Product (GDP), which is about 32% ³⁵. The findings of this study also show that two institutions in the state of São Paulo (USP and UNIFESP) were responsible for the Ph.D. of about 54% of the researchers. This fact can contribute partly to the concentration of the CNPq researchers in the Southeast region of Brazil³⁶. It is important to highlight that only six researchers with a grant in Medicine (1.1%) were from the North and Midwest regions of Brazil, which reinforces the heterogeneous spatial distribution in the country. This finding should subsidize specific government actions to address such regional differences³⁷.

Our analysis showed that CNPq male researchers have proportionally more grants in category 1 than female researchers. We believe that even though males had a longer Ph.D. time than females, the relatively small difference of 1.5 years cannot explain the imbalance in the grant categories. Concerning the gender disparities in science, Larivière et al.³⁸ have recently presented a bibliometric analysis confirming that gender inequalities persist in research output worldwide. Moreover, although there are more female than male undergraduate and graduate students in many countries, there are relatively few female full professors, and gender inequalities in hiring, earnings, funding, satisfaction, and patenting persist. As expected, in Brazil the state of affairs is quite similar, and women are in lower proportion in the higher positions of the academic career, that is, those positions associated with higher income and higher academic prestige³⁹.

Another point to be emphasized in our study is the assessment of the scientific output by CNPq researchers in Medicine in quantitative and qualitative terms through the analysis of bibliometric indicators. From the quantitative point of the view, our study showed an important scientific output with an expressive number of publications of scientific articles in periodicals indexed at the WoS database. Among the 76512 articles published by the Medicine CNPq researchers, 36584 (47.8%) were indexed in the WoS database. Another point to be emphasized is the increment of the scientific output over the last 5 years, a fact that has also been observed in other areas of knowledge^29–31,40. CNPq researchers in Medicine had an increase of about 50% in the number of published articles, in comparison to the annual output across the career over the last 5 years. This quantitative increment in scientific production correlates with the general increase in scientific production in Brazil, and possibly reflects the various fostering mechanisms implemented by the various national research support agencies^{27,28,41–45}. Unfortunately, the current economic crisis has already resulted in cuts to federal and state science funding. This will probably impair Brazilian research and possibly hamper the scientific output increment over the next years⁴⁶.

Another point assessed in our study was the impact of scientific publications by CNPq researchers in Medicine through the analysis of the number of citations. The most widespread method for judging the impact of biomedical articles is citation count, which is the number of citations received⁴⁷. This metric was first introduced by Gross and Gross⁴⁸. It is generally assumed that articles with higher impact receive more citations. Although quantifying the quality of individual scientists is difficult, the general view is that the citation count of a paper (relative to citation habits in its field) has been considered a useful measure of its quality¹⁰. In our analysis, the median number of citations per researcher and number of citations per article was 754 and 13.9, respectively. There were significant differences among the grant categories regarding the number of citations received in WoS. Nevertheless, it is important to recognize that it takes time for a paper to accumulate its full complement of citations¹⁰. This fact represents a serious limitation on the value of citation analyses for younger authors, who presumably are the researchers of CNPq grant in category 2.

Another relevant topic that should be pointed out in this study is the remarkable difference in the citation counts among researchers from the diverse subareas of Medicine. In this regard, the areas of Cardiology and Neurosciences stand out with a median of about 12000 citations per researcher. These data emphasize the quality of the scientific output of this group of researchers. In a previous analysis, we have shown that of the 587 journals identified as being used by the CNPq researchers in Cardiology, 340 (58%) are indexed in the Web of Science database, with a median impact factor (IF) of 2.65²⁷. It is notable that approximately 16% of these journals have IF > 5. In the database of the Web of Science, there are 97 indexed journals in the area of Cardiology, and only 10 (10.5%) have IF > 5. We have described a similar pattern for the Brazilian researchers in Neuroscience²⁶. Among the journals used by the CNPq Neurosciences researcher, 61% were indexed at WoS with a median IF of 2.58. It is noteworthy that approximately 14% of the journals used by researchers in neurosciences have an IF > 5. The same database shows that among 128 journals indexed in the field of Psychiatry and 239 in the field of Neurosciences, only 14 (11%) and 39 (16.3%) had an IF greater than 5, respectively. Therefore, researchers in these areas published papers in indexed journals above the median of the remaining specialties. Interestingly, our data also have shown that there was a positive correlation between the number of papers published in journals indexed on WoS and the number of citations received by the researchers.

Our results should be considered in light of some methodological limitations. In this respect, the possible major weakness was the difficulty to get, in a reliable way, important research metrics, including h-index and m-index. We have tried to recover this information from the Lattes curriculum of each researcher. CNPq has developed the Lattes curriculum system in order to record the scientific output of Brazilian researchers²⁴. The current version of the system allows researchers to update their Lattes curriculum vitae (CV) and others to consult the English version of the CVs using a Web system (http://lattes.cnpq.br/english). According to a recent comment in Nature, the Brazilian experience with the Lattes database is a powerful example of good practice. This provides high-quality data on about 1.6 million researchers and about 4,000 institutions⁹. However, we have found that the information regarding h-index was lacking in about 30% of the CVs in the Lattes platform and, moreover, many researchers have not updated this information in an appropriate way. For this reason, we had to obtain the h-index from WoS by using scientific names informed by each researcher in their Lattes curriculum. Consequently, partially due to these difficulties, we were not able to address some relevant issues regarding the impact of the scientific output of this group of researchers. However, some characteristics of the study may increase the strength of our results, including the strategies mentioned above and the systematic search of the Lattes and WoS databases, according to a well-established protocol.

CONCLUSION

This study of the profile of CNPq researchers in the area of Medicine has shown that the majority of them are males, concentrated in the southeast region of Brazil, particularly in the state of São Paulo. We have shown that CNPq researchers in the field of Medicine have a relevant quantitative and qualitative scientific output, although with a significant discrepancy among the diverse subareas included in this field of knowledge. In this regard, Cardiology and Neurosciences stand out among the several specialties. This overall scientific production has increased significantly in recent years, but again with a relevant asymmetry among the diverse areas.

Finally, our findings suggest that the Assessor Committee in Medicine follows the criteria that were set for awarding productivity grants. According to the objective criteria, including scientific production and human resources training, there is consistency in ranking the grant holders among the diverse grant categories. Nevertheless, it is possible that fine adjustments might still be needed, especially in indices harder to measure, such as leadership and innovation. Further studies addressing some critical issues like research groups' productivity, collaborative efforts, and the impact of the scientific output might contribute to our better understanding of this dynamic area of research.

Acknowledgments

This study was partially supported by CNPq (National Council for Scientific and Technological Development) and FAPEMIG (Research Support Foundation of Minas Gerais).

REFERENCES

• ¹
  Braun T. How to improve the use of metrics: get experts on board. Nature. 2010;465(7300):870-2.
• ²
  Braun T. Editorial. Scientometrics. 2012;92:207-8.
• ³
  Bergstrom CT. How to improve the use of metrics: use ranking to help search. Nature. 2010;465(7300):870-2.
• ⁴
  Wilcox AJ. Rise and fall of the Thomson impact factor. Epidemiology. 2008;19(3):373-4.
• ⁵
  Taylor M, Perakakis P, Trachana V. The siege of science. Ethics Sci Environ Polit. 2008;8(1):17-40.
• ⁶
  Nederhof AJ. Policy impact of bibliometric rankings of research performance of departments and individuals in economics. Scientometrics. 2008;74(1):163-74.
• ⁷
  Browman HI, Stergiou KI. Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely. Ethics Sci Environ Polit. 2008;8:1-3.
• ⁸
  Picinin CT, Pilatti LA, Kovaleski JL, Graeml AR, Pedroso B. Comparison of performance of researchers recipients of CNPq productivity grants in the field of Brazilian production engineering. Scientometrics. 2016;109(2):855-70.
• ⁹
  Lane J. Let's make science metrics more scientific. Nature. 2010;464(7288):488-9.
• ¹⁰
  Lehmann S, Jackson AD, Lautrup BE. Measures for measures. Nature. 2006;444(7122):1003-4.
• ¹¹
  Simons K. The misused impact factor. Science. 2008;322(5899):165.
• ¹²
  Sahel JA. Quality versus quantity: assessing individual research performance. Sci Transl Med. 2011;3(84):84cm13.
• ¹³
  Mugnaini R, Jannuzzi PM, Quoniam L. Indicadores bibliométricos da produção científica brasileira: uma análise a partir da base Pascal. Ci Inf. 2004;33(2):123-31.
• ¹⁴
  Pan RK, Fortunato S. Author impact factor: tracking the dynamics of individual scientific impact. Sci Rep. 2014;4:4880.
• ¹⁵
  Panaretos J, Malesios CC. Assessing scientific research performance and impact with single indices. Scientometrics. 2009;81(3):635-70.
• ¹⁶
  Petersen AM, Fortunato S, Pan RK, Kaski K, Penner O, Rungi A, et al. Reputation and impact in academic careers. Proc Natl Acad Sci U S A. 2014;111(43):15316-21.
• ¹⁷
  Adam D. The counting house. Nature. 2002;415(6873):726-9.
• ¹⁸
  Bornmann L, Daniel H-D. Committee peer review at an international research foundation: predictive validity and fairness of selection decisions on post-graduate fellowship applications. Research Evaluation. 2005;14(1):15-20.
• ¹⁹
  Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):498-502.
• ²⁰
  Rodrigues LO, Gouvea MM, Marques FFC, Mourão SC. Overview of the scientific production in the Pharmacy area in Brazil: profile and productivity of researchers granted with fellowships by the National Council for Scientific and Technological Development. Scientometrics. 2017;110:1157-71.
• ²¹
  Conselho Nacional de Desenvolvimento Científico e Tecnológico, Tabelas de Áreas do Conhecimento. [cited 2018 Mar 03]. Available from: http://www.cnpq.br/documents/10157/186158/TabeladeAreasdoConhecimento.pdf
  » http://www.cnpq.br/documents/10157/186158/TabeladeAreasdoConhecimento.pdf
• ²²
  Santos NCF, Candido LFO, Kuppens CL. Produtividade em pesquisa do CNPq: análise do perfil dos pesquisadores da química. Quim Nova. 2010;33(2):489-95.
• ²³
  Produtividade em Pesquisa, Anexo III. [cited 2018 Mar 03]. Available from: http://www.cnpq.br/view/-/journal_content/56_INSTANCE_0oED/10157/2958271?COMPANY_ID=10132 Accessed 04.10.2017, 2010.
  » http://www.cnpq.br/view/-/journal_content/56_INSTANCE_0oED/10157/2958271?COMPANY_ID=10132
• ²⁴
  Arruda D, Bezerra F, Neris VA, Toro PR, Wainer J. Brazilian computer science research: gender and regional distributions. Scientometrics. 2009;79(3):651-65.
• ²⁵
  Hoppen NHF, Vanz SAS. Neurosciences in Brazil: a bibliometric study of main characteristics, collaboration and citations. Scientometrics. 2016;109(1):121-41.
• ²⁶
  Romano-Silva MA, Correa H, Oliveira MCL, Quirino IG, Colosimo EA, Martelli DR, et al. Perfil e análise da produção científica dos pesquisadores brasileiros em Neurociência Clínica. Rev Psiq Clín. 2013;40(2):53-8.
• ²⁷
  Oliveira EA, Ribeiro AL, Quirino IG, Oliveira MC, Martelli DR, Lima LS, et al. Perfil e produção científica dos pesquisadores do Conselho Nacional de Desenvolvimento Científico e Tecnológico na área de Cardiologia. Arq Bras Cardiol. 2011;97(3):186-93.
• ²⁸
  Oliveira EA, Pecoits-Filho R, Quirino IG, Oliveira MC, Martelli DR, Lima LS, et al. Perfil e produção científica dos pesquisadores do CNPq nas áreas de Nefrologia e Urologia. J Bras Nefrol. 2011;33(1):31-7.
• ²⁹
  Oliveira EA, Colosimo EA, Martelli DR, Quirino IG, Oliveira MCL, Lima LS, et al. Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted? Scientometrics. 2012;90(2):429-43.
• ³⁰
  Mendes PHC, Martelli DRB, Souza WP, Quirino Filho S, Martelli Junior H. Perfil dos pesquisadores bolsistas de produtividade científica na medicina no CNPq, Brasil. Rev Bras Educ Med. 2010;34(4):535-41.
• ³¹
  Martelli-Junior H, Martelli DR, Quirino IG, Oliveira MC, Lima LS, Oliveira EA. CNPq-supported medical researchers: a comparative study of research areas. Rev Assoc Med Bras (1992). 2010;56(4):478-83.
• ³²
  Bornmann L, Wallon G, Ledin A. Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation. 2008;17(2):149-56.
• ³³
  Haeffner-Cavaillon N, Graillot-Gak C. The use of bibliometric indicators to help peer-review assessment. Arch Immunol Ther Exp (Warsz). 2009;57(1):33-8.
• ³⁴
  Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569-72.
• ³⁵
  Instituto Brasileiro de Geografia e Estatística. 2014. [cited 2017 Mar 26]. 2017. Available from: http://saladeimprensa.ibge.gov.br/en/noticias.html?view=noticia&id=1&busca=1&idnoticia=3315
  » http://saladeimprensa.ibge.gov.br/en/noticias.html?view=noticia&id=1&busca=1&idnoticia=3315
• ³⁶
  Velloso A, Lannes D, Meis L. Concentration of science in Brazilian governmental universities. Scientometrics. 2004;61(2):207-20.
• ³⁷
  Sturmer G, Viero CC, Silveira MN, Lukrafka JL, Plentz RD. Profile and scientific output analysis of physical therapy researchers with research productivity fellowship from the Brazilian National Council for Scientific and Technological Development. Braz J Phys Ther. 2013;17(1):41-8.
• ³⁸
  Larivière V, Ni C, Gingras Y, Cronin B, Sugimoto CR. Bibliometrics: global gender disparities in science. Nature. 2013;504(7479):211-3.
• ³⁹
  Moschkovich M, Almeida AMF. Gender inequality in academic career in Brazil. Dados. 2015;58(3):749-89.
• ⁴⁰
  Oliveira EA, Peicots-Filho R, Martelli DR, Quirino IG, Oliveira MCL, Duarte MG, et al. Is there a correlation between journal impact factor and researchers' performance? A study comprising the fields of clinical nephrology and neurosciences. Scientometrics. 2013;97:149-60.
• ⁴¹
  Guimarães JA. A pesquisa médica e biomédica no Brasil. Comparações com o desempenho científico brasileiro e mundial. Cien Saude Colet. 2004;9(2):303-27.
• ⁴²
  Petherick A. High hopes for Brazilian science. Nature. 2010;465(7299):674-5.
• ⁴³
  Regalado A. Science in Brazil. Talented but underfunded: Brazil's future scientists. Science. 2010;330(6009):1311.
• ⁴⁴
  Regalado A. Science in Brazil. Tapping a deep, ‘pre-salt’ bounty. Science. 2010;330(6009):1308-9.
• ⁴⁵
  Regalado A. Science in Brazil. Brazilian science: riding a gusher. Science. 2010;330(6009):1306-12.
• ⁴⁶
  Gibney E. Brazilian science paralysed by economic slump. Nature. 2015;526(7571):16-7.
• ⁴⁷
  Fu LD, Aliferis CF. Using content-based and bibliometric features for machine learning models to predict citation counts in the biomedical literature. Scientometrics. 2010;85(1):257-70.
• ⁴⁸
  Gross PL, Gross EM. College libraries and chemical education. Science. 1927;66(1713):385-9.

Publication Dates

• Publication in this collection
  03 June 2019
• Date of issue
  2019

History

• Received
  16 Sept 2018
• Accepted
  02 Oct 2018