Impact of SciELO and MEDLINE indexing on submissions to Jornal de Pediatria

Abstracts

OBJETIVO: Avaliar o impacto da indexação no SciELO e MEDLINE sobre o número de artigos submetidos ao Jornal de Pediatria. MÉTODOS: Análise do total de artigos submetidos, artigos estrangeiros submetidos e índices de aceitação, nos seguintes períodos: estágio I - pré-site (janeiro/2000-março/2001); estágio II - site (abril/2001-julho/2002); estágio III - SciELO (agosto/2002-agosto/2003); estágio IV - MEDLINE (setembro/2003-dezembro/2004). RESULTADOS: Houve uma tendência significativa de aumento linear no número de submissões, durante o período do estudo (p = 0,009). O número de originais submetidos nos estágios I a IV foi, respectivamente: 184, 240, 297 e 482. O número de submissões foi similar nos estágios I e II (p = 0,148), mas foi significativamente maior no estágio III (p < 0,001 vs. estágio I; p = 0,006 vs. estágio II) e no estágio IV (p < 0,001 vs. estágios I e II; p < 0,05 vs. estágio III). A taxa de aceitação diminuiu durante o período do estudo. O número de 10 artigos originais publicados é estável desde o número de março/abril de 2001, quando a revista atingiu um limite de páginas impressas que motivou a adoção de critérios de julgamento mais rígidos e uma diminuição relativa dos índices de aceitação. O número de submissões estrangeiras nos estágios I a IV foi, respectivamente, 1, 2, zero e 17, sendo p < 0,001 para a comparação do estágio IV com os anteriores. CONCLUSÕES: A indexação no SciELO se associou a um aumento das submissões de artigos brasileiros ao Jornal de Pediatria, enquanto a indexação no MEDLINE levou a um aumento de submissões brasileiras e estrangeiras.

Base de dados; artigo de revista; medição de impacto

SPECIAL ARTICLE

Impact of SciELO and MEDLINE indexing on submissions to Jornal de Pediatria

Danilo Blank^I; Claudia Buchweitz^II; Renato S. Procianoy^III

^IAssociate editor, Jornal de Pediatria; Editor, JPED/Brazilian Society of Pediatrics

^IIExecutive editor, JPED/Brazilian Society of Pediatrics

^IIIEditor-in-Chief, Jornal de Pediatria and JPED/Brazilian Society of Pediatrics

^{Correspondence}

ABSTRACT

OBJECTIVE: To evaluate the impact of SciELO and MEDLINE indexing on the number of articles submitted to Jornal de Pediatria.

METHODS: Analysis of total article submission, submission of articles from foreign countries and acceptance figures in the following periods: stage I - pre-website (Jan 2000-Mar 2001); stage II - website (Apr 2001-Jul 2002); stage III - SciELO (Aug 2002-Aug 2003); stage IV - MEDLINE (Sep 2003-Dec 2004)

RESULTS: There was a significant trend toward linear increase in the number of submissions along the study period (p = 0.009). The number of manuscripts submitted in stages I through IV was 184, 240, 297, and 482, respectively. The number of submissions was similar in stages I and II (p = 0.148), but statistically higher in Stage III (p < 0.001 vs. Stage I and p = 0.006 vs. Stage II) and Stage IV (p < 0.001 vs. stages I and II, and p < 0.05 vs. stage III). The rate of article acceptance decreased during the study period. The number of original articles published has been stable since the 2001 March/April issue (n = 10), when the journal reached a printed page limit, leading to stricter judgment criteria and a relative decrease in acceptance rate. The number of foreign submissions in stages I through IV was 1, 2, zero and 17, respectively, with p < 0.001 for the comparison of stage IV with previous stages.

CONCLUSIONS: SciELO indexing was associated with an increase in Brazilian manuscript submissions to Jornal de Pediatria, whereas MEDLINE indexing led to an increase in both Brazilian and foreign submissions

Key words: Database, journal article, impact assessment.

Introduction

Publishing is an essential part of scientific research. However, printing a paper is not sufficient - it is also necessary that the work be read by the broadest possible audience, and that it be cited by others. Because of that, most journals work hard to get indexed in international databases, based on the common sense conviction that indexing will naturally be followed firstly by worldwide visibility, and thereafter by increased citation.^1-3

However, evidence of this is lacking in the literature. In fact, a thorough search returned only editorials rejoicing over a given journal having been indexed in a particular database, or commenting on the difficult path toward achieving such goal and on the prospects considering this high-valued accomplishment.^4-7 We retrieved a single cursory follow up evaluation which in effect showed that the number of citations of articles published in five Brazilian journals more than doubled (as gauged by the Institute for Scientific Information's impact factor) after their inclusion in the Scientific Electronic Library Online (SciELO).¹

Therefore, we set out to evaluate the impact of the Jornal de Pediatria's inclusion in SciELO in 2002⁸ and in MEDLINE in 2003,⁹ by taking the rate of article submission as a proxy indicator of visibility and attributed quality.

Methods

We analyzed total article submission to Jornal de Pediatria from 2000 through 2004. Secondary outcomes were submission of articles from foreign countries and acceptance figures for the same period.

Since there were no changes in either the editorial board or the methods of manuscript submission throughout the study period, we considered the potential impact of three events on submission rates: the first was the launch of our bilingual (Portuguese/English) free-access website, in March 2001; the second was the indexing in SciELO, in July 2002; and the third was the indexing in MEDLINE, in August 2003. Thus, four stages were considered: stage I, which we called "pre-website" (15 months), stage II, "website" (16 months), stage III, "SciELO" (13 months), and stage IV, "MEDLINE" (16 months).

Statistical analysis comprised simple regression for trend analysis, one-way ANOVA on rank transformed data with Duncan's posthoc test to compare the number of submissions in each period, and Fisher's Exact test with Finner-Bonferroni p-value adjustment to compare foreign submissions in the four periods.

Results

There was a significant trend toward linear increase in the number of submissions to Jornal de Pediatria along the study period (p = 0.009) (Figure 1). Table 1 shows the number of articles submitted in each category in stages I through IV. The number of submissions was not statistically different in stages I and II (p = 0.148), but it was statistically higher in stage III (p < 0.001 vs. stage I and p = 0.006 vs. stage II) and stage IV (p < 0.001 vs. stages I and II, and p < 0.05 vs. stage III). The upward variation in the submission of letters to the editor was the most striking among the article categories, followed by original papers.

There was an erratic and pronounced fluctuation in monthly submissions in stages III and IV (Table 2)

Since the absolute number of accepted articles per issue remained stable throughout the study period, the rate of acceptance naturally decreased. This was most pronounced for case reports.

The number of foreign articles submitted in stages I through IV was 1, 2, zero and 17, respectively, with p < 0.001 for the comparison of stage IV with previous stages.

Discussion

We observed an unequivocal association between SciELO indexing and a rise in manuscript submissions from Brazil to Jornal de Pediatria, while MEDLINE indexing clearly led to an increase in both Brazilian and foreign submissions. These findings substantiate the empirically held views that authors submit more papers to journals that are indexed in international databases. ^1,10-16

Authors prefer indexed journals because of their visibility, and also because research funding agencies attribute special value to those publications.^10,17 In Brazil, this is one of the major criteria employed by the Federal Coordinating Agency for the Improvement of Higher Education (CAPES) to rank graduate programs. CAPES classifies journals by means of a system known as Qualis, according to which top journals are indexed in MEDLINE and have a high impact factor, as measured by Thomson Scientific's Journal Citation Reports (JCR), ^18,19 next come MEDLINE-indexed journals with a low impact factor; and third are journals indexed in MEDLINE but not covered by the JCR.

The fact that research funding agencies in Brazil, and perhaps in other countries as well, value above all coverage by the JCR deserves further remarks. First of all, many have pointed out that counting citations is a flawed way to appraise academic worth, among other reasons because it does not measure citations to any single article.^20-25 Second, if an article is cited frequently as an example of poor research, it will nevertheless raise the journal's impact factor. In addition, journals with a print version in a language other than English will hardly get coverage from organizations like Thomson-ISI, allegedly because their impact factor is too low - which is probably the case, since international readers will most likely not cite articles published in a language that they cannot read. It should be noted that the number of MEDLINE citations to English language articles has increased from about half the total records in that database before the 1970s to nearly 90% today.²⁶

Moreover, whenever MEDLINE cites a non-English article (meaning an article whose original print version is in any language other than English, irrespective of an English version being provided), its title is listed between square brackets. So, although Jornal de Pediatria - like many journals officially indexed as "non-English" - does provide a link to free English full text, it is fair to presume that those brackets alone cause too many readers to shy away. Two interesting questions that stem from this scenario are: how many investigators searching MEDLINE use language filters, or how many will access and cite articles whose titles appear in square brackets? And - would ISI coverage raise a journal's the impact factor, just as MEDLINE indexing raises submissions?

We are certain that Jornal de Pediatria has been publishing higher-quality articles in preparation for and because of indexing. This partially explains the striking decrease in the rate of acceptance, which has halved along the study period. The number of original articles published has been stable since the 2001 March/April issue, when Jornal de Pediatria reached its printed page limit. This, together with the increase in submissions, led to stricter judgment criteria, and the rate of article acceptance naturally decreased. This effect was most pronounced for case reports, indicating a change in Jornal de Pediatria's characteristics, with higher priority being placed on original research papers.

In conclusion, we have shown that the indexing of Jornal de Pediatria in SciELO and MEDLINE increased its worldwide visibility, as well as an overall perception of attributed quality, thus stimulating researchers to submit their manuscripts for publication.

References

This study was originally presented in the Saturday, September 17, 2005, plenary session of the 5th International Congress on Peer Review and Biomedical Publication: Blank D, Buchweitz C, Procianoy R. Impact of SciELO and MEDLINE indexing on the submission of articles to a "non-English" journal. Abstracts of the 5th International Congress on Peer Review and Biomedical Publication; 2005 Sep 16-18; Chicago, USA. Chicago: JAMA/BMJ; 2005. p. 18-19.

• 1. Alonso WJ, Fernandez-Juricic E. Regional network raises profile of local journals. Nature. 2002;415:471-2.
• 2. Fernández E, Plasència A. Dime cuánto nos citan y te diré… el factor de impacto bibliográfico de Gaceta Sanitária. Gac Sanit. 2003;17:179-80.
• 3. Aksnes DW. Citations and their use as indicators in science policy. Studies of validity and applicability issues with a particular focus on highly cited papers [thesis]. Enschede-West, Netherlands: University of Twente; 2005.
• 4. Eysenbach G. Journal of Medical Internet Research is now indexed in Medline. J Med Internet Res 2001;3(3):e25. www.jmir.org/2001/3/e25/ Accessed: Nov 6, 2005.
• 5. Guimarães V. História da Indexação dos ABE&M: Razão, Trabalho e Emoção. Arq Bras Endocrinol Metab. 2004;48:217-19.
• 6. Bressan RA, Miguel EC, Mari JJ, Rohde LA, Mercadante MT. Chegamos ao MEDLINE, para onde vamos agora?. Rev Bras Psiquiatr. 2004;26:1.
• 7. Vermund SH, Acuna G. Regional journals in medicine and public health: a look to the future upon the indexing of the Revista Chilena de Infectologia. Rev Chilena Infectol. 2005;22:11-20.
• 8. Piva JP, Garcia PC, Blank D, Amantea S. Os desafios e a afirmação de uma revista científica. J Pediatr (Rio J). 2002;78:1-2.
• 9. Procianoy RS. O Jornal de Pediatria está no Index Medicus/MEDLINE. J Pediatr (Rio J). 2003;79:279.
• 10. Vohora SB, Vohora D. Why are Indian journals' impact factors so low? Nature. 2001;412:583.
• 11. Kater CE. Reflexos imediatos da indexação internacional e atualização da política editorial dos ABE&M. Arq Bras Endocrinol Metab. 2004;48:440-2.
• 12. Aleixandre-Benavent RA, Valderrama-Zurián JC, Castellano-Gómez M, Simó-Meléndez R, Navarro-Molina C. Factor de impacto nacional e internacional de Anales de Pediatria. An Pediatr (Barc). 2004;61:201-6.
• 13. Bressan RA, Miguel EC, Mari JJ, Rohde LA, Mercadante MT. Chegamos ao ISI. Rev Bras Psiquiatr. 2005;27:170-1.
• 14. Van Dalen H, Henkens K. Signals in science: on the importance of signaling in gaining attention in science. Scientometrics. 2005;64:209-33.
• 15. Monastersky R. The number that's devouring science. The Chronicle of Higher Education. 2005;52:A12. http://chronicle.com/weekly/v52/i08/08a01201.htm Accessed: Nov 6, 2005.
• 16. National Library of Medicine. MEDLINE Factsheet. www.nlm.nih.gov/pubs/factsheets/medline.html Accessed: Nov 6, 2005.
• 17. Coelho PM, Antunes CM, Costa HM, Kroon EG, Sousa Lima MC, Linardi PM. The use and misuse of the "impact factor" as a parameter for evaluation of scientific publication quality: a proposal to rationalize its application. Braz J Med Biol Res. 2003;36:1605-12.
• ¹⁸
  CAPES. Critérios Qualis e de conceitos relacionados à publicação. http://qualis.capes.gov.br/ Accessed: Oct 30, 2005.
• ¹⁹
  ISI Journal Citation Reports. http://scientific.thomson.com/products/jcr/ Accessed: Nov 6, 2005.
• 20. Glänzel W, Moed HF. Journal impact measures in bibliometric research. Scientometrics. 2002;53:171-93.
• 21. Coura JR, Willcox L C. Impact factor, scientific production and quality of Brazilian medical journals. Mem Inst Oswaldo Cruz. 2003;98:293-7.
• 22. Porta M, Copete JL, Fernandez E, Alguacil J, Murillo J. Mixing journal, article, and author citations, and other pitfalls in the bibliographic impact factor. Cad Saude Publica. 2003;19:1847-62.
• 23. Walter G, Bloch S, Hunt G, Fisher K. Counting on citations: a flawed way to measure quality. Med J Aust. 2003;178:280-1.
• 24. Leff D. Making an impact: the rise of the impact factor as a measure of journal quality. J Am Diet Assoc. 2005;105:29-30.
• 25. Van Raan AF. Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics. 2005;62:133-43.
• ²⁶
  National Library of Medicine, Bibliographic Services Division. MEDLINE: Number of citations to English language articles. www.nlm.nih.gov/bsd/medline_lang_distr.html Accessed: Nov 13, 2005.

Publication Dates

• Publication in this collection
  03 Apr 2006
• Date of issue
  Dec 2005