Comments on the article by Araújo: are the criteria for peer review and publication clear?

Peer review; peer review; research; scientific and technical publications

LETTER TO THE EDITOR

Hospital Universitário Pedro Ernesto, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ - Brazil

Mailing Address

Keywords: Peer review; peer review, research; scientific and technical publications.

Dear Editor,

I read with interest the article by Araújo¹, which approaches the theme of peer review, providing instructions for the careful assessment of scientific papers.

Although the discussion about peer review is not recent, the theme has been given little attention in Brazil. In a search on the PubMed database regarding publications of the last couple of years, the term "peer review" appeared 1176 times, but that number dropped to ten when the search was limited to publications in Portuguese².

In a recent study, van Rooyen et al.³ have emphasized that having the reviewer's name made available on the website alongside the published paper might influence the result of the review. Those authors have reported that, when revealing to reviewers that their names would be disclosed alongside the published paper, more than half of the potential reviewers declined the task. In addition, among those who accepted the invitation, the time taken to complete the review was longer than among reviewers not informed about the possibility of having their names disclosed. Thus, an intrinsic bias in the review process has been shown.

Peer review can also involve aspects beyond the scientific merit of submission⁴. Consider the scenario: an editor receives simultaneously two papers on similar themes, one developed by a renowned research group A and the other developed by an emerging research group B. Considering both the redaction and research quality of both papers flawless, it is not rare that editors of high-impact journals choose to publish the paper by group A, because of the elevated scientific concept of its researchers. Thus, some scientific papers worthy of being published could be considered or rejected based on the scientific concept of the research group involved.

A careful and impartial review of a paper is mandatory. In addition, reviewers and editors of scientific journals have to clearly disclose their criteria for paper review and publication.

Peace and blessings.

Response Letter

We appreciate the interest in our article. We agree with the author of the Letter to the Editor that the peer review process has been insufficiently discussed, especially in Brazil. In reality, although it is an almost universal procedure in scientific journals, as emphasized in our article¹, it is also surprisingly little validated. That author also mentions the potential impact on the quality of the process of having the reviewer's identity disclosed. In practice, journals have several strategies, ranging from the rarest, in which authors and reviewers are identified, to the other extreme, most common, in which both are kept anonymous during the review process. Unfortunately, there is no evidence allowing the comparison of those strategies, especially in Brazil. However, one can speculate that identifying the names of authors and reviewers would impact (positive or negatively?) the review process of the Arquivos Brasileiros de Cardiologia.

Despite its universality and appreciation by the scientific community, peer review has proved to have low reproducibility^2,3. In practice, different reviewers' reports for the same paper usually disagree, and, sometimes, can even be antagonistic - one reviewer recommending acceptance without revision, while another recommends rejection with no chance of resubmission. If it could be properly tested, one might demonstrate that a certain reviewer, depending on his/her "mood", can make disparate decisions when assessing the same paper. It is worrisome that the decisions of peer review, so important for authors, are made in a system with so little reproducibility, of questionable validity, and, differently from the most conventional judgment forms (Common Justice, for example), with no chance of broad defense or appeal for a new and maybe fairer judgment⁴.

With our article¹ we aimed at contributing to improve the peer review process, but we understand that, with the exponential increase in the scientific production and consequent overload for reviewers (worth noting that it is a non-paid task, with strict deadlines and modest impact on the personal curriculum), new models have to be developed.

For the near future, we propose: 1) improvement of a model of pre-submission questioning, in which authors submit their unfinished papers to the editor and/or associated editors, verifying the interest of the latter in receiving a full submission and obtaining a response in up to ten days (Is there a high potential of acceptance?); 2) a pool of journals of related areas, sharing a large body of reviewers and the same publication guidelines, to which authors would submit their papers, identifying their journal priority order; the theme editor and/or associated editors of the journal, listed as top priority by authors, would rapidly decide whether the papers fit or not their publication scope, level and priority. If affirmative, they would tell authors and send them the review (already with at least a 50% chance of acceptance); if negative, they would decline the submission and the process would restart with the second journal listed by the authors, and so on. Occasional reviewer's reports and authors' replies, as well as the new versions of the papers, would be aggregated in the process, making the judgment of subsequent reviewers easier.

In conclusion, the peer review process has a long road to follow. Discussions such as this one, provoked by that Letter to the Editor, can be extremely useful for the scientific community.

Sincerely,

Claudio Gil Soares de Araújo

References

Referências (Resposta)

• 1. Araújo CG. Revisão por pares: um processo científico em constante aprimoramento. Arq Bras Cardiol. 2012;98(2):e32-5.
• ²
  Peer review Limits activated: published in the last two years. Limits activated: Portuguese, published in the last two years.
• 3. van Rooyen S, Delamothe T, Evans SJ. Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ. 2010;341:c5729
• 4. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
• 1. Araújo CGS. Revisão por pares: um processo científico em constante aprimoramento. Arq Bras Cardiol. 2012;98(2):e32-5.
• 2. Bornmann L, Mutz R, Daniel HD. A reliability-generalization study of journal peer reviews: a multilevel meta-analysis of inter-rater reliability and its determinants. PloS One. 2010;5(12):e14331.
• 3. Jackson JL, Srinivasan M, Rea J, Fletcher KE, Kravitz RL. The validity of peer review in a general medicine journal. PloS One. 2011;6(7):e22475.
• 4. Jefferson T, Alderson P, Wager E, Davidoff F. Effects of editorial peer review: a systematic review. JAMA. 2002;287(21):2784-6.

Comments on the article by Araújo: are the criteria for peer review and publication clear?

Paulo Roberto Benchimol-Barbosa

Publication Dates

• Publication in this collection
  04 Sept 2012
• Date of issue
  July 2012

History

• Received
  06 Mar 2012
• Accepted
  02 Apr 2012
• Reviewed
  06 Mar 2012