Systematic reviews and meta-analyses during the SARS-CoV-2 pandemic

Dear Editor,

The article “Meta-analyses on COVID-19: scoping review and quality analysis”,(¹) published in this journal, shows its relevance when planning meta-analysis and carrying out pertinent discussion about the role of this type of methodology in the current pandemic.

An essential instrument of evidence-based medicine, meta-analysis represents the top of the scale of level of evidence and grade of recommendation.(²)However, the emergency pandemic situation caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), identified in January 2020, brought several implications related to how we conduct, read and analyze systematic reviews and meta-analyses, since there were no original studies on the novel disease.

The lack of original studies while searching for the best evidence for decision-making, together with the high transmission rate and severity of the disease, influenced the use of non-traditional databases, such as preprint bases. Available updated materials are crucial elements in a disease that progresses daily. Preprint databases aimed to accelerate the reporting of research data, making it easier to gather information, but not relying on peer review. The absence of peer review makes publication of preprints susceptible of erroneous analysis,(³) such as the manuscript on reduction of hospital morbidity by use of hydroxychloroquine plus azithromycin, which was withdrawn by the authors after the controversy over the study design. Conventional databases, such as MEDLINE and Embase almost exclusively have articles published after peer review; however, they may take months to be published. As e.g., the media preprints database had most of the original works related to coronavirus disease 2019 (COVID-19) included in 2021.(⁴)

In addition, systematic reviews on SARS-CoV-2 quickly become outdated, favoring greater use of peculiar forms of review, including “rapid reviews”, “living systematic reviews”, and “scoping reviews”.

The rapid review is a synthesis of knowledge that aims to provide information with greater speed as compared to systematic reviews. In this process, the components are omitted or simplified to provide faster information for decision-making. There is a usual selection of works already known in the environment and the reviewer chooses the steps that will be limited, and later provide explanation with the effect of the methodological choice. However, this methodology allows for much bias when compared to systematic reviews, for simplifying this process.(⁵)

In general, a rapid review takes approximately four months or less and is considered useful in emergency situations, such as epidemics and disaster relief. As an example, the study by Wu et al.(⁶) provided a comprehensive overview of the safety profile of COVID-19 vaccines using rapid review. A systematic review, however, usually takes from 12 to 24 months to be carried out and is preferred in situations where evidence will be used to inform decisions, or in developing guidelines that will be implemented in large scale.

Another type of review that gained notoriety during the pandemic is the living systematic review. This is a systematic review that is frequently updated, including new relevant evidence as it emerges and keeping review findings constantly updated. It uses standard systematic review methods and maintains an explicit and a priori commitment to a predetermined frequency of research and review updates. In this sense, live systematic reviews are most appropriate when the field of research covered by the systematic review is moving quickly and new evidence is emerging, as observed with COVID-19. Siemieniuk et al.(⁷) performed a living systematic review on drug treatments for COVID-19, comparing the effects on the disease. Nonetheless, this type of study demands a continuous and active workflow, with coordinated effort over extended periods by the review team, searching for new articles.(⁸,⁹)

Pires et al.(¹) published a scoping review. This type of review is a research synthesis that aims to map the existing literature on a particular topic or research area to identify key concepts and knowledge gaps. It has a comprehensive character, and during the current pandemic, it has been useful to provide the attending physicians with the fundamental concepts on management of COVID-19. On the other hand, scoping review can often present generic ideas and requires physicians to investigate more about the topic, searching for other sources of evidence for a more informed approach.(¹⁰,¹¹)

It is worth mentioning these changes in the profile of the main review articles derived from the COVID-19 pandemic require a more critical view of readers and physicians. Critical evaluation is an evidence-based medicine tool for judging methodological deficiencies. The judgment of a meta-analysis in the context of a rapidly progressing pandemic should be intensified, determining the methodological quality of the study design and the level of bias in the analysis.(¹²)A critical view is important to know the advantages and disadvantages of using preprints, rapid reviews, living systematic reviews and scoping reviews. Considering the articles analyzed in the study by Pires et al.,(¹) it was found that, although the tool used identified some meta-analyses of better quality, most studies had mediocre quality and would bring more damage than benefits when used for decision-making. Literature reviews are crucial to identify the state-of-the-art and evidence gaps of the study object and enable targeting the specificities of study methodologies to offer more accurate answers, especially in the unprecedented condition of COVID-19.

REFERENCES

• ¹
  Pires GN, Bezerra AG, Oliveira TB, Chen SF, Malfatti VD, Mello VF, et al. COVID-19 meta-analyses: a scoping review and quality assessment. einstein (São Paulo). 2021;19:eAO6002. Review.
• ²
  Galvão TF, Pereira MG. Avaliação da qualidade da evidência de revisões sistemáticas. Epidemiol Serv Saúde. 2015;24(1):173-5. Review.
• ³
  King A. Fast news or fake news?: The advantages and the pitfalls of rapid publication through pre-print servers during a pandemic. EMBO Rep. 2020;21(6):e50817.
• ⁴
  Else H. How a torrent of COVID science changed research publishing - in seven charts. Nature. 2020;588(7839):553.
• ⁵
  Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13:224. Review.
• ⁶
  Wu Q, Dudley MZ, Chen X, Bai X, Dong K, Zhuang T, et al. Evaluation of the safety profile of COVID-19 vaccines: a rapid review. BMC Med. 2021;19(1):173. Review.
• ⁷
  Siemieniuk RA, Bartoszko JJ, Ge L, Zeraatkar D, Izcovich A, Kum E, et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ. 2020;370:m2980. Update in: BMJ. 2020;370:m3536. Update in: BMJ. 2020;371:m4852. Update in: BMJ. 2021;372:n858. Erratum in: BMJ. 2021;373:n967.
• ⁸
  Andrenelli E, Negrini F, de Sire A, Arienti C, Patrini M, Negrini S, Ceravolo MG; International Multiprofessional Steering Committee of Cochrane Rehabilitation REH-COVER action. Systematic rapid living review on rehabilitation needs due to COVID-19: update to May 31st, 2020. Eur J Phys Rehabil Med. 2020;56(4):508-14.
• ⁹
  Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I, Thomas J; Living Systematic Review Network. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol. 2017;91:23-30.
• ¹⁰
  Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371-85. Review.
• ¹¹
  Hanneke R, Asada Y, Lieberman L, Neubauer LC, Fagen M. The scoping review method: mapping the literature in “structural change” public health interventions. SAGE Research Methods. 2017:doi.org/10.4135/9781473999008.
  » https://doi.org/10.4135/9781473999008
• ¹²
  Zlowodzki M, Poolman RW, Kerkhoffs GM, Tornetta P 3rd, Bhandari M; International Evidence-Based Orthopedic Surgery Working Group. How to interpret a meta-analysis and judge its value as a guide for clinical practice. Acta Orthop. 2007;78(5):598-609.

Publication Dates

• Publication in this collection
  24 June 2022
• Date of issue
  2022

History

• Received
  4 Feb 2022
• Accepted
  13 May 2022