COVID-19 meta-analyses: a scoping review and quality assessment

ABSTRACT

Objective:

To carry out a scoping review of the meta-analyses published regarding about coronavirus disease 2019 (COVID-19), evaluating their main characteristics, publication trends and methodological quality.

Methods:

A bibliometric search was performed in PubMed^®, Scopus and Web of Science, focusing on meta-analyses about COVID-2019 disease. Bibliometric and descriptive data for the included articles were extracted and the methodological quality of the included meta-analyses was evaluated using A Measurement Tool to Assess Systematic Reviews.

Results:

A total of 348 meta-analyses were considered eligible. The first meta-analysis about COVID-19 disease was published on February 26, 2020, and the number of meta-analyses has grown rapidly since then. Most of them were published in infectious disease and virology journals. The greatest number come from China, followed by the United States, Italy and the United Kingdom. On average, these meta-analyses included 23 studies and 15,200 participants. Overall quality was remarkably low, and only 8.9% of them could be considered as of high confidence level.

Conclusion:

Although well-designed meta-analyses about COVID-19 disease have already been published, the majority are of low quality. Thus, all stakeholders playing a role in COVID-19 deseases, including policy makers, researchers, publishers and journals, should prioritize well-designed meta-analyses, performed only when the background information seem suitable, and discouraging those of low quality or that use suboptimal methods.

Keywords:
Bibliometrics; Coronavirus; COVID-19; Meta-analysis; Betacoronavirus; SARS-CoV-2; Scientometrics; Systematic review

RESUMO

Objetivo:

Realizar uma revisão de escopo das metanálises publicadas sobre a doença pelo coronavírus 2019 (COVID-19), avaliando suas principais características, tendências de publicação e qualidade metodológica.

Métodos:

Uma busca bibliométrica foi realizada em PubMed®, Scopus e Web of Science, com foco em metanálises sobre a doença pelo COVID-19. Foram extraídos dados bibliométricos e descritivos dos artigos incluídos, e a qualidade metodológica foi avaliada usando a ferramenta A Measurement Tool to Assess Systematic Reviews.

Resultados:

Um total de 348 metanálises foram consideradas elegíveis. A primeira delas foi publicada em 26 de fevereiro de 2020, e, desde então, o número dessas publicações cresceu rapidamente. A maioria foi publicada em periódicos de infectologia e virologia. Grande parte é proveniente da China, seguida dos Estados Unidos, da Itália e do Reino Unido. Em média, as metanálises incluíram 23 estudos e 15.200 participantes. Em geral, a qualidade metodológica foi baixa, e apenas 8,9% delas podem ser consideradas de algum grau de confiabilidade

Conclusão:

Embora algumas metanálises bem conduzidas sobre a doença pelo COVID-19 tenham sido publicadas, a maioria apresenta baixa qualidade. Todos os envolvidos na abordagem da doença pelo COVID-19, incluindo formuladores de políticas, pesquisadores, editoras e periódicos, devem dar prioridade a metanálises de alta qualidade, realizadas apenas quando os dados são viáveis, e desencorajar as de baixa qualidade ou conduzidas com métodos subótimos.

Descritores:
Bibliometria; Coronavírus; COVID-19; Metanálise; Betacoronavírus; SARS-CoV-2; Cientometria; Revisão sistemática

INTRODUCTION

Since coronavirus disease 2019 (COVID-19) was recognized as a serious public health threat, researchers from all over the world have devoted a great deal of time and effort to characterize and understand this new disease, resulting in an unprecedented surge in the number of publications.^(1–3) On average, more than 200 new articles about COVID-19 have been indexed in PubMed^®, every day.⁽⁴⁾

All stakeholders involved in academic research and scientific publishing have directed efforts to enable an efficient publication outflow: governments and supporting agencies are releasing special funding and grants for research about COVID-19; ethics committees and other regulatory agencies are prioritizing COVID-19-related projects;⁽⁵⁾ publishers are applying open access policies to COVID-19 articles; and journals are reviewing COVID-19 articles using fast-track processes.^(6,7)

All these efforts have the ultimate goal of enhancing knowledge and generating evidence about COVID-19. From an evidence-based perspective, meta-analyses are usually regarded as the experimental approach that generates the highest level of scientific evidence. Thus, not surprisingly, meta-analyses regarding COVID-19 are already being published. Prompted by this remarkable and constant growth in publication output, a discussion in respect of the quality and ethical standards of these articles has already begun.^(6,8–10)

Both the need to release and publish data rapidly, and the shortened peer-review times, can result in a reduction in the quality of the published reports. Since meta-analyses are also subjected to same publication environment, this reduction in quality might also be true for them. In order to quantify the problem and foresee potential drawbacks in the evidence synthesis of COVID-19 research, it is important to quantify the amount of meta-analyses being published, their characteristics, their methods and the average quality of these reports.

OBJECTIVE

To carry out a scoping review of the meta-analyses published regarding coronavirus disease 2019, evaluating their main characteristics, publication trends and methodological quality.

METHODS

A bibliometric search was performed in PubMed^®, Scopus and Web of Science, to retrieve all meta-analyses published relating to COVID-19. The search strategy, which was initially designed for PubMed^® and adapted to other databases, is described below. It was performed on August 18, 2020 and no publication filters were used. The protocol used was registered in the Open Science Framework (https://osf.io/tnps2/).

Search was performed using the following terms: (meta-analysis [publication type] OR meta-analysis as topic [mesh] OR meta-analysis [tiab] OR meta-analyses [tiab] OR meta-analysis [tiab] OR metaanalyses [tiab] OR meta-analysis [tiab] OR metanalyses [tiab]) AND (covid-19 [supplementary concept] OR covid-19 OR covid19 OR “novel coronavirus” OR “sars-cov-2” OR “2019-ncov”).

The resulting articles were screened and evaluated in a two-step process. In the first, titles and abstracts were reviewed by two independent authors. Discrepancies were resolved by consensus. Next, the full texts of the articles selected in the first step were evaluated by a single author per article and double-checked by a second reviewer. Only meta-analyses dealing with COVID-19-related issues were considered eligible. Regarding the PI(E)CO strategy, COVID-19 could be addressed as an intervention (I), exposure (E) or outcome (O), while there was no restriction regarding the population (P) or comparators (C). Six main exclusion criteria were applied: articles published before 2019; protocols of systematic reviews and meta-analyses; research designs other than meta-analyses (systematic reviews with no meta-analyses were excluded); meta-analyses not related to COVID-19; meta-analyses for which data came from sources other than previously published articles (e.g., meta-analyses of geographical or meteorological data), and articles for which the full text could not be found. Meta-analyses in which COVID-19 data were considered concurrently with other diseases, such as the severe acute respiratory syndrome (SARS) or the Middle East respiratory syndrome (MERS), were also considered eligible.

Data extraction was performed by a single author per article. The following data was extracted: bibliometric information (author, publication year, publication date, journal and country – based on the first affiliation of the first author), number of articles and individuals included in the meta-analyses, pre-publication registration (e.g., International Prospective Register of Systematic Reviews – PROSPERO), databases screened and research designs considered eligible (only randomized controlled trials (RCT), only non-randomized designs or both). The articles were classified into the same 13 categories used by PROSPERO to categorize COVID-19-related protocols of systematic reviews. Finally, the methodological quality of the included meta-analyses was evaluated using A Measurement Tool to Assess Systematic Reviews (AMSTAR 2.0).⁽¹¹⁾All authors applying tool had undergone training with a senior reviewer, and the evaluation criteria of each item were discussed and standardized in a meeting. To assure consistency and accuracy in data abstraction, two rounds of calibration were performed: in the first, all the authors involved in data extraction analyzed a set of ten articles from the sample. In the second round, a sample of 130 articles were evaluated by pairs of reviewers. Reviewers could start data extraction and quality assessment independently, only when consistency was assured.

All data were analyzed qualitatively, presented in a descriptive fashion when needed (percentages and absolute frequency for categorical data, and mean and standard deviation for numeric variables). This scoping review was performed in accordance with the Preferred Reporting Items for Systematic Reviews andMeta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines.⁽¹²⁾

The assignment of authors to each step in this systematic review is disclosed in the registered protocol (https://osf.io/tnps2/).

RESULTS

Out of 1,296 meta-analyses, 348 were considered eligible (Figure 1). The first article in this sample was indexed on February 26, 2020.⁽¹³⁾ In this short period since the publication of this article (176 days until August 18), an average of 1.95 meta-analyses were published per day. This number has been increasing, as shown in figure 2. Among these articles, 336 (96.5%) focused only on COVID-19, while 12 (3.4%) merged COVID-19 data with data for other diseases (SARS: 12; MERS: 11; Ebola and influenza: 1). The meta-analyses included an average of 22.94±21.49 studies (median: 16) and 15,250.57±46,876.16 participants (median: 4,121).

Figure 1
Flowchart of article identification and inclusion

Figure 2
Temporal analysis of publication output. Number of meta-analyses published per month is shown since the publication of the first meta-analysis (February 26). Articles whose publication date could not be determined, or that are scheduled for September 2020 onwards are not counted (n=329). Data from August should not be interpreted as a reduction in the number of meta-analyses published, as the search strategy was made on August 18, 2020

The studies were published in 193 different journals. The Journal of Medical Virology contributed the most, publishing 35 meta-analyses on COVID-19 (9.1%), followed by the Journal of Infection, with 19 articles (4.9%). Eight journals published between six and nine meta-analyses each, 39 published between two and four meta-analyses each, and the remaining 144 journals published a single meta-analysis each. Most meta-analyses were published in infectious disease or virology journals (78 articles in 15 journals), followed by clinical/internal/general medicine (38 articles in 19 journals), cardiology and vascular medicine (24 articles in 20 journals) and gastroenterology (19 articles in 14 journals) (Figures 3A and 3B).

Figure 3
Publication output per subject area and country. (A) Number of articles on each subject area; (B) Number of journals in each subject area; (C) Number of articles per country. The time since the first local case of COVID-19, the ability to include studies published in local languages, and the recent surge in the publication of meta-analyses in general may have contributed to China being ranked as producer of the largest number of analyses, with an output double that of the United States

A total of 37 countries were listed in the publication output (Figure 3C). China was the most productive country, with 117 articles (33.6%), followed by the United States with 54 articles (15.1%), and Italy and the United Kingdom, with 22 articles each (12.6%).

Only 58 meta-analyses (16.7%) registered a protocol, being PROSPERO the most commonly used protocol registry. Six meta-analyses (1.6%) included only RCT, while the remaining were based on observational studies, or on a combination of multiple research designs.

Regarding the search strategies of the meta-analyses, 107 of them (30.7%) screened five or more databases; while 23 (6.6%) screened a single database, a non-recommended practice due to the increased risk of selection bias. PubMed^® and MEDLINE^® were the most frequently employed database (342 articles; 98.3%), followed by Embase (204 articles; 58.6%), Web of Science (130 articles; 37.4%), Cochrane Central (129 articles; 37.1%), and Scopus (94 articles; 27.0%). Pre-print sources, such as medRxiv and bioRxiv, were used in 74 articles (21.3%). Regional databases from China were used in 74 articles (21.3%).

The most common focuses of the included meta-analyses were disease prognosis (200 articles; 57.5%), epidemiology (130 articles; 37.4%), diagnosis (48 articles; 13.8%), health impacts (43 articles; 12.4%) and treatments (42 articles; 12.1%).

Quality assessment using AMSTAR 2.0 revealed that only 31 of the meta-analyses (8.9%) were of a quality that allowed a high overall confidence in the results of the review, while 186 (53.4%) were of a critically low level. Quality analyses according to each item in AMSTAR 2.0 are shown in figure 4.

Figure 4
Quality assessment of meta-analyses about COVID-19 using A Measurement Tool to Assess Systematic Reviews. (A) Number of meta-analyses according to different levels of confidence; (B) Evaluation of each criterion included in A Measurement Tool to Assess Systematic Reviews. Green means that an item was adequately addressed; yellow means it was partially addressed, and red means it was not addressed

DISCUSSION

Meta-analyses are regarded as the highest level of evidence. Therefore, healthcare decision makers frequently rely on them for evidence-based guidance.⁽¹⁴⁾ In the current pandemic situation, healthcare practices and policies need to be developed in a rapid fashion, making meta-analyses an important source of evidence. However, it must be kept in mind that the quality of the meta-analyses does not rely solely on the methodology used, but also on the number and quality of the articles included. Thus, the main question to answer is: do we have enough high-quality studies to perform meta-analyses about COVID-19?.

The building blocks of meta-analyses are the studies they include. The reasonability about performing a meta-analyses depends on the number of articles being analyzed, and the robustness of its results depends on the quality of the articles included (according to the concept of “garbage in, garbage out”). Appropriate statistical methods need to be employed in meta-analyses. Indeed, increasing the amount of low-bias meta-analyses (i.e. those with reliable methods and robust methodology) might increase the proportion of reliable scientific findings overall.⁽¹⁵⁾ Although the total number of articles about COVID-19 is considerable, questions have been raised about the overall quality of the studies,^(6,8–10) and in respect of meta-analyses, some cases of methodologically flawed studies have already been reported.^(16,17)

The meta-analyses included in this study comprise a fair amount of data, with an average of 23 studies and 15,000 participants per article. This is a surprising finding, since we would expect the meta-analyses published to be much smaller, given that they were published a short time after the start of the pandemic. It probably reflects the fact that epidemiological studies on COVID-19, with very large cohorts, were published relatively soon after the initial outbreak. However, although there does not seem to be lack of data, the quality of these meta-analyses is remarkably low, with only 31 (8.9%) considered to be of high quality.

It should also be noted that no meta-analysis based on RCT only has yet been published. This is explained by several meta-analyses performed on topics that are only solvable by observational studies (such as prevalence of risk factors). However, meta-analyses about themes that depend on properly performed RCT (such as pharmacological interventions and vaccines) do not seem feasible in the short term.

The sources and topics of the meta-analyses also provided interesting results. More than 50% of studies come from only four countries: China, United States, Italy and United Kingdom. Although these countries are among those most affected by COVID-19, it would be misleading to make this correlation, since they were already among the top six meta-analyses producers, even before the pandemic.⁽¹⁸⁾ It is interesting to note that China's production is almost twice that of the United States, probably reflecting the geometric increase in their meta-analyses publication output, as from 2009.⁽¹⁹⁾

Most of the meta-analyses were published in virology and infectious disease journals, addressing prognostic factors and epidemiology. This is to be expected, given that these were the first topics for which a reasonable amount of data became available. Although PROSPERO has already been registering systematic reviews about treatments and vaccines, meta-analyses on these topics seem rather unfeasible in the short term (and will be less informative if based on the limited number of completed the RCT available).

One positive aspect of the included meta-analyses is the inclusion of pre-print sources (18.0%) and national databases (mostly Chinese; 22.5%). It is estimated that about a quarter of all relevant evidence about COVID-19 is not in PubMed^®, but might be available in pre-print sources⁽²⁰⁾ and not including them represents an obvious publication bias. The same is true in respect of the inclusion of national databases, since a fair amount of data about epidemiology and clinical description are available only in Chinese.

The overall number of meta-analyses being published has increased considerably in recent years,⁽²¹⁾ and this mass publication of meta-analyses has been heavily criticized.^(22,23) Among the main problems relating to the increased number of meta-analyses being published are duplicated efforts, conflicting results, low quality and limited practical value.⁽²²⁾ While our study focused on an analysis of the quality of the studies, one might expect these same problems are also present in COVID-19-related meta-analyses. One example of conflicting results in meta-analyses about COVID-19 was found in respect of smoking. A meta-analysis published in May 2020 concluded that smoking does not represent a risk factor for severe cases of COVID-19.⁽²⁴⁾However, two subsequent meta-analyses highlighted methodological errors in the original analysis, and came to an opposite conclusion.^(16,25) Since then, several other systematic reviews have been performed in relation to smoking, including a recent meta-analysis performed using robust methods (a living review with Bayesian meta-analysis, currently in its seventh edition).⁽²⁶⁾ Its conclusions reinforced the complexity of this association, since they found current smokers are at reduced risk of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, but former smokers are at increased risk of severe cases and death from COVID-19. Thus, despite the growing number of meta-analyses about COVID-19 inconsistencies, low quality and methodological weaknesses among them might reduce their potential to generate applicable findings. This is particularly important for policy-makers and health care managers, who might wish to use meta-analyses to drive their practices and policies.

The idea that meta-analyses represent the unequivocal highest level of medical evidence is outdated,⁽²⁷⁾ and this affirmation can certainly not be applied to all individual meta-analyses. Weak or imprecise meta-analyses actually produce less evidence than, for instance, a well conducted large observational study or a proper multicenter randomized controlled trial. Based on these considerations and the results of our review, we make the following recommendations to different stakeholders about how to deal with meta-analyses regarding COVID-19.

To policy makers: meta-analyses are undeniably important for policy makers, so that they can design and implement policies employing on reliable evidence-based information. However, current meta-analyses about COVID-19 are of limited quality and, in some cases, have discrepant results. Thus, in these times of uncertain evidence, decision-making should not blindly rely on meta-analyses, but rather on a deep analysis of literature and the evidence it contains.

To researchers and authors of meta-analyses: new meta-analyses about COVID-19 are welcome, as long as they provide new high-quality evidence. Low-quality meta-analyses (premature, redundant, dubious, inconclusive, or performed with suboptimal practices) are counterproductive, for impairing proper evidence synthesis. Thus, we suggest that prospective authors of meta-analyses perform a serious critical appraisal about the need for their meta-analyses. The following factors should be evaluated: whether there is enough data to perform a meta-analysis; if the available data seems reliable, comparable and free from heterogeneity; whether other meta-analyses in the same field have already been published or protocols have been registered. Researchers should rely on a search of protocol registrations for systematic reviews and meta-analyses to avoid duplicated and redundant efforts. Quality and methodological robustness should not be overlooked in favor of rapid publication, or it is likely that the quality of the review will be poor (as seen in our results).

To journals and publishers: meta-analyses usually lead to a fair amount of citations and good visibility to journals publishing them.⁽²⁸⁾ Thus, in the current situation, journals might feel tempted to publish meta-analyses, while ignoring faulty methods and low-quality procedures. Interestingly, as the number of articles about COVID-19 has increased, so has the number of articles being withdrawn and retracted.⁽²⁹⁾Journals should comply with their social role of being a media for evidence-based science. Recent discussions have highlighted the role of journals in promoting the publication of high-quality meta-analyses, while discouraging those that do not follow optimal methodological standards.⁽³⁰⁾ This is especially relevant in the current situation. Peer review can be made faster, as long as its quality and fairness are not compromised. We would suggest that a researcher with experience in meta-analyses is always included as a reviewer, or to have a statistician on the editorial board, to ensure that faulty or low-quality meta-analyses are not published.

A few points need to be considered for a proper interpretation of our study: this is a scoping review, rather than a systematic review. Thus, rather than aiming to perform a meta-analysis ourselves, our review was intended to describe a panorama about the current meta-analyses about COVID-19, focusing on the general framework of the field, instead of on the individual aspects of each study; - we decided to exclude systematic reviews with no meta-analyses. This is because our focus was on evidence synthesis, which is much reduced in reviews not including meta-analyses, and also, in many cases, the term “systematic review” is misused.⁽³¹⁾ However, an enormous amount of systematic reviews has been published, whose quality has already been appraised in a previous article.⁽³²⁾ This is a time-restrained review. In the near future, meta-analyses on most aspects of COVID-19 will probably become viable. However, a study focusing attention on the quality of meta-analyses published so far in respect of COVID-19 is timely; and the call to prioritize attention on high-quality meta-analyses is important.

CONCLUSION

Meta-analyses are an important tool in evidence-based medicine. The number of meta-analyses in respect of COVID-19 is increasing and will continue to rise. We believe that meta-analyses will play an important role in COVID-19, defining its epidemiological characteristics, the most suitable treatments, and supporting policy-making. However, while we do support the need for meta-analyses about COVID-19, it should not be taken as a license to produce them indiscriminately.

Although some very well-designed meta-analyses about COVID-19 have already been published, the majority published so far are of poor quality, and are likely to causes more harm than good. Few meta-analysis about COVID-19 in our review could be considered as of high quality, while more than a half were considered of critically low quality. Thus, all stakeholders playing a role in COVID-19 research and publication (including policy makers, researchers, publishers and journals) should focus on proper evidence-based research, supporting well designed meta-analyses, performed only when the background information seems suitable, and discourage those of low quality or that use suboptimal methods.

ACKNOWLEDGMENT

Gabriel Natan Pires is supported by grants and funding from the Associação Fundo de Incentivo à Pesquisa (AFIP). Sergio Tufik and Monica Levy Andersen are supported by grants and funding from AFIP, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES - Finance Code 001) and the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). Samuel Fen I Chen, Thainá Baenninger de Oliveira and Victor Luiz Selva Pinto are supported by grants from CAPES.

REFERENCES

• ¹
  Lou J, Tian SJ, Niu SM, Kang XQ, Lian HX, Zhang LX, et al. Coronavirus disease 2019: a bibliometric analysis and review. Eur Rev Med Pharmacol Sci. 2020;24(6):3411-21.
• ²
  Škorić L, Glasnović A, Petrak J. A publishing pandemic during the COVID-19 pandemic: how challenging can it become? Croat Med J. 2020;61(2):79-81.
• ³
  Kambhampati SB, Vaishya R, Vaish A. Unprecedented surge in publications related to COVID-19 in the first three months of pandemic: a bibliometric analytic report. J Clin Orthop Trauma. 2020;11(Suppl 3):S304-6.
• ⁴
  Gale RP. Conquest of COVID-19. Publish it to death? Br J Haematol. 2020; 190(3):358-60.
• ⁵
  Zhang H, Shao F, Gu J, Li L, Wang Y. Ethics Committee Reviews of applications for research studies at 1 hospital in China during the 2019 novel coronavirus epidemic. JAMA. 2020;323(18):1844-6.
• ⁶
  Smith MJ, Upshur RE, Emanuel EJ. Publication ethics during public health emergencies such as the COVID-19 pandemic. Am J Public Health. 2020:e1-e2. doi: 10.2105/AJPH.2020.305686. [Epub ahead of print].
  » https://doi.org/10.2105/AJPH.2020.305686
• ⁷
  Treweek S, Jüni P, Li T, Collin J, Briel M, Chan AW, et al. COVID-19 randomised trial protocols: rapid publication without barriers. Trials. 2020;21(1):327.
• ⁸
  Peyrin-Biroulet L. Will the quality of research remain the same during the COVID-19 pandemic? Clin Gastroenterol Hepatol. 2020;18(9):2142.
• ⁹
  Shah SS, Kulkarni N, Mahant S. Rapid publication, knowledge sharing, and our responsibility during the COVID-19 pandemic. J Hosp Med. 2020;15(5):261.
• ¹⁰
  Alexander PE, Debono VB, Mammen MJ, Iorio A, Aryal K, Deng D, et al. COVID-19 coronavirus research has overall low methodological quality thus far: case in point for chloroquine/hydroxychloroquine. J Clin Epidemiol. 2020;123:120-6. Review.
• ¹¹
  Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.
• ¹²
  Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169(7):467-73.
• ¹³
  Sun P, Qie S, Liu Z, Ren J, Li K, Xi J. Clinical characteristics of hospitalized patients with SARS-CoV-2 infection: a single arm meta-analysis. J Med Virol. 2020;92(6):612-7.
• ¹⁴
  Mulrow CD. Rationale for systematic reviews. BMJ. 1994;309(6954):597-9.
• ¹⁵
  Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
• ¹⁶
  Guo FR. A flaw on a meta-analysis of smoking and the severity of COVID-19: the association should have been endorsed. J Public Health (Oxf). 2020; 42(3):653-4.
• ¹⁷
  Santamaría-Gadea A, de Los Santos G, Alobid I, Mullol J, Mariño-Sánchez F. Errors and biases in meta-analysis of the prevalence of olfactory dysfunction in patients with COVID-19. Otolaryngol Head Neck Surg. 2021;164(2):455-6.
• ¹⁸
  Fontelo P, Liu F. A review of recent publication trends from top publishing countries. Syst Rev. 2018;7(1):147. Review.
• ¹⁹
  Ioannidis JP, Chang CQ, Lam TK, Schully SD, Khoury MJ. The geometric increase in meta-analyses from China in the genomic era. PLoS One. 2013; 8(6):e65602.
• ²⁰
  Shokraneh F, Russell-Rose T. Lessons from COVID-19 to future evidence synthesis efforts: first living search strategy and out of date scientific publishing and indexing industry (submitted). J Clin Epidemiol. 2020;123:171-3.
• ²¹
  Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018;555(7695):175-82. Review.
• ²²
  Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485-514.
• ²³
  Chevret S, Ferguson ND, Bellomo R. Are systematic reviews and meta-analyses still useful research? No. Intensive Care Med. 2018;44(4):515-7.
• ²⁴
  Lippi G, Henry BM. Active smoking is not associated with severity of coronavirus disease 2019 (COVID-19). Eur J Intern Med. 2020;75:107-8.
• ²⁵
  Guo FR. Active smoking is associated with severity of coronavirus disease 2019 (COVID-19): an update of a meta-analysis. Tob Induc Dis. 2020;18:37.
• ²⁶
  Simons D, Shahab L, Brown J, Perski O. The association of smoking status with SARS-CoV-2 infection, hospitalization and mortality from COVID-19: a living rapid evidence review with Bayesian meta-analyses (version 7). Addiction. 2020 Oct 2:10.1111/add.15276. Review.
  » https://doi.org/10.1111/add.15276
• ²⁷
  Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21(4):125-7.
• ²⁸
  Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005;293(19):2362-6.
• ²⁹
  Agoramoorthy G, Hsu MJ, Shieh P. Queries on the COVID-19 quick publishing ethics. Bioethics. 2020;34(6):633-4.
• ³⁰
  Annane D, Jaeschke R, Guyatt G. Are systematic reviews and meta-analyses still useful research? Yes. Intensive Care Med. 2018;44(4):512-4.
• ³¹
  Paul M, Leibovici L. Systematic review or meta-analysis? Their place in the evidence hierarchy. Clin Microbiol Infect. 2014;20(2):97-100. Review.
• ³²
  Pericàs JM, Torrallardona-Murphy O, Arenas A, Valero H, Nicolás D. Profile and quality of published reviews on COVID-19. Eur J Clin Invest. 2020 May 30:e13293. doi: 10.1111/eci.13293. [Epub ahead of print].
  » https://doi.org/10.1111/eci.13293

Publication Dates

• Publication in this collection
  15 Mar 2021
• Date of issue
  2021

History

• Received
  09 July 2020
• Accepted
  30 Oct 2020