A GUIDE FOR SYSTEMATIC REVIEWS OF OBSERVATIONAL STUDIES

GUÍA PARA REVISIONES SISTEMÁTICAS DE ESTUDIOS OBSERVACIONALES

ABSTRACT

Objective:

to describe a guide for the development of systematic reviews of observational studies and systematization of international guidelines and tools, focusing on diverse evidence for innovation and clinical practice.

Method:

this theoretical-conceptual study was initiated during the development of a systematic review with meta-analysis of observational studies, using international guidelines and tools.

Results:

a guide was constructed to develop systematic reviews of observational studies. Diverse information about several stages and requirements for conducting a systematic review based on international guidelines and tools was systematized, aiming to ensure scientific rigor in manuscripts written by professionals from the health area.

Conclusion:

this study contributes to research in the health area by innovatively synthesizing guidance on the systematic review method and approaches. The references herein used serve as a starting point for understanding the procedures and international tools necessary for a systematic review of observational studies.

DESCRIPTORS:
Systematic review; Data analysis; Study guide; Methods; Research in Nursing

RESUMEN

Objetivo:

describir una guía para desarrollar revisiones sistemáticas de estudios observacionales y sistematizar pautas y herramientas internacionales, concentrándose en evidencias para la innovación y la práctica clínica.

Método:

este estudio teórico-conceptual se inició mientras se desarrollaba una revisión sistemática con metaanálisis de estudios observacionales en la que se utilizaron directrices y herramientas internacionales.

Resultados:

se elaboró una guía para desarrollar revisiones sistemáticas de estudios observacionales. Se sistematizó diversa información sobre varias etapas y requisitos para realizar una revisión sistemática basada en directrices y herramientas internacionales, con el objetivo de garantizar el rigor científico en los manuscritos redactados por profesionales del área de salud.

Conclusión:

este estudio representa un innovador aporte a la investigación en el área de salud porque sintetiza las pautas sobre el método y los enfoques de las revisiones sistemáticas. Las referencias utilizadas en este trabajo son un buen punto de partida para comprender los procedimientos y las herramientas internacionales necesarios para una revisión sistemática de estudios observacionales.

DESCRIPTORES:
Revisión sistemática; Análisis de datos; Guía de estudio; Métodos; Investigación en Enfermería

RESUMO

Objetivo:

descrever um guia para o desenvolvimento de revisões sistemáticas de estudos observacionais, sistematização de orientações e ferramentas internacionais, com foco em evidências para inovação e prática clínica.

Método:

este estudo teórico-conceitual foi iniciado durante o desenvolvimento de uma revisão sistemática com metanálise de estudos observacionais em que foram usadas diretrizes e ferramentas internacionais.

Resultados:

foi construído um guia para desenvolver revisões sistemáticas de estudos observacionais. Foram sistematizadas informações sobre várias etapas e requisitos para realizar revisão sistemática pautada em diretrizes e ferramentas internacionais, visando rigor científico nos manuscritos redigidos por profissionais da área da saúde.

Conclusão:

este estudo é uma contribuição à pesquisa na área da saúde que inova ao sintetizar a orientação sobre o método e abordagens de revisão sistemática. As referências aqui usadas são um ponto de partida para compreender os procedimentos e ferramentas internacionais necessários a uma revisão sistemática de estudos observacionais.

DESCRITORES:
Revisão sistemática; Análise de dados; Guia de estudo; Métodos; Pesquisa em enfermagem

INTRODUCTION

Knowledge systematization through systematic reviews (SRs) is a scientific foundation for innovating clinical care practices, justifying the reevaluation and updating of care actions, organizing and managing services, curricula and educational practices in the health area, as well as justifying investments to improve health services and the population's quality of life.

Thus, it is necessary to thoroughly understand the studies that present the components of this construction1.

The lack of studies with proper planning and application of outdated references, both nationally and internationally circulated, can lead to errors from the planning stage to the analysis of evidence and its contributions to the clinical practice.

SRs on risk factors and prognosis oftentimes include comparative observational studies because randomized controlled trials are inadequate or impractical for addressing certain types of questions, although they are the ideal research design for evaluating intervention effects. Some of the reasons follow2^-3: (1) ethical implications of randomizing patients to expose them to potential risk factors; (2) some outcomes or diseases may occur after inclusion of the study population for extended follow-up periods; and (3) some individuals may refuse to be exposed depending on the exposure and period.

Therefore, observational studies are applied to study exposures that are difficult or impossible to conduct in randomized controlled trials (RCTs), for example, air pollution or smoking. Moreover, observational studies are the most suitable for studying long-term causes and latency periods, such as the carcinogenic effects of environmental exposure or drugs. The most commonly used designs to evaluate risk factors are cohort and case-control studies2.

Systematic reviews of observational studies can provide information on the association between exposure and outcome4. The inclusion and exclusion criteria must be clearly defined and consistently applied to all the studies included. The criteria should be defined with a focus on the research question and the characteristics (exposure and outcome) of the study population. The quality of the studies included should be assessed using appropriate tools and criteria, and the assessment should be used to inform the analysis and interpretation of the results.

Therefore, the objective of this study was to produce a guide for developing systematic reviews of observational studies and systematizing international guidelines and tools with a focus on evidence.

METHOD

This theoretical-conceptual research was grounded on the literature and the process of using tools. It was developed during the development of a systematic review of observational studies on maternal mortality in the international context. International guidelines were used to rigorously develop the method and the use of tools.

SYSTEMATIC REVIEWS

When conducting an SR, its authors should consult methodological guidelines produced by international organizations that recommend methods and operational procedures: Cochrane3, Joanna Briggs Institute5, Enhancing the Quality and Transparency of Health Research (EQUATOR) Network6 or Grading of Recommendations Assessment, Development and Evaluation (GRADE)7, for example.

The results from systematic reviews provide an overview of the current state of evidence and gaps in knowledge, highlighting the strengths and limitations of individual studies (or of teams), and guide investments necessary for further research. The result of a systematic review is knowledge about a specific exposure factor (qualitative synthesis), whereas the statistical method used (meta-analysis) summarizes the association measure of results from different studies (quantitative synthesis)3.

Research studies should follow procedures to ensure scientific rigor and enable replication, contributing to reassert or refute paradigms and guidelines. In systematic reviews, the method requires rigor according to international guidelines because the research must answer a specific question. Quality and validity must be identified, selected and critically evaluated to synthesize a set of published scientific evidence, thus obtaining an overall overview of the topic under study and the probability of applying the findings3^,8.

The method

The method to be applied in the development of systematic reviews and meta-analyses should be based on international guidelines, highlighting those produced and disseminated by the Cochrane Handbook for Systematic Reviews of Interventions3, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)10 and Conducting Systematic Reviews and Meta-Analyzes of Observational Studies of Etiology (COSMOS-E)2.

Following one of the prerogatives of the guidelines for conducting SRs3, the research team should consist of at least two researchers. For quality and success of an SR, it is important that the team of researchers gathers a set of competencies that contribute to conducting the systematic review, including critical analysis, clinical knowledge, use of tools, knowledge and management of computer resources, and application of translators.

When defining the research question, the research method and the selection of guidelines, the researchers must be clear about the process. This includes an exhaustive and rigorous literature search, planning, construction and registration of the protocol, as well as documentation from planning to publication of the results.

In systematic reviews with meta-analysis, the statistical method used should include appropriately selecting the effect size metric, assessing heterogeneity and conducting sensitivity analyses to evaluate robustness of the results, quality of the studies included, and potential bias or confounding sources, as identified in more recent systematic reviews with meta-analysis9.

The stages

The nine stages for conducting a systematic review are illustrated below (Figure 1); the meta-analysis will be included in another manuscript.

Figure 1 -
Guide for Systematic Reviews of Observational Studies.

Stage 1 - Objective of the systematic review

As a first step, the researcher must clearly understand the problem they wish to study. Thus, the purpose and objectives must be defined, including a justification regarding potential knowledge gaps, contextualization and the reasons for its execution. In addition to that, the existence of an equivalent systematic review to the desired one should be checked in health area databases (e.g., Medline/PubMed and Cochrane Library) and in the PROSPERO platform where ongoing systematic review protocols are registered (https://www.crd.york.ac.uk/prospero). Therefore, every researcher should look for prior research studies and technologies related to the topic.

Stage 2 - Planning and protocol

Planning of an SR should begin with the systematization of a protocol that should be registered on a public domain platform. The protocol is a document where all stages and procedures to be followed (and previously known to the researchers involved) are described in detail to ensure consistency in execution of the review and the fulfillment of the commitments made. Once written, it should be checked against the requirements of the platform where it will be registered. Proper registration is an important attribute conferred by international institutions. In planning, the SR protocol is essential to ensure integrity of the research and transparency of the process. In addition to that, the protocol can reduce arbitrariness in decisions regarding data extraction and utilization, anticipate potential issues, avoid selection bias, reduce redundancy of efforts and enhance collaboration when available11^-12.

Systematic reviews produced in collaboration with Cochrane, Campbell Collaborations and the Joanna Briggs Institute require the preparation of a protocol. In publications of completed systematic reviews, it is uncommon to report on other organizations that do not require a protocol. If a journal does not mandate protocol registration (or if it has been previously published), this does not absolve the researcher and their team from the obligation to prepare a protocol, delineating all research procedures. In theses and dissertations, it is advisable for the protocol to be an appendix so that reliability and reproducibility of the review can be ensured by the same researcher or by another. Protocol registration is also a commitment of the researcher to continue the studies, updating them periodically.

Ensuring quality in researchers' output involves several requirements for publishing different types of studies, defining a standard of operation and description of the method to be used. We highlight the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) international network (https://www.equator-network.org/), which produces guidelines for reporting and provides tools to improve the reliability, transparency and quality of published research studies, such as the PRISMA statement (published from 2010 onwards and periodically updated)10^-11.

From the need to prepare and register systematic review protocols to increase availability and accessibility of the methods and reduce duplication of efforts and publication bias, the International Prospective Register of Systematic Reviews (PROSPERO) registration platform (https://www.crd.york.ac.uk/prospero/) was created, which is hosted and maintained by the Center for Reviews and Dissemination (CRD) at York University and is funded by the United Kingdom National Institute for Health Research (NIHR)12^-13.

Systematic reviews in health and social care are registered on the PROSPERO platform, where protocols are prospectively registered and maintained as a permanent record. A comprehensive list of publicly and freely accessible systematic reviews is provided for consultation and registration, aiming to minimize the risk of bias in the studies, promote transparency in the process and avoid inadvertent duplications13.

To elaborate the protocol, the researcher must provide the required items in PROSPERO and follow the definitions set forth in the Preferred Reporting Items for Systematic Review and Meta-Analyses Protocols (PRISMA-P 2015), aimed at improving the quality of systematic review protocols. Its effect is similar to the positive impact achieved by other reporting guidelines, as it summarizes and documents the development of review protocols. The PRISMA-P checklist contains 17 items and 9 sub-items. The items are classified into three main sections: Administrative information, Introduction and Methods11^-12.

Stage 3 - Formulation of the research question and search strategy

The researcher should consider the socially relevant topic from different perspectives regarding subtopics, research methods and design, theoretical approach and research objective. For this, they need to thoroughly study the chosen topic within what has already been published in the global context.

Thus, the researcher must present the research details in the literature, explaining and justifying its scope. The search should be comprehensive and exhaustive within the published studies on the topic in question3^,14.

When defining the question and/or hypothesis to be addressed in the research (and in the systematic review rationale), they must be clear, and the inclusion and exclusion criteria for selecting the studies to be reviewed must be explicitly described. The search strategy (encompassing the variables of interest) should be rigorously developed, with high sensitivity to find all publications on the topic, using several databases (such as Medline/PubMed, Scopus or Embase) and following recognized international standards15.

The most useful components for finding the correct information can be identified using the keywords in the PICO acronym. This clarifies the research objective and helps select relevant studies. Different types of SRs are aligned with review questions in the PICO format (Population, Intervention, Comparator and Outcome). However, this acronym was initially intended for SRs of studies with randomized controlled trial designs. Therefore, it should be adapted according to the type of SR13.

For each type of systematic review, variations of the PICO strategy are defined, namely: Efficacy reviews (PICO); Experiential (qualitative) reviews (PICo); Prognostic reviews (PFO) and Methodology reviews (SDMO); Cost/Economic Evaluation reviews (PICOC); Prevalence and/or Incidence reviews (CoCoPop); Diagnostic Test Accuracy reviews (PIRD); Etiology and/or Risk reviews (PEO); and Expert Opinion/Policy reviews (PICo). In these strategies, the letter S (Study design) can be added, indicating the types of studies that will be included in the review13^,16.

The research question and search strategy should be formulated using Medical Subject Headings (MeSH) terms, resorting to controlled terms, free-text terms and truncation, included in an advanced form in the databases defined in the protocol. Afterwards, the search strategy must be planned to accommodate the different terms in each database included in the review.

In the latest PRISMA update10, the search item was modified to recommend that researchers present the search strategies for all databases, registries and websites searched item 7 of the checklist).

Below we exemplify the Boolean operators (delimiters) represented by the AND (restrictive combination) and OR (additive combination) connectors used with the descriptors, along with the search period definition. When using the PEOS strategy (P: Population/Participants; E: Exposure; O: Outcome and S: Study design/Type of study). Chart 1 presents an example for better understanding by the readers.

Chart 1 -
Example of a search strategy used in the Medline/PubMed database (Florianópolis, Santa Catarina; 2023).

PEOS strategy
(P) Population Study population	(“woman” OR “Women” OR “pregnant woman” OR “Pregnant Women” OR “Mothers” OR “women health” OR “maternal health” OR “Pregnant Women” OR “Pregnancy” OR “pregnant” OR “maternal” OR “Maternal Behavior” OR “Materna” OR “Maternal Age” OR “Maternal Health Services” OR “Maternal-Child Health Services”) AND
(E) Exposure Risk factors	(“risk factor” OR “Risk Factors” OR “population at risk”) AND
(O) Outcome Outcomes of interests	(“maternal death” OR “maternal mortality” OR “Maternal Mortality” OR “Pregnancy Complications” OR “obstetric complications”) AND
(S) Study design Including observational studies	(“epidemiologic studies” OR “case control studies” OR “cohort studies” OR “case control” OR “cohort study” OR “cohort studies” OR “cohort” OR “follow up study” OR “follow up studies” OR “observational study” OR “observational studies” OR “longitudinal” OR “retrospective” OR “cross sectional” OR “cross-sectional studies”)

The inclusion criteria must be detailed; they are essential in the protocol, which will be constantly consulted by the researcher while screening the studies. They should record details of the population and outcomes of interest that address the research question, with a clear definition of the clinical and epidemiological context in which the outcome of interest was evaluated in the studies to be included and the type of study design most suitable for inclusion in the SR. The exclusion criteria can range from restrictions on publication language, type of study and/or publication, clinical and epidemiological contexts that were not the SR focus, among others previously defined by the researchers17.

Stage 4 - Screening of the studies

This stage (also known as screening for inclusion) requires the researcher to provide a clear description of which studies were considered for review and which were removed. For removed/excluded studies, the reasons for their non-inclusion should be recorded, justifying why the resulting review is still comprehensive.

The articles found are exported to the EndNote bibliographic manager, v. X9 desktop (https://endnote.com/), commercial version; however, it is possible to use the free version called EndNote web (https://www.myendnoteweb.com/). This manager is used to select the primary studies in the systematic review, providing resources for knowledge synthesis methods, with transparency and reproducibility18. Other bibliographic managers can be used, such as Mendeley (https://www.mendeley.com/?interaction_required=true) and Zotero (https://www.zotero.org/).

In the first screening stage, duplicate articles are excluded, generating an export file for a free tool (https://www.rayyan.ai/). In this program, the titles and abstracts of potentially relevant studies are independently selected by two researchers, observing adherence to the research question and the inclusion and exclusion criteria previously defined in the protocol, with title and abstract reading to select the articles.

Rayyan is a free web-based tool designed to assist researchers working on systematic reviews and other knowledge synthesis projects to accelerate the study screening and selection process19.

In the Rayyan platform, the articles are identified for full-text reading (by two independent and blinded reviewers). Once the selection process is complete, blinding is disabled and any and all divergent decisions regarding selection of the articles are solved in a conciliation meeting with the presence of a third researcher not involved in the selection to solve conflicts and any tie-breaking in the final selection of the articles to be included in the review.

The researcher needs to study the method to identify potential threats to the validity of the studies. One of the main risks in observational studies is selection bias, whereby there may be large observed and unobserved differences due to non-randomization in the characteristics of the patients between case and control groups. Such differences can lead to biased estimates of treatment effects when one or more characteristics of the cases present differences (confounders) related to the outcomes being measured20.

Following the PRISMA recommendation, the researcher should describe the entire study screening process (including the exclusion reasons) in the Results section, using a flowchart as per the model available on the following website: http://prisma-statement.org/prismastatement/flowdiagram.aspx?AspxAutoDetectCookieSupport=110.

Stage 5 - Data extraction

After identifying all studies to be included in the review, the researchers must systematically extract the data that had been predefined for the SR from each of the materials. We recommend preparing a table for data extraction from the studies, containing information such as study design, characteristics of the population, data source, interventions, outcomes and results.

To develop this stage, the authors should construct a matrix (which should be attached to the review protocol) to make clear which data they want to collect. In turn, this should demonstrate the researchers' clarity, clinical expertise on the research topic and prior literature review.

The following will be included in this extraction matrix: eligibility criteria for studies based on predefined inclusion and exclusion criteria; outcomes and variables of interest; nomenclature and definition; prior literature review; dichotomous definition for the categories of variables; definition of risk groups and reference, their measurement and assessment; data for conducting the meta-analysis (number of cases and controls, Odds Ratio [OR], adjusted Odds Ratio [OR_a], confidence intervals); confounders adjusted in the studies; and data for subgroup analysis and meta-regression (country, economic level, clinical and geographical context, year of publication, study duration).

It is highly recommended to use specific software such as a review management tool to assist in data management and maintain consistent and standardized records of the decisions made throughout the review, for example: Covidence (https://www.covidence.org/home), RevMan (https://revman.cochrane.org/ - /myReviews) and EPPI-Reviewer. Such tools offer a central repository for review data that can be accessed remotely via a website. We highlight functionalities such as recording independent assessments of studies for inclusion, risk of bias and data extraction3.

EPPI-Reviewer Web (https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2914) is a software program for managing and analyzing data for all types of systematic reviews. It has features for both qualitative and quantitative analyses, such as meta-analysis and thematic synthesis, which can make the systematic review more efficient. It was developed to support the EPPI-Centre work program in conducting systematic reviews across a wide range of public policy areas. This service is provided non-profit by United Kingdom partners to support authors conducting systematic reviews. Its authorship is associated with the research unit of the Education Institute at the University of London21.

Data extraction should be carried out by two independent researchers. For this purpose, we recommend EPPI-Reviewer, a coding tool built to extract data from articles. It is a customized extraction tool defined by the authors based on a previously constructed extraction matrix.

After completing the extraction stage, it is possible to generate data files for filtering and correcting the database, in order to build tables with a synthesis of all the information from the studies included and from the database itself.

Stage 6 - Quality of the evidence assessment

The researcher should assess the risk of bias in individual studies using appropriate scales and tools according to the design of each study. The risk of bias analysis of the studies included in the systematic review should be clearly presented in the Results section, describing its impact on the systematic review and meta-analysis. Some collaborations, such as the Joanna Briggs Institute (JBI), enable establishing a cutoff value to define the acceptable study quality for inclusion in the SR. After their application, they may lead to the exclusion of studies, including those previously selected. However, it is up to the researchers to decide in advance whether to include studies with a high risk of bias, recording them in the protocol. Primary studies with a high risk of bias in the meta-analysis can be included as long as appropriate meta-analytical methods are applied. In the sensitivity analysis, removing the meta-analysis of studies with a high risk of bias can be defined to assess the impact on the outcome and conclusions3.

Some internationally recognized scales, such as the Newcastle-Ottawa Scale (NOS)22, the Joanna Briggs Critical Appraisal tools23, ROBINS-I tool Cochrane and INS-E tool Cochrane, can assess the risk of bias in observational studies24.

To assess the risk of bias using the NOS scale, it is possible to incorporate it into EPPI-Reviewer following the recommendations for systematic reviews. This stage should also be performed by two independent researchers, and any and all discrepancies should be discussed with a third researcher until consensus is reached.

When NOS is applied, the risk of bias is assessed in the following three aspects: selection of the study groups; comparability of the groups; and ascertainment of exposure for case-control studies or outcome of interest for cohort studies. A maximum number of stars can be assigned to the first (four), second (two) and third (three) aspects22.

A summary can be presented using a “traffic light plot” system, created with the robvis tool (https://mcguinlu.shinyapps.io/robvis/). This eases the production of high-quality figures that summarize the risk of bias assessments conducted as part of the systematic review or research synthesis project25.

The level of evidence of the articles is assessed by applying the Grading of Recommendation Assessment Development and Evaluation (GRADE) system.

GRADE is a system developed to grade the quality of the evidence and the strength of the resulting recommendations. When applying the GRADE system, a synthesis table should be constructed with the outcome results accompanied by the quality of the evidence, including the association measure value and its respective confidence interval. In addition to that, a judgment regarding each of the factors that alter the quality of the evidence should be added. Observational studies start with low quality (2 points), which makes it easier to make decisions based on the evidence produced by the review26.

In the GRADE system, the quality of the evidence is classified as high, moderate, low or very low. The GRADE PRO web-based system (http://guidelinedevelopment.org and http://gradepro.org) can be used to construct tables summarizing the evidence; access and use are free and unrestricted for non-commercial projects. These tables include the review results along with the quality of the evidence, content (or the association measure value) and the respective confidence interval. They also allow for the addition or judgment of each of the factors that alter the quality of the evidence; observational studies start with low quality (2 points), which eases decision-making based on the evidence produced by the review26.

Stage 6 - Synthesis of the studies

This stage (also known as analysis) involves combining the data extracted from studies using appropriate techniques, whether quantitative, qualitative or both (or only a narrative synthesis when, after exhaustive analysis, it is determined that another type of synthesis is not applicable). They must be planned and described in the protocol, including how heterogeneity will be assessed and the choice of effect measure (e.g., Odds Ratio, Risk Ratio, Risk Difference, or another one for dichotomous outcomes). The synthesis planning (particularly the statistical methods) ensures that the analysis or interpretation of the review findings will not be affected by judgments made during the process14.

Stage 7 - Meta-analysis

It will not be presented here because it will be included in another manuscript.

Stage 8 - Writing and publication of the manuscript

In addition to the standard principles to be followed in writing research articles, the systematic literature review process should be reported in sufficient detail so that the review results can be independently reproduced. In systematic reviews produced in collaboration with JBI and Cochrane, the reports have their own structures.

We recommend using the PRISMA (http://prisma-statement.org/PRISMAstatement/checklist.aspx) and PRISMA for abstracts (http://prisma-statement.org/Extensions/Abstracts) checklists to draft the manuscript, including them as supplementary materials (it is frequently required for submission to journals)10.

In addition to synthesizing the diverse evidence found, the final manuscript should discuss the study strengths and limitations in terms of knowledge production. This includes an overall interpretation of the results in the context of other evidence; noting the limitations of the evidence included in the review and the processes employed in it, highlighting the implications of the results for the clinical practice, policy and future research studies10.

The financial support sources and the role of funders or sponsors in the review should be described in the manuscript. We recommend that the authors make the data, analytical code and other materials used available, indicating where they can be accessed, such as repositories or supplementary materials. Additionally, we recommend that the researcher update the systematic review as new studies are added to the databases consulted10.

CONCLUSION

This guide is a contribution to research in the health area and an innovation by bringing together guidance on systematic review methods and approaches. References serve as a starting point for better understanding the international procedures and tools required for systematic reviews of observational studies.

Encouraging the use of evidence-based clinical practice contributes to ensuring that health interventions are based on the best available evidence. In addition to that, this guide enhances the research quality and impact, ensuring scientific advancement and, thus, improving the quality of the care provided to the population.

REFERENCES

• 1. Johnston A, Kelly SE, Hsieh S-C, Skidmore B, Wells GA. Systematic reviews of clinical practice guidelines: A methodological guide. J Clin Epidemiol [Internet]. 2019 [cited 2022 Nov 9];108:64-76. Available from: https://doi.org/10.1016/j.jclinepi.2018.11.030
  » https://doi.org/10.1016/j.jclinepi.2018.11.030
• 2. Dekkers OM, Vandenbroucke JP, Cevallos M, Renehan AG, Altman DG, Egger M. COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology. PLoS Med [Internet]. 2019 [cited 2021 Jan 9];16(2):e1002742. Available from: https://doi.org/10.1371/journal.pmed.1002742
  » https://doi.org/10.1371/journal.pmed.1002742
• 3. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022) [Internet]. Chichester, (UK): Cochrane; 2022 [cited 2022 Nov 9]. Available from: http://www.training.cochrane.org/handbook
  » http://www.training.cochrane.org/handbook
• 4. Moola S, Munn Z, Sears K, Sfetcu R, Currie M, Lisy K, et al. Conducting systematic reviews of association (etiology): The Joanna Briggs Institute's approach. Int J Evid Based Healthc [Internet]. 2015 [cited 2021 Nov 21];13(3):163-9. Available from: https://doi.org/10.1097/XEB.0000000000000064
  » https://doi.org/10.1097/XEB.0000000000000064
• 5. Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis [Internet]. JBI; 2020 [cited 2021 Nov 10]. Available from: https://jbi-global-wiki.refined.site/space/MANUAL
  » https://jbi-global-wiki.refined.site/space/MANUAL
• 6. The EQUATOR Network. Reporting guidelines for main study types The EQUATOR Network Webpage [Internet]. The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network; 2023 [cited 2023 May 9]. Available from: https://www.equator-network.org/
  » https://www.equator-network.org/
• 7. GRADE working group. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) [Internet]. GRADE; 2023 [cited 2023 Mar 10]. Available from: https://www.gradeworkinggroup.org/
  » https://www.gradeworkinggroup.org/
• 8. Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data. Int J Evid Based Healthc [Internet]. 2015 [cited 2023 Mar 10];13(3):147-53. Available from: https://doi.org/10.1097/XEB.0000000000000054
  » https://doi.org/10.1097/XEB.0000000000000054
• 9. Nik Hazlina NH, Norhayati MN, Shaiful Bahari I, Mohamed Kamil HR. The prevalence and risk factors for severe maternal morbidities: A systematic review and meta-analysis. Front Med (Lausanne) [Internet]. 2022 [cited 2023 Feb 3];9:861028. Available from: https://doi.org/10.3389/fmed.2022.861028
  » https://doi.org/10.3389/fmed.2022.861028
• 10. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. Updating guidance for reporting systematic reviews: Development of the PRISMA 2020 statement. J Clin Epidemiol [Internet]. 2021 [cited 2022 Dec 21];134:103-12. Available from: https://doi.org/10.1016/j.jclinepi.2021.03.001
  » https://doi.org/10.1016/j.jclinepi.2021.03.001
• 11. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev [Internet]. 2015 [cited 2020 Mar 10];4(1):1. Available from: https://doi.org/10.1186/2046-4053-4-1
  » https://doi.org/10.1186/2046-4053-4-1
• 12. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: Elaboration and explanation. BMJ [Internet]. 2015 [cited 2022 Mar 20];349:g7647. Available from: https://doi.org/10.1136/bmj.g7647
  » https://doi.org/10.1136/bmj.g7647
• 13. Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol [Internet]. 2018 [cited 2023 Jan 20];18(1):5. Available from: https://doi.org/10.1186/s12874-017-0468-4
  » https://doi.org/10.1186/s12874-017-0468-4
• 14. Muka T, Glisic M, Milic J, Verhoog S, Bohlius J, Bramer W, et al. A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research. Eur J Epidemiol [Internet]. 2020 [cited 2022 Nov 20];35:49-60. Available from: https://doi.org/10.1007/s10654-019-00576-5
  » https://doi.org/10.1007/s10654-019-00576-5
• 15. Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: Making research syntheses more transparent and easy to replicate. Res Synth Methods [Internet]. 2015 [cited 2023 Jan 20];6(1):87-95. Available from: https://doi.org/10.1002/jrsm.1127
  » https://doi.org/10.1002/jrsm.1127
• 16. Cardoso V, Trevisan I, Cicolella DA, Waterkemper R. Systematic review of mixed methods: Method of research for the incorporation of evidence in nursing. Texto Contexto Enferm [Internet]. 2019 [cited 2022 Nov 20];28:e20170279. Available from: https://doi.org/10.1590/1980-265X-TCE-2017-0279
  » https://doi.org/10.1590/1980-265X-TCE-2017-0279
• 17. Mittal N, Goyal M, Mittal PK. Understanding and appraising systematic reviews and meta-analysis. J Clin Pediatr Dent [Internet]. 2017 [cited 2023 Jan 4];41(5):317-26. Available from: https://doi.org/10.17796/1053-4628-41.5.317
  » https://doi.org/10.17796/1053-4628-41.5.317
• 18. Mendes KDS, Silveira RCCP, Galvão CM. Use of the bibliographic reference manager in the selection of primary studies in integrative reviews. Texto Contexto Enferm [Internet]. 2019 [cited 2022 Nov 20];28:e20170204. Available from: https://doi.org/10.1590/1980-265X-TCE-2017-0204
  » https://doi.org/10.1590/1980-265X-TCE-2017-0204
• 19. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan - A web and mobile app for systematic reviews. Syst Rev [Internet]. 2016 [cited 2020 Feb 20];5(1):210. Available from: https://doi.org/10.1186/s13643-016-0384-4
  » https://doi.org/10.1186/s13643-016-0384-4
• 20. Drukker M, Weltens I, van Hooijdonk CFM, Vandenberk E, Bak M. Development of a methodological quality criteria list for observational studies: The observational study quality evaluation. Front Res Metr Anal [Internet]. 2021 [cited 2023 Jan 10];6:675071. Available from: https://doi.org/10.3389/frma.2021.675071
  » https://doi.org/10.3389/frma.2021.675071
• 21. Thomas J, Graziosi S, Brunton J, Ghouze Z, O'Driscoll P, Bond MK. EPPI-Reviewer: Advanced software for systematic reviews, maps and evidence synthesis [Internet]. London, (UK): EPPI-Centre, UCL Social Research Institute, University College London; 2022 [cited 2022 Jan 20]. Available from: https://eppi.ioe.ac.uk/cms/About/AboutEPPIReviewer/tabid/2967/Default.aspx
  » https://eppi.ioe.ac.uk/cms/About/AboutEPPIReviewer/tabid/2967/Default.aspx
• 22. Zeng X, Zhang Y, Kwong JS, Zhang C, Li S, Sun F, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta‐analysis, and clinical practice guideline: A systematic review. J Evid Based Med [Internet]. 2015 [cited 2020 Nov 20];8(1):2-10. Available from: https://doi.org/10.1111/jebm.12141
  » https://doi.org/10.1111/jebm.12141
• 23. Munn Z, Stone JC, Aromataris E, Klugar M, Sears K, Leonardi-Bee J, et al. Assessing the risk of bias of quantitative analytical studies: Introducing the vision for critical appraisal within JBI systematic reviews. JBI Evidence Synthesis [Internet]. 2023 [cited 2023 Apr 10];21(3):467-71. Available from: https://doi.org/10.11124/JBIES-22-00224
  » https://doi.org/10.11124/JBIES-22-00224
• 24. ROBINS-E Development Group. Risk of Bias in Non-randomized Studies - of Exposure (ROBINS-E) [Internet]. 2022 [cited 2022 Jun 15]; Launch version, 1 June. Available from: https://www.riskofbias.info/welcome/robins-e-tool
  » https://www.riskofbias.info/welcome/robins-e-tool
• 25. McGuinness LA, Higgins JP. Risk‐of‐bias VISualization (robvis): An R package and Shiny web app for visualizing risk‐of‐bias assessments. Res Synt Met [Internet]. 2021 [cited 2022 Apr 10];12(1):55-61. Available from: https://doi.org/10.1002/jrsm.1411
  » https://doi.org/10.1002/jrsm.1411
• 26. Schünemann HJ, Mustafa RA, Brozek J, Steingart KR, Leeflang M, Murad MH, et al. GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy. J Clin Epidemiol [Internet]. 2020 [cited 2022 Apr 10];122:129-41. Available from: https://doi.org/10.1016/j.jclinepi.2019.12.020
  » https://doi.org/10.1016/j.jclinepi.2019.12.020

NOTES

• ORIGIN OF THE ARTICLE

  Article extracted from the thesis - Prevention of Maternal Mortality during Childbirth”, presented to the Graduate Program in Nursing of Universidade Federal de Santa Catarina, in 2023.

• TRANSLATED BY

  Leonardo Parachú.

Edited by

EDITORS

Associated Editors: Bruno Miguel Borges de Sousa Magalhães, Maria Lígia Bellaguarda. Editor-in-chief: Elisiane Lorenzini.

Publication Dates

• Publication in this collection
  13 May 2024
• Date of issue
  2024

History

• Received
  28 Aug 2023
• Accepted
  19 Dec 2023