Realist review as a methodology for using evidence in health policies: an integrative review

ABSTRACT

Objective:

To identify and analyze the concepts of realist review and the recommended methodology for its development in the health area.

Method:

An integrative review which sought theoretical-methodological qualitative or quantitative studies through 11 data sources which analyzed or proposed theories and/or realist review methodology.

Results:

Twenty-three (23) studies, mainly published in the United Kingdom, were included. The realist review aims at explaining what makes a complex policy, program, or intervention work, in which aspects, for whom, in what context, to what extent, and why, by constructing theory, decompressing the functioning mechanisms, complex policy contexts or programs which generate results. Such methodology relies on the involvement of stakeholders in the problem from the early stages of the review process.

Conclusion:

The realist review has the potential to provide relevant and applicable subsidies for researchers, workers, policy makers and health decision makers.

DESCRIPTORS
Health Policy; Evidence-Informed Policy; Health Plan Implementation; Nursing Research; Measurements, Methods and Theories; Review

RESUMO

Objetivo:

Identificar e analisar os conceitos de revisão realista e a metodologia recomendada para seu desenvolvimento na área da saúde.

Método:

Revisão integrativa que, por meio de 11 fontes de dados, buscou estudos teórico-metodológicos, qualitativos ou quantitativos, os quais analisaram ou propuseram teorias e/ou metodologia de revisão realista.

Resultados:

Foram incluídos 23 estudos, publicados majoritariamente no Reino Unido. A revisão realista visa explicar o que faz com que uma política, programa ou intervenção complexa funcione, em quais aspectos, para quem, em qual contexto, em que extensão e por que, por meio da construção de teoria, descompactando os mecanismos de funcionamento e contextos de políticas ou programas complexos que geram resultados. Tal metodologia conta com o envolvimento dos interessados no problema desde os estágios iniciais do processo de revisão.

Conclusão:

A revisão realista tem a potencialidade de fornecer subsídios relevantes e aplicáveis para pesquisadores, trabalhadores, formuladores de políticas e tomadores de decisão em saúde.

DESCRITORES
Política de Saúde; Política Informada por Evidências; Implementação de Plano de Saúde; Pesquisa em Enfermagem; Medidas, Métodos e Teorias; Revisão

RESUMEN

Objetivo:

Identificar y analizar los conceptos de revisión realista y la metodología recomendada para su desarrollo en el área sanitaria.

Método:

Revisión integrativa que, mediante 11 fuentes de datos, buscó estudios teóricos y metodológicos, cualitativos o cuantitativos, los que analizaron o propusieron teorías y/o metodología de revisión realista.

Resultados:

Fueron incluidos 23 estudios, publicados mayoritariamente en el Reino Unido. La revisión realista tiene el fin de explicar qué hace que una política, programa o intervención compleja funcione, en cuáles aspectos, para quiénes, en cuál contexto, en qué extensión y por qué, mediante la construcción de teoría, descompactando los mecanismos de funcionamiento y entornos de políticas o programas complejos que generan resultados. Dicha metodología cuenta con la participación activa de los interesados en el problema desde los estadíos iniciales del proceso de revisión.

Conclusión:

La revisión realista tiene la potencialidad de proporcionarles subsidios relevantes y aplicables a investigadores, trabajadores, diseñadores de políticas y tomadores de decisión en salud.

DESCRIPTORES
Política de Salud; Política Informada por la Evidencia; Implementación de Plan de Salud; Investigación en Enfermería; Mediciones, Métodos y Teorías; Revisión

INTRODUCTION

Systematic use of evidence is fundamental for elaborating, implementing, monitoring and evaluating public policies in the health area. However, ineffective, unnecessarily expensive and unequal actions and programs are still present in health services and public management, which reveals limitations in the use of and access to research evidence to base health policies on⁽¹⁾.

The permanence and even the deepening of differences between the epidemiological profiles corresponding to diverse forms of social class reproduction constitute an enormous challenge for policymakers. The change in health inequalities depends on profound changes in their social determination and requires strengthening the understanding of the processes which generate inequalities⁽²⁾. Nevertheless, this change demands that the states generate public policies which guarantee universal rights and public services for the whole population⁽³⁾.

In the world of synthesizing and implementing evidence, there are clear limitations to take into account fundamental dimensions of reality, such as those that engender health inequalities. In the case of multimorbidity in older adults, for example, the available studies concentrate their analysis on the biomedical dimension, which consequently inadvertently supports practices, research and policies which are in disagreement with the health needs of those in this phase of life⁽⁴⁾, especially if one considers the differences between social classes. Thus, promoting the systematic use of knowledge focused on improving the efficiency of services and health conditions of the various social groups remains a major challenge for health systems⁽⁵⁾.

In this direction, in order to design public health policies which respond to this diversity of needs and to promote the modification of the different epidemiological profiles, it is essential to reflect on the complexity of the Unified Health System (SUS – Sistema Único de Saúde). It is a tangled network of large-scale interrelationships in processes of constant development and adaptation. The establishment of public policies in complex systems requires using research synthesis results that are capable of explaining what works, for whom, under what circumstances, how and why. This process enables in-depth and comprehensive understandings and analysis of problems and solutions. It is becoming increasingly clear that research summaries restricted to identifying evidence of greater efficacy are not sufficient for political decision-making^(6–8), especially when such evidence is restricted to the effect of biological and individual interventions impotent to overcome barriers related to social relations, which are the basis of health inequalities.

In recent years, various types of revisions of scientific literature on health have been developed and refined in view of the needs of policy makers which are not addressed by existing systematic review methods⁽⁹⁾. In this sense, the realist review is an innovative and promising proposal because, in addition to effectiveness, it aims to understand the mechanisms, complex dynamics and variability of interventions, so that policy makers have sufficient information to make decisions^(6,8).

Recognizing the realist review as a method of synthesizing evidence led to developing standards for publishing realist syntheses. Realist reviews aim at developing and analyzing principles, concepts, models, theories and social interventions at a systemic level⁽⁸⁾. Initiatives in the health, education, transportation, security and housing areas were reported, evidencing the power of this methodology in diverse contexts^(10–11). The realist review is increasingly and specifically being used in the health area to study both clinical-medical and health-promoting interventions to support the synthesis of complex evaluations. A number of themes have been the focus of this type of review, such as interventions and strategies for suicide prevention, management of depression and obesity, human resource management, interventions to improve precarious housing, sanitation and water supply situations, informed health promotion by evidence, threats to legislative interventions in public health and transformations of large systems⁽¹²⁾. However, until recently it was not a very well-known or widespread method in Brazil⁽⁸⁾.

A literature review must follow the same parameters of primary research, meaning that the study must present all the necessary dimensions for knowledge construction in a coherent and articulated manner, namely the epistemological, the theoretical and the methodological dimensions⁽¹³⁾.

This review was carried out with the objective of identifying and analyzing the concepts of realist review and the recommended methodology for its development in the health area, guided by the following question: how is the realist review used in the health area? Is there coherence between the theoretical references, purposes and methodology?

METHOD

Study design

This is an integrative review (IR), a methodology that integrates findings from studies with different designs, thereby enabling understanding of relevant problems and the definition of new concepts. IR encompasses the revision of theories and methodologies, since it allows simultaneous inclusion of empirical and theoretical studies⁽¹⁴⁾ and is appropriate to investigate emerging areas, such as the case of policy reviews.

Scenario

The IR was performed according to the following steps: 1. Form a group to develop the IR; 2. Prepare the introduction of the review; 3. Formulate the question and purpose of the review; 4. Describe the review methodology; 5. Analyze and interpret empirical data; 6. Present results; 7. Interpret and discuss results; and 8. Disseminate the results⁽¹⁵⁾.

Therefore, and in considering the qualitative tradition of the integrative review, which seeks to review, critique and synthesize the results using interpretative and critical methods, this review will guide the analysis according to the critical paradigm⁽¹⁶⁾, and more particularly under the assumptions of Historical-Dialectical Materialism (HDM). In this area, the debate of theory with reality (thesis and antithesis) enables innovative syntheses which expose social contradictions. It is a reference which attempts to elucidate the phenomena under study, seeking to overcome apparent and descriptive explanations⁽¹³⁾.

Selection criteria

In this context, theoretical-methodological and qualitative or quantitative studies, as well as the gray literature that analyzed or proposed theories and/or realist review methodology were included. Studies that used the realist review as a methodology to review a research topic were excluded. The search was conducted in April 2018 and incorporated references published in Portuguese, English and Spanish in any year and indexed until March 2018 in the databases.

The denomination methodology is adopted in this work and not method for the set of procedures which involve performing the realist review. It is understood that methodology is more adequate because it reveals the interconnection between the methodological procedures and the theoretical framework of the research, incorporating varied methods which explain how the research will be carried out, as well as the tools and the processes that are used⁽¹⁷⁾.

Data collection

The expression “Realist Review” was used in English or Portuguese for the search in scientific and gray literature in the following databases: COCHRANE Library, EVIPNet, Google Scholar, Health Systems Evidence, LILACS, PDQ-Evidence, PubMed, Rx for change, SciELO, SCOPUS and Teses CAPES. The references of the included databases were also explored, in addition to manual searches. After identifying the studies, the references were independently selected by title and abstract by two reviewers.

Data analysis and processing

After reading the entire publication, the data were read, evaluated, included/excluded, extracted and analyzed in a narrative form through an instrument composed of the following items: (1) Title; (2) Year; (3) Authors; (4) Affiliated country of the lead author; (5) Knowledge area of the first author; (6) Study objective; (7) Definition and purposes of the realist review; (8) Theoretical reference framework; (9) Realist review development stages; (10) Summary of selected text; (11) Others. There are no conflicts of interest between the authors of this article.

Ethical aspects

Not applicable because this is an integrative review study.

RESULTS

A total of 664 references were identified, 626 in the 11 data sources and 38 through manual searches and references of the references. After exclusion of duplicate publications and selection by title and abstract, 45 references were analyzed in their entirety (five were excluded due to lack of availability of full texts). In total, 23 references were included, according to Chart 1 and Figure 1.

Chart 1
Data source, search strategies and identified references selected by title and abstract – São Paulo, SP, Brazil, 2018.

Data source	Search strategy	Identified references
PubMed	Realist review [Title/Abstract]	216
LILACS	Realist [Palavras] and review [Palavras]	1
COCHRANE Library	realist:ti,ab,kw and review:ti,ab,kw (Word variations have been searched)	21
SCOPUS	TITLE-ABS-KEY (“Realist review”)	257
EVIPNet	ti:(realist) AND ti:(review)	3
Health Systems Evidence	“Realist review”	20
Rx for change	“Realist review”	3
PDQ-Evidence	“Realist review”	12
SciELO	Revisão [Resumo] and Realista [Resumo]	6
Teses CAPES	“Revisão realista”	2
Google Scholar	“Realist review”	85

Figure 1
Flowchart of the article selection process.

It was observed that most of the articles in this study are recent, mainly published since 2011, with affiliation of the first authors to universities (100%), in the Health area (68%), and with links to institutions of the United Kingdom (68%) (Table 2).

Chart 2
Author(s), year of publication, country of affiliation of the main author, knowledge area of the first author and definitions and purpose of the realist review – São Paulo, SP, Brazil, 2018.

Author(s)/Year/Country of affiliation of the main author/Knowledge area of the first author	Definitions and purposes of the realist review
Pawson (2001)(18) United Kingdom Sociology and Social Policy	Aims to obtain information about what works, for whom and in under what circumstances, applicable to contexts, aspects and subjects in specific situations.
Pawson (2002)(10) United Kingdom Sociology and Social Policy	Favors constructing program theories to satisfy the needs in diverse social situations based on existing evidence.
Pawson et al. (2005)(6) United Kingdom Sociology and Social Policy	Synthesis of research with explanatory focus that aims to unpack the mechanisms of how complex programs work (or why they fail), in specific contexts and configurations. The search for understanding what works in social interventions involves the attempt to establish causal relationships, and it is necessary to understand the underlying mechanism that connects them, as well as the context in which the relationship occurred.
Dixon-Woods et al. (2005)(30) United Kingdom Social Sciences and Health	Acknowledges the theories behind a particular program or intervention. It seeks evidence in different types of texts: qualitative and quantitative formal study reports, case studies, media reports, among other sources, and integrates them as evidence proof or to refute the theory.
Mays et al. (2005)(31) United Kingdom Public Health	Develops theory to explain why interventions or programs work (or not) for specific groups in characteristic contexts.
Tractenberg et al. (2011)(8) Brazil Administration	Seeks the theoretical understanding of the mechanisms underlying the interventions to provide an explanation of how and why the intervention works (or not) in a given context.
Wong (2012)(19) United Kingdom Policy and Innovation in Health	Understands complex interventions and does not aim to prove that an intervention works, but to explain how, why, to whom, under what circumstances and to what extent it works. Context influences which mechanism will be triggered to produce specific results.
Kirst et al. (2012)(27) Canada Public Health	Recognizes that the effects of interventions such as programs and policies crucially depend on context and implementation. It does not adhere to a strict hierarchy of evidence in which randomized clinical trials are considered the best type of study.
Rycroft-Malone et al. (2012)(32) United Kingdom Health Sciences	Approach to reveal implicit theories by examining the interactions between mechanism, context and outcome, and to identify strategies and interventions to enable healthcare informed by evidence.
Saul et al. (2013)(29) Canada Research Center	Structured method to respond to local policy needs. The Rapid Realist Review is a tool used to apply realist precepts to the knowledge synthesis process in order to build a product that is relevant to local policy makers, in the decisions and necessary responses.
Tractenberg (2013)(26) Brazil Administration/Management	Qualitative research synthesis with an interpretative and explanatory purpose, used to understand the processes or mechanisms inherent in complex social interventions.
Jagosh et al. (2013)(33) Canada Medicine	A theory-oriented interpretive approach to configuring contextual factors and outcome-related change mechanisms.
Gough (2013)(34) United Kingdom Social Sciences	Revisions of mixed methods with three overlapping principal components: 1) unpack explicit and implicit assumptions of context, mechanism and outcome; 2) iterative aggregative test of the empirical data in particular CMO configurations; 3) explore and explain findings in different contexts from CMO interaction.
Wong et al. (2013)(20) United Kingdom Social Sciences	Relates to a single research question or set of questions, which can be summarized as what works, for whom, under what circumstances, how and why. The reviewers seek the contextual influences that supposedly triggered the relevant mechanism(s) for each idea to generate the outcome(s) of interest.
Wong et al. (2013)(21) United Kingdom Primary Care	A theory-based approach to synthesize existing evidence. It is emphasized that theories are basic units of analysis.
Fawcett (2013)(35) United States Nursing	It enables researchers to better understand why an assessment, intervention or policy tool is effective or not for use with specific individuals in a given environment.
Edgley, Stickley, Timmons (2016)(28) United Kingdom Social Sciences and Mental Health	Critical realist review includes factors related to interventions and social justice. It tries to evaluate the logic of a study design about a certain phenomenon.
Wong et al. (2014)(22) United Kingdom Primary Care	Systematic, theory-oriented interpretive techniques which have been developed to make sense of heterogeneous evidence on complex interventions applied in various contexts to subsidize policy.
Tricco et al. (2016)(36) United Kingdom Institute of Knowledge	Data collection involves the process of interrogating the evidence to refine the medium-range theory using a set of instruments that can evolve as the review progresses.
Pawson, Greenhalgh, Brennan (2017)(23) United Kingdom Sociology and Social Policy	Locates the primary studies to explore the form and direction of the continuous theory refinement process in order to understand in what circumstances and by what processes these system tensions are solved.
Wong (2018)(24) United Kingdom Primary Care	Establishes a realistic explanation of the links between context and outcome through mechanisms. It has a particular approach to the analysis, succinctly captured in the heuristic context + mechanism = outcome (or C + M = O).
Wong (2018)(37) United Kingdom Primary Care	A theory-oriented approach to evidence synthesis. Realist reviews and evaluations are best suited to make sense of complex interventions, where context influences outcomes, and to answer questions such as what works, who, in what contexts, to what extent, how and why.
Wong (2018)(25) United Kingdom Primary Care	Its central point is to develop the initial program theory, which is confirmed, refuted or refined through data collection. It deals with complex interventions and their many components, which interact in a linear or non-linear way with context-sensitive results.

Data synthesis: realist review concept

Similarities between the definitions and purposes are observed in the included studies, mainly related to the theoretical development of the realist review methodology by a group of researchers from the United Kingdom. Pawson, Greenhalgh, and Wong published 10 studies in total, together or alone, including the three oldest and the four most recent (Chart 2)^{(6,10,18–25)}.

Integrating the presented concepts and purposes makes it possible to affirm that the realist review is a structured methodology of qualitative and systematized synthesis used to respond to the needs of complex policies, programs and interventions, since it enables understanding what works, how, for who, and within a specific context for developing theories that base a specific program and/or intervention with an explanatory and interpretive focus^{(6,8,18–37)}.

The realist review or synthesis was structured according to the philosophical foundations of realism, which is one of the few approaches of the Social Sciences that tries to account for the complexity of interventions. Realism presupposes the existence of the external reality (real world), which is filtered (meaning it is perceived, interpreted and answered) through the senses, desires, language and culture. This human processing initiates constant actions of self-generated changes in all social institutions; a vital process that needs to be accommodated in evaluating social programs^(22,25).

Therefore, this approach shifts the focus of epidemiological research, which is concerned with what and if the intervention works, to research that explains what makes the program/intervention work, for who, in what aspects, in which context and why^(6,38). Thus, it is fundamental to consider that the proposed interventions depend on complex interactions, negotiations and conflicts between individuals, relationships, institutions and infrastructures. Contextual variables include different environmental, historical, political, social, economic, institutional and other factors⁽²⁶⁾.

Understanding complex interventions should include the following findings: interventions composed of several components with interactions among components producing results; the human character of the decisions defines the nature of these interactions; the components do not interact in a linear way; and interactions are influenced by the context of and beyond the proposed intervention^(19,25).

By problematizing the complexity of contexts and identifying theories about how and why an intervention works, realist review provides key elements of how outcomes of interest are generated by inherent mechanisms in social relationships in different contexts. In this direction, it is emphasized that interventions do not produce results, but offer resources: the results depend on how recipients respond to resources, which vary according to the context⁽³⁸⁾. The included studies demonstrated that the review analyzes the relationship between context (C) and mechanism (M) which generates outcome (O) (CMO), thus defined⁽¹²⁾: Context (C): referring to historical social reality and intervention dynamics, as a collective in which a program is implemented, the nature and scope of existing social networks or the infrastructure of the constructed program; Mechanism (M): refers to the interaction of the generating force (resources, logic, perception, understanding, among other aspects of social relations) and internal knowledge of the collectivity, which leads to the results. Identifying the mechanisms generates advances in synthesis, as it problematizes beyond the description “of what has happened” to theorize “why it happened, to whom and under what circumstances”. The possibility that researchers modify the research design is inherent in the mechanism, being sensitive to the practical experiences of the involved group; Outcomes (O): result from the interaction between context and mechanism (C + M = O), in relation to the proposed intervention; they may be intentional or unintentional, proximal, intermediate or final. Examples of outcomes: improving health status and utilization of health services, strengthening social groups around rights, improving access to formal education, developing program infrastructure, and improving research processes.

In this direction in order to synthesize the CMO configuration, one can exemplify the case of a neighborhood which experienced a high unemployment context, constituting a problem for which a job training program was offered. However, it was observed that the program resulted in a low enrollment rate. The reason was that people had difficulty reaching the place due to lack of public transportation (mechanism)⁽¹²⁾. The construction of a program theory encompasses hypotheses, and explanations of what works (or not).

Theories highlighted by realist reviews can be found in different parts of a text, such as in method, discussion, history, and others, not only specifically in results and conclusions, as in systematic reviews. These theories can be presented according to the macro, meso or micro perspectives⁽²⁶⁾.

Two specific types of realist review have been described: critical and rapid. The critical realist review includes analyzes of social context, social justice, and factors related to interventions based on social science theories and methodologies for health care, policy and practice interventions. The authors emphasize that interventions have a history and place within a wide range of interventions, and typically within complex bureaucracies, which are designed by managers, operationalized (sometimes imperfectly) by professionals, and accepted or resisted by patients and/or the public⁽²⁸⁾. Rapid realist review is considered an adaptation of realist review assumptions. Maintaining the philosophical realist approach, this type of review proposes to develop the extraction, analysis and data synthesis stages in a shorter period than the conventional review. The theory construction was considered a support procedure for identifying intervention families and explaining why they produced results, generating specific changes in context which then triggered specific mechanisms, rather than the primary outcome. Such a support procedure is designed to engage knowledge users and review stakeholders to narrowly define research questions and thus simplify the review process, which can be completed over a period of 3 to 6 months⁽²⁹⁾.

The following describes rapid and realist review stages. It was not possible to obtain the realist review stages by means of the researched literature.

Realist review stages

The realist review stages were integrated, as shown in Chart 3. Although the presentation is in sequence, the stages occur in an overlapping and iterative way⁽⁶⁾, with the involvement of stakeholders from the initial process stages up to completion of the study in order to develop powerful conclusions and ensure the practical relevance of the final product. It has an average duration of 12 to 18 months^(25,32,39).

Chart 3
Realist Review Development Stages – São Paulo, SP, Brazil, 2018.

1. Formation of realist review team ( 6 , 19 , 27 , 33 , 39 )

2. Panorama of the theme and locate possible theories ( 19 , 25 )

3. Title ( 20 )

4. Abstract ( 20 )

5. Introduction

5.1. Rationale for review and identification of the research question(6,20–21)

6. Review objectives and focus ( 20 )

7. Methods

7.1. Rationale for using a realist review(6,20)

7.2. Scope of the literature(6,20,32)

7.3. Description of the search process(20–21,25)

7.4. Description of the selection processes and evaluation of documents(6,20–21)

7.5. Description of the data extraction process(6,20,32)

7.6. Description of the analysis and synthesis process(6,20–21)

7.7. Changes in the review process(20)

8. Results

8.1. Selection and inclusion flow diagram(20,25)

8.2. Evaluation of the included studies(6)

8.3. Description of the characteristics of the documents(20,25)

8.4. Extraction, analysis and synthesis of the findings(6,20–21,25)

8.5 Theory test(6,20–21,25)

8.6. Review of the results(32)

8.7. Construction of a hierarchical theoretical framework(6,20)

9. Discussion

9.1. Summary of the findings(20)

9.2. Strengths, limitations and future directions of research(20)

9.3. Comparison with existing literature(20)

10. Conclusion and recommendations ( 6 , 20 )

11. Financing and conflict of interest ( 20 )

12. Dissemination ( 6 , 32 )

The included studies presented guidelines for developing the realist review, which are described below:

Stage 1. Formation of a realist review team: include team members from different knowledge areas with a high degree of experience and training in academic/research (critical evaluation of empirical studies) and service (program implementation) domains to perform the stages in parallel, iteratively modify the initial protocol and revisit (if necessary) all stages of the process as the review progresses^{(6,19,27,33,39)}.

Stage 2. Panorama of the theme and to locate possible theories: to carry out a brief mapping of the literature for the panorama of the proposed theme and to identify the theories that explain the phenomenon of interest. Theories found may be deepened, refuted or included during the review. This stage allows prior knowledge of the proposed theme to support the next stages of the realist review, and may be carried out by bibliographic review or expert consultation^(19,25).

Stage 3. Title: identify the document as a realist synthesis or review⁽²⁰⁾.

Stage 4. Abstract: include details about the study's history, review question or objectives, research strategy, selection, evaluation, analysis, synthesis, and sources methods, main results and the implications of the realist review for practice⁽²⁰⁾.

Stage 5. Introduction.

Stage 5.1. Rationale for the review and identification of the research question:

In developing the rationale, it is relevant to explain the purpose of the review and the likely contribution; describe the review question(s), considering the nature and content of the intervention, the circumstances or the context for its use, the intentions or the objectives of the policy. Realist research contains some or all elements of: what works, how, why, for whom, to what extent, under what circumstances and in what duration.

Refine the purpose of the review, theoretical integrity – at this stage, researchers should reflect whether the intervention works as expected. Carry out the theoretical judgment, meaning which theories around the intervention fit better. Compare how the intervention works in different contexts, for different groups. A reality test, which considers how the intention of policy and intervention translates into practice. Articulate the key theories to explore. At this stage it is relevant to draw up a list of program theories for the exploratory search. Thus, the authors recommend some actions due to the volume of texts analyzed: group, categorize or synthesize theories and design a theory-based evaluation structure to be filled out with evidence^(6,20–21).

Stage 6. Objectives and focus of review: indicate the purpose(s) of the review. Define and provide a justification for the focus of the review⁽²⁰⁾.

Stage 7. Methods.

Stage 7.1. Rationale for using the realist review: explain why the realist review was considered the most appropriate method^(6,20);

Stage 7.2. Scope of the literature: describe and justify the initial exploratory process of the literature; a basic exploratory research can be carried out to familiarize the theme. Report the discussion with stakeholders in the review^(6,20,32);

Stage 7.3. Description of the search process: justify the search process indicating the locations in which the electronic databases were searched such as the name of the database, search terms, date of coverage and the last search. Define the necessary materials (quantitative, qualitative and gray literature, for example) and saturation points of the data to close the search for evidence. If stakeholder consultations, interviews, and/or focus groups are included, indicate how participants were identified and selected^(20–21,25);

Stage 7.4. Description of selection processes and evaluation of the documents: justify judgments about the inclusion and exclusion of document data; use the judgment to complement the evaluation checklists to consider the evidence as “fit for purpose”: relevance – does the research address the theory being tested? Rigor – does the research support the conclusions drawn by researchers or reviewers? Find relevant theories in literature^(6,20–21);

Stage 7.5. Description of the data extraction process: describe and justify the data or information extracted from the included documents; progressively focus on the key theories of the program, refining inclusion criteria in light of emerging data. Elaborate a list of theories; Group, categorize or synthesize theories; Elaborate a theoretical framework to be filled out with the identified evidence^(6,20,32);

Stage 7.6. Description of the analysis and synthesis process: describe the analysis and synthesis processes in detail. Include information on the analyzed constructs and the analytical process; intentional sampling may occur to test a subset, and explore new hypotheses by “snowballing”^(6,20–21);

Stage 7.7. Changes in the review process: describe and justify changes in the review process compared to the initial planning⁽²⁰⁾.

Stage 8. Results.

Stage 8.1. Selection and inclusion flow diagram: detail the number of documents evaluated for eligibility and included in the review, with a description of the reasons for exclusion at each stage, as well as indication of origin (databases, reference lists, and so on)^(20,25);

Stage 8.2. Evaluation of the included studies: evaluate the included studies from the previously adopted criteria⁽⁶⁾;

Stage 8.3. Description of document characteristics: describe the characteristics of the documents included in the review. Search for additional studies in included references^(20,25);

Stage 8.4. Extraction, analysis and synthesis of the findings: extract data from the studies and present the main results and conclusions, with specific focus on the construction. Synthesize the data to determine what works, for whom, how, and under what circumstances. Review the purpose of the review to boost the synthesis process. Use the contradictory evidence to generate evidence on the contextual influence^{(6,20–21,25)}.

Stage 8.5. Theory test: confirmation, refutation or refining of the initially identified theory. Develop a more refined realist program theory to explain the results^{(6,20–21,25)}.

Stage 8.6. Revision of results: involving developer/decision-makers in reviewing the results⁽³²⁾.

Stage 8.7. Hierarchical theoretical framework construction: develop the theoretical framework and its fundamental elements to subsidize the program, to determine what works, for whom, how and under what circumstances^(6,20). Evaluate and show the relationship between the theories raised and their components.

Stage 9. Discussion.

Stage 9.1. Summary of findings: summarize the main results, considering the review objective(s), research question(s), focus and target audience(s)⁽²⁰⁾;

Stage 9.2. Strengths, limitations and future directions of research: discuss the review's strengths and limitations. Include the considerations of all stages of the review process and the commentary on the general force of evidence that supports the explanatory ideas which have emerged⁽²⁰⁾;

Stage 9.3. Comparison with existing literature: indicate the applicable context, compare and contrast the results of the review with the existing literature on the same topic⁽²⁰⁾.

Stage 10. Conclusion and recommendations: list the main implications of the results. If appropriate, provide recommendations for policy and practice; present conclusions as a series of contextualized decision points of the general format (“if A, then B” or “in the case of C, D is unlikely to work”). Develop recommendations and conclusions with key stakeholders, especially focusing on any leverage which can be drawn into current political contexts^(6,20).

Stage 11. Funding and conflict of interest: provide details of the funding source for the review, the role played by the funder and the conflicts of interest of the reviewers, when it occurs⁽²⁰⁾.

Stage 12. Dissemination, implementation and evaluation: disseminate the review with results, conclusions and recommendations. Involve key stakeholders, practitioners and policy makers to implement recommendations in specific contexts. Evaluate in terms of the extent to which programs are adjusted to consider the contextual influences revealed by the review, since the same program may be expanded in one configuration, modified or abandoned in another^(6,32).

Rapid Realist Review Stages

Specifically regarding the rapid realist review, 10 stages were proposed and are summarized in Chart 4. The rapid realist review shares similarities with the traditional realist review methodology with some fundamental differences. However, when reflecting on the differences, the time for the rapid realist review is reduced (3 to 6 months), with inclusion of a panel of experts in the review process in order to favor defining the research questions, conclusions and recommendations, which maximizes the review process occurring over a shorter period than the traditional realist review, and is therefore adequate for questions that require readiness in the response⁽²⁹⁾.

Chart 4
Development stages of rapid realist review⁽²⁹⁾.

1. Development of project scope

2. Development of specific research questions

3. Identification of how the findings and recommendations will be used

4. Development of search terms

5. Identification of articles and documents to be included in the review (indexed and gray literature)

6. Quality review of included studies

7. Data extraction from the literature

8. Validation of the findings with experts

9. Synthesis of results in a final report

10. Discovery of results

In the rapid realist review methodology, the engagement of specialists favors validating the findings, since the literature search is limited by the review time, and is not comprehensive when compared with other types of studies. Therefore, this limitation is overcome with the participation of specialists which favors validating the included studies and consistent interpretation of the results, thus allowing integration between the findings of the literature, the practical experience of the specialists and the political perspectives for developing theories⁽²⁹⁾.

Stage 1. Development of the project scope: fundamental step for developing the rapid realist review. Define together with the parties involved who are fundamental and users of the knowledge produced⁽²⁹⁾;

Stage 2. Development of specific research questions: refine the questions to obtain sufficient evidence to answer them⁽²⁹⁾;

Stage 3. Identification of how the findings and recommendations will be used: formulate the purpose statement to identify how the review findings will be used by the target audience. The use of review products is a key element in rapid realist review methodology⁽²⁹⁾;

Stage 4. Development of research terms: collaboratively identify terms that may be relevant to the scope, purpose, and question of the research project⁽²⁹⁾;

Stage 5. Identification of articles and documents to include in the review (indexed and gray literature): structure list of publications as identified by knowledge users and subject matter experts. Use the terms identified for structuring lists of documents that may be included in the review. The search strategy should be robust but not comprehensive and exhaustive⁽²⁹⁾;

Stage 6. Quality review: define search terms based on the most relevant results for the review. Researchers should simultaneously check with knowledge users and outside experts to identify publications which would be considered fundamental to the review. Contact with specialists to identify publications accelerates the search process of texts. This step combined with the validation phase (number 8 below) helps to include important texts⁽²⁹⁾;

Stage 7. Data extraction from the literature: an important step for structuring an extraction model of the described elements of the publications that can contribute to answering the questions asked. Extracting data using identical methods to that proposed by the traditional realist revision. The findings are analyzed to construct a form of realist program theory that addresses the focus and agreed scope of the review⁽²⁹⁾;

Stage 8. Validation of findings with experts: content review by subject specialists and who have direct experience in the field to fill gaps which are not identified in the literature⁽²⁹⁾;

Stage 9. Synthesis of results in a final report: present a format that meets the needs of knowledge users based on the results presented in stage 3, and the results produced by stages 7 and 8⁽²⁹⁾;

Stage 10. Discovery of results: working with knowledge users to discover the possibilities of applying the results of the review within the context. Program theories are presented as a tool for understanding how changes in context have interacted with mechanisms to produce results of interest. Program theories also help to describe possible unintended consequences resulting from changes in context and their resulting interactions or iterations with the mechanisms⁽²⁹⁾.

DISCUSSION

The 23 studies included in this review presented definitions, purposes, theoretical framework and development stages of the realist review, constituting an innovative methodology for integrating knowledge for health policies. It is possible to identify several methodologies and denominations of literature reviews with a wide variety of strategies and objectives. However, frequent inconsistencies or overlaps between the definitions of nominally different types of revision have been described, which requires caution in the choice and use of the methodology by the executing team and revision of the terminologies for greater consistency and safety in the application^(15,40).

In the realist review area, revisions were identified that did not report: the term “realist”, although they demonstrated the proper stages of this methodology; transparency in the selection, evaluation, analysis and synthesis of studies, thus demonstrating limitations of uniformity in the practical development of the review⁽¹¹⁾.

Quality and timely access to evidence, collaboration, and relationship building with research formulators were considered important and innovative factors to influence and strengthen the use of evidence⁽⁴¹⁾. New research should focus on the gaps of current methods and develop strategies for updating and comparing different types of reviews, focusing on the greater usefulness, validity and applicability of the reviews⁽³⁹⁾.

Realist review is not standardized or reproducible as it is in traditional systematic reviews, since it is interactive and cyclical, extolling the guiding principles rather than the rules that regulate it, linking quality processes to judgment, explicitness and reflexivity of the reviewers. The results are never generalizable, since all their conclusions are contextual and reflect the epistemological and ontological assumptions of the group of researchers^(6,28).

From the HDM perspective, these non-linear and dialectical characteristics confer dynamicity, historical contextualization, consideration of the various angles that may be involved and stakeholder participation to the review work in formulating theories about the program under analysis. These characteristics indicate the potential of the realist review to find the essence of program functioning, the real answer to health needs. In addition, participatory methods are fundamental to understanding the reality under analysis and to highlighting the possibilities of transforming it, since concrete experiences provide data and interpretations which external researchers have difficulty grasping⁽¹³⁾. Another issue that differentiates the realist review from the traditional systematic review is the search for evidence. Despite some similarities in the methodology of the systematic and realist review, important differences were cited. Traditional systematic reviews effectively focus on the search in a judicious way to minimize bias, but by disregarding details related to the complexity and context of the interventions, they run the risk of being overly simplistic and decontextualized, restricting the use of knowledge in decision-making⁽⁴²⁾. The realist review does not adhere to the strict hierarchy of evidence, where randomized clinical trials are automatically considered the best type of evidence-seeking research. Therefore, evidence from any source can be included such as policy reviews, qualitative and other evidence obtained from stakeholder consultations, focus groups with experts, and gray literature, among others⁽²⁷⁾.

Other potentialities related to the search phase of the realist review in relation to other reviews are: the search by type of research is not specified, but by relevance to the review question; it is not restricted to research or program evaluations per se, but is related to the theory behind the program; the cyclical, iterative and interactive process allows for partial advances in later stages; and it is possible to revise the criteria whenever necessary^(8,20).

Involving stakeholders from the early stages of the process to ensure the practical relevance of the final product is an important feature of the realist review. The use of an expert panel enables decision-making in policy contexts, especially in issues with limited literature, incorporating practical knowledge and research results according to the needs of those involved^(29,39).

In relation to the transformation of large systems, extensive consultations were required to obtain explicit recommendations of governmental actions which could increase the probability of success of complex transformation projects of the health system⁽⁴³⁾. Specifically in the rapid realist review, the possibility of losing references during the process is minimized by engaging the expert panel who validate the identified content and ensure that critical and emerging parts are considered⁽²⁹⁾. In a review on participatory research, six decision-making partners, funding agency representatives, public health organizations, ethics council, and a community-engaged scholarship organization were involved in the team⁽¹²⁾.

It is important to note that there is already a wide range of rapid review or brief review methodologies available in the literature. It is a review methodology with acknowledgment and unquestionable potential to respond promptly to the needs of caregivers, health managers and policymakers⁽⁴⁴⁾.

Thus, realist reviews face the challenge of training members of the review reference group and ensuring sustained engagement according to the availability and willingness of those involved, especially in political contexts with frequent changes in staff, including during project execution⁽²⁹⁾.

As a limitation, it is pointed out that this IR did not aim to exhaust the discussion about the issues involved in the realist review, as is the case of the theoretical basis of realism and the analysis of primary studies which used review as the methodology.

CONCLUSION

This IR enabled mapping the scientific production on realist review concepts and the methodology recommended for its development in the health area. There were no discrepancies of epistemological nature among the studies, which enabled reconciling the concepts and integrating the stages emerging from the included studies, without facing contradictions. It can be argued that the realist review has the potential to provide relevant and applicable subsidies to policy makers and decision-makers; to show what works and expose the contradictions (what does not work and why) of an intervention; and to provide a set of powerful elements for political decisions and to implement evidence in health.

It is indicated that there is radical participation of those involved and that the processes are not reduced to pointing out the more immediate contradictions, since they can thus compromise verification of the essential contradiction of the program.

The implications for the research involve the development of realist reviews in a transparent and clear way in the selection, evaluation, analysis and synthesis stages of studies. Discussions and training are also needed for the methodology to be expanded and used. The implications for policy-makers and decision-makers are related to involvement and understanding the methodology to support programs and policies considering the complexity of the health system, and focusing on the response to the health needs of different social groups.

REFERÊNCIAS

• ¹
  Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 1: what is evidence-informed policymaking? Health Res Policy Syst. 2009;7 Supl 1:S1-7.
• ²
  Spiegel JM, Breilh J, Yassi A. Why language matters: insights and challenges in applying a social determination of health approach in a North-South collaborative research program. Global Health. 2015;11:9.
• ³
  Allen L, Williams J, Townsend N, Mikkelsen B, Roberts N, Foster C, et al. Socioeconomic status and non-communicable disease behavioural risk factors in low-income and lower-middle-income countries: a systematic review. Lancet Glob Health. 2017;5(3):e277-e289.
• ⁴
  Northwood M, Ploeg J, Markle-Reid M, Sherifali D. Integrative review of the social determinants of health in older adults with multimorbidity. J Adv Nurs. 2018;74(1):45-60.
• ⁵
  Dias RISC, Barreto JOM, Vanni T, Candido AMSC, Moraes LH, Gomes MAR. Estratégias para estimular o uso de evidências científicas na tomada de decisão. Cad Saúde Colet. 2015;23(3):316-22.
• ⁶
  Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review – a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(1):21-34.
• ⁷
  Greenhalgh T, Kristjansson E, Robinson V. Realist review to understand the efficacy of school feeding programmes. BMJ. 2007;335:858-61.
• ⁸
  Tractenberg L, Struchiner M. Revisão realista: uma abordagem de síntese de pesquisas para fundamentar a teorização e a prática baseada em evidências. Ciênc Inf. 2011:40(3):425-38.
• ⁹
  Moher D, Stewart L, Shekelle P. All in the Family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more. Syst Rev. 2015;4:183.
• ¹⁰
  Pawson R. Evidence-based policy: the promise of ‘realist synthesis. London: Sage; 2002.
• ¹¹
  Berg RC, Nanavati J. Realist review: urrent practice and future prospects. J Res Pract. 2016;12(1):1-28.
• ¹²
  Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012;90(2):311-46.
• ¹³
  Soares CB, Campos CMS, Yonekura T. Marxism as a theoretical and methodological framework in collective health: implications for systematic review and synthesis of evidence. Rev Esc Enferm USP. 2013;47(6):1403-9. DOI: http://dx.doi.org/10.1590/S0080-623420130000600022
  » http://dx.doi.org/10.1590/S0080-623420130000600022
• ¹⁴
  Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546-53.
• ¹⁵
  Soares CB, Hoga LAK, Peduzzi M, Sangaleti C, Yonekura T, Silva DRAD. Integrative review: concepts and methods used in nursing. Rev Esc Enferm USP. 2014;48(2):335-45. DOI: http://dx.doi.org/10.1590/S0080-6234201400002000020
  » http://dx.doi.org/10.1590/S0080-6234201400002000020
• ¹⁶
  Willis JW. Foundations of qualitative research: interpretive and critical approaches. London: Sage; 2007.
• ¹⁷
  Cordeiro L, Soares CB, Rittenmeyer L. Unscrambling method and methodology in action research traditions: theoretical conceptualization of praxis and emancipation. Qual Res. 2016;17(4):1-13
• ¹⁸
  Pawson R. Evidence based policy: II: the promise of ‘realist synthesis. London: Centre for Evidence Based Policy; 2001.
• ¹⁹
  Wong G. The Internet in medical education: a worked example of a realist review. In: Hannes K, Lockwood C, editors. Synthesizing qualitative research: choosing the right approach. Chichester: John Wiley & Sons; 2012. p. 83-112.
• ²⁰
  Wong G, Westhorp G, Pawson R, Greenhalgh T. Realist synthesis RAMESES training materials. London: RAMESES Project; 2013.
• ²¹
  Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013;29;11:21. DOI: 10.1186/1741-7015-11-21.
  » https://doi.org/10.1186/1741-7015-11-21
• ²²
  Wong G, Greenhalgh T, Westhorp G, Pawson R. Development of methodological guidance, publication standards and training materials for realist and meta-narrative reviews: the RAMESES (Realist And Meta-narrative Evidence Syntheses-Evolving Standards) project. Health Serv Deliv Res. 2014;2:30.
• ²³
  Pawson R, Greenhalgh J, Brennan C. Guidance on guidelines: Understanding the evidence on the uptake of health care guidelines. J Eval Clin Pract. 2017;24(1):105-16.
• ²⁴
  Wong G. Getting to grips with context and complexity− the case for realist approaches. Gac Sanit. 2018;32(2):109-10.
• ²⁵
  Wong G. Realist reviews in health policy and systems research. In: Langlois EV, Daniels K, Akl EA, editors. Evidence synthesis for health policy and systems: a methods guide. Geneva: World Health Organization; 2018.
• ²⁶
  Tractenberg L. Applying knowledge cartography techniques and tools to facilitate the process of realist synthesis. Eletr J Bus Res Methods. 2013;11(2):105-15.
• ²⁷
  Kirst M, O'Campo P. Realist review methods for complex health problems. In: O'Campo P, Dunn JR, editors. Rethinking social epidemiology. New York: Springer; 2012. p. 231-46.
• ²⁸
  Edgley A, Stickley T, Timmons S, Meal A. Critical realist review: exploring the real, beyond the empirical. J Furth High Educ. 2016;40(3):16-330.
• ²⁹
  Saul JE, Willis CD, Bitz J, Best A. A time-responsive tool for informing policy making: rapid realist review. Implement Sci. 2013;5(8):103.
• ³⁰
  Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005;10(1):45-53.
• ³¹
  Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005;10 Suppl 1:6-20.
• ³²
  Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B, et al. Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012;7:33.
• ³³
  Jagosh J, Pluye P, Wong G, Cargo M, Salsberg J, Bush PL, et al. Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment. Res Synth Methods. 2014;5(2):131-41.
• ³⁴
  Gough D. Meta-narrative and realist reviews: guidance, rules, publication standards and quality appraisal. BMC Med. 2013;11:22.
• ³⁵
  Fawcett J. Thoughts about conceptual models, theories, and literature reviews. Nurs Sci Q. 2013;26(3):285-8.
• ³⁶
  Tricco AC, Antony J, Soobiah C, Kastner M, MacDonald H, Cogo E, et al. Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalization of the methodological steps. J Clin Epidemiol. 2016;73:29-35.
• ³⁷
  Wong G. Making theory from knowledge syntheses useful for public health. Int J Public Health. 2018;63:555-6. DOI: 10.1007/s00038-018-1098-2
  » https://doi.org/10.1007/s00038-018-1098-2
• ³⁸
  Randell R, Greenhalgh J, Dowding D. Using realist reviews to understand how health IT works, for whom, and in what circumstances. J Am Med Inform Assoc. 2015;22:e216-e217.
• ³⁹
  Tricco AC, Tetzlaff J, Moher D. The art and science of knowledge synthesis. J Clin Epidemiol. 2011;64(1):11-20.
• ⁴⁰
  Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91-108.
• ⁴¹
  Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;3(14):1-12
• ⁴²
  Pawson R. Evidence-based policy: a realist perspective. London: Sage; 2006.
• ⁴³
  Best A, Greenhalgh T, Lewis S, Saul JE, Carroll S, Bitz J. Large-system Transformation in health care: a realist review. Milbank Q. 2003;90(3):421-456.
• ⁴⁴
  Haby MM, Chapman E, Clark R, Barreto J, Reveiz L, Lavis JN. Designing a rapid response program to support evidence-informed decision-making in the Americas region: using the best available evidence and case studies. Implement Sci. 2016;11(1):117.

Publication Dates

• Publication in this collection
  2 Dec 2019
• Date of issue
  2019

History

• Received
  21 Aug 2018
• Accepted
  14 Mar 2019