Using implementation science to evaluate mental health intervention: methodological proposal

Abstract

Objective

To describe a methodological proposal for evaluating the implementation process of an intervention focusing on the organization of mental health care in Primary Health Care from the perspective of implementation science.

Methods

Descriptive methodological study supported by the ImpRes-BR tool, which aims to guide the design of implementation research. The Mental Health in PHC intervention was described and its implementation strategies were systematized from the perspective of implementation science. From this, were presented the elements of the proposal for evaluating the implementation process of this intervention named SMAPS-API, guided by four domains of ImpRes-BR and prepared jointly by representatives of research and implementation teams.

Results

As a product of the ImpRes-BR domain related to the characteristics of implementation research, the objective of the SMAPS-API was defined, the intervention implementation stage was identified and the mixed methods multiple case study design was adopted. The Consolidated Framework for Implementation Research was considered as a structure for identifying the determinants of implementation. Implementation outcomes were defined as acceptability, adoption, appropriateness, feasibility, fidelity, penetration and sustainability potential. All decision makers involved were considered and the Power/Interest Matrix was used as a tool to understand the involvement and engagement of these actors in the implementation process.

Conclusion

The study demonstrates the path to defining variables and tools adopted to evaluate the implementation of an intervention to organize mental health care in Primary Care, contributing to the application of implementation science in Brazil.

Keywords
Implementation science; Primary health care; Mental health; Process assessment, health care

Resumo

Objetivo

Descrever proposta metodológica de avaliação do processo de implementação de uma intervenção com foco na organização do cuidado em saúde mental na Atenção Primária à Saúde, sob a ótica da ciência da implementação.

Métodos

Trata-se de um estudo metodológico descritivo, apoiado pela ferramenta ImpRes-BR, que visa guiar o delineamento de pesquisas de implementação. A intervenção Saúde Mental na APS foi descrita, e suas estratégias de implementação sistematizadas, na perspectiva da ciência da implementação. A partir disso, apresentaram-se os elementos da proposta de avaliação do processo de implementação dessa intervenção, denominada SMAPS-API, orientada por quatro domínios do ImpRes-BR e elaborada conjuntamente por representantes dos times de pesquisa e implementação.

Resultados

Como produto do domínio da ImpRes-BR referente às características da pesquisa de implementação, foi definido o objetivo da SMAPS-API, identificada a etapa de implementação da intervenção e adotado o delineamento de estudo de casos múltiplos com abordagem de métodos mistos. Considerou-se o Consolidated Framework for Implementation Research como estrutura para identificação dos determinantes da implementação. Foram definidos como desfechos de implementação aceitabilidade, adoção, adequação, viabilidade, fidelidade, capilaridade e potencial de sustentabilidade. Foram considerados todos os tomadores de decisão envolvidos e utilizada a Matriz Poder/Interesse como ferramenta para compreender o envolvimento e engajamento desses atores no processo de implementação.

Conclusão

O estudo demonstra o percurso para a definição de variáveis e ferramentas adotadas para avaliação da implementação de uma intervenção para organização do cuidado em saúde mental na Atenção Primária, contribuindo com a aplicação da ciência da implementação no Brasil.

Descritores
Ciência da implementação; Atenção Primária à Saúde; Saúde mental; Avaliação de processos em cuidados de saúde; Pesquisa sobre serviços de saúde

Resumen

Objetivo

Describir una propuesta metodológica de evaluación del proceso de implementación de una intervención con foco en la organización del cuidado en salud mental en la Atención Primaria de Salud, bajo la óptica de la ciencia de la implementación.

Métodos

Se trata de un estudio metodológico descriptivo, con el apoyo de la herramienta ImpRes-Br, que busca guiar el diseño de estudios de implementación. Se describió la intervención en salud mental en la APS y se sistematizaron sus estrategias, bajo la perspectiva de la ciencia de la implementación. Luego se presentaron los elementos de la propuesta de evaluación del proceso de implementación de esta intervención, denominada SMAPS-API, orientada por cuatro dominios de la ImpRes-Br y elaborada junto con representantes del equipo de investigación e implementación.

Resultados

Como producto del dominio de la ImpRes-Br referente a las características del estudio de implementación, se definió el objetivo de la SMAPS-API, se identificó la etapa de implementación de la intervención y se adoptó el diseño de estudio de casos múltiples con enfoque de métodos mixtos. Se utilizó el Consolidated Framework for Implementation Research como estructura para la identificación de los determinantes de la implementación. Se definieron como resultados de implementación la aceptabilidad, la adopción, la adaptación, la viabilidad, la fidelidad, la capilaridad y el potencial de sustentabilidad. Todos los tomadores de decisiones involucrados fueron considerados, y se utilizó la Matriz Poder/Interés como herramienta para comprender la participación y el compromiso de estos actores en el proceso de implementación.

Conclusión

El estudio muestra el recorrido para definir las variables y herramientas adoptadas para la evaluación de la implementación de una intervención para organizar el cuidado en salud mental en la Atención Primaria, lo que contribuye con la aplicación de la ciencia de la implementación en Brasil.

Descriptores
Ciencia de la implementación; Atención primaria de salud; Salud mental; Evaluación de procesos, atención de salud

Introduction

Considering the relevant and growing burden of diseases associated with mental health conditions, early interventions stand out as public health needs, especially in the community context.(¹,²) In this sense, Primary Health Care teams (PHC) have the potential to act in mental health care for the population.(²) However, it is necessary to strengthen the processes related to identifying the need for mental health care and organizing and offering care in the context of PHC.(³,⁴)

The Consolidation Ordinance Number 3 of 2017, which is an update of Ordinance Number 3.088 of 2011, reaffirms the central role of PHC in the coordination and longitudinal monitoring of people with mental disorders, highlighting the need to improve the actions carried out in this context. Several initiatives aimed at qualifying and organizing the mental health care work process in PHC have been described in the literature.(⁵-¹¹) Studies highlight that the current professional health training model does not adequately meet the demands in mental health in PHC, indicating the urgency of innovative educational strategies to fill this gap.(⁶-¹⁰)

One of the examples of attempts to reduce these gaps is the Mental Health Gap Action Programme Intervention Guide (mhGAP-IG).(¹²,¹³) The literature demonstrates that the use of the tool contributes to increasing knowledge and improving attitude and confidence of professionals who are not specialists in mental health.(³) Even though the mhGAP-IG is an instrument designed to reduce gaps in mental health care, the availability and use of the guide by health professionals is insufficient to transform care in PHC.(³) The need to understand and consider other factors interfering in the organization of mental health care in PHC is reinforced. These factors include not only aspects related to professional qualification, but also to the work process that permeates routine services.

From this perspective, implementation science (IS) emerges as a field of study with the potential to contribute to the identification and analysis of the factors involved in the processes of implementing efforts in different care contexts (such as PHC) in all their complexity, including barriers and facilitators for the success of a given intervention in the real world.(¹⁴) Note that IS has been equally useful both in evaluating implementation processes and the impact of interventions.(¹⁵-¹⁷)

In Brazil, IS is as a new field that has been gaining strength in the health area.(¹⁸-²³) Although multiple tools are available in the field of IS, there are gaps in the national scenario regarding the systematization of evaluation proposals for the implementation process. Among the challenges to be overcome, the weakness of the evidence dissemination process stands out, considering the cultural differences in Brazil and the lack of institutionalization of the use of evidence.(²¹)

Considering the power of IS to investigate in depth the mechanisms linked to the success of health efforts carried out in the real scenario of Brazilian health services, the aim of the present study was to describe a methodological proposal for evaluating the process of implementing an intervention focused on the organization of mental health care in PHC from the perspective of IS.

Methods

This is a methodological study focused on the construction of scientific instruments and paradigms. It has an educational meaning by demonstrating how the scientific innovation was proposed and executed with the intention of supporting future studies in the area of IS in Brazil.(²⁴) This work presents the path for outlining the proposal to evaluate the implementation process of the project Implementation of the Mental Health Care Line in PHC for Network Organization, known as Mental Health in PHC (Portuguese acronym: SMAPS) (http://bit.ly/3FeTy20). To contextualize the proposed evaluation of its implementation, the SMAPS is described below, covering its proponents and executors and the implementation strategies used. Finally, the tool used to guide the evaluation of the SMAPS implementation process is presented.

The SMAPS project was proposed and executed between 2021 and 2023 by the Projects and New Services area of the Directorate of Primary Care and Networks of the Sociedade Beneficente Israelita Brasileira Albert Einstein (SBIBAE) with support of the National Council of Health Secretaries (Portuguese acronym: CONASS) and the Ministry of Health through the Support Program for Institutional Development of the Unified Health System (Proadi-SUS). The SMAPS aims to support state and municipal health departments to organize the Mental Health Care in PHC units, strengthening the role of PHC in organizing the Psychosocial Care Network (Portuguese acronym: RAPS) in the Unified Health System (Portuguese acronym: SUS) provided in six Health Regions of three Brazilian federative units, covering 62 municipalities, 340 health services, 7,364 care professionals and 522 management professionals. Previous experience with implementing the Health Care Planning (HCP) methodology was established as a criterion for selecting the participating Health Regions.

The SMAPS uses two multifaceted implementation strategies: the HCP methodology(²⁵,²⁶) and training to use the mhGAP-IG. Both involve discrete strategies, for example, training, capabilities and development of relationships between interested parties; adaptations and adjustments to the context; support for professionals; offer of interactive assistance and use of evaluative and interactive strategies. Figure 1 presents the SMAPS implementation strategies in detail, according to the systematization recommended by Proctor et al.(²⁷)

Figure 1
Mental Health implementation strategies in PHC

The HCP method provides for a set of educational actions based on active methodologies and continuous improvement cycles developed through workshops and in-service training, which include moments of discussion and change in the modus operandi of teams and services. This involves the engagement of different decision makers, such as managers and technicians who work at the state, regional and municipal levels and their teams, including professionals who work in health units. In SMAPS, the HCP is performed according to the following steps:

• Previous and preparatory stages: presentation and invitation to states, regions and respective municipalities to join the intervention; definition of PHC services (at least one per municipality) and local actors to be involved.

• Thematic operational steps aimed at the discussion and organization of work processes, namely: (i) the organization of the Care Line in SMAPS: Diagnosis of RAPS; (ii) territory and population-based management in mental health: local diagnosis; family registration; territorialization; family risk stratification; identification of users with mental health care needs; (iii) access to RAPS: appointment schedule; reception/ambience; psychosocial interventions; collaborative practices; (iv) mental health care management: embracement; attention to acute events; stepped care approach; care programming; preparation and monitoring of the integrated care plan.

• Control stage: definition of an action plan for the sustainability of the intervention.

In addition to using HCP as a management, planning and organizational strategy based on the identification of the need for technical qualification of non-specialist professionals for mental health care, the SMAPS provides training for professionals to use the mhGAP-IG, which is an evidence-based decision tool for assessing, managing and monitoring priority conditions in mental health. The mhGAP-IG training takes place through face-to-face training including lectures and the use of active methodologies. The training of health professionals (ToHP) seeks to encourage the development of essential skills related to priority conditions in mental health. In order to allow the capillarization of this training, the training of professionals as multipliers is also planned, which is composed of ToHP content and a block to prepare them to replicate the content based on the discussion on teaching-learning models for adults (training of trainers and supervisors – ToTS). More details about the formation of mhGAP-IG can be found in a previous study.(²⁶)

The SMAPS implementation evaluation study, called SMAPS-API, was designed by the research team linked to the institution proposing the project (composed of coordination and researchers), together with a key representative of the implementation team (Project Specialist). Figure 2 describes the composition and relationship between the research and implementation teams. Meetings were held between the SMAPS coordination and researchers and the SMAPS specialist to define the scope of the study, identify the decision makers (part of the implementation team to be involved later in the study) and the project implementation stage, as well as well as the execution schedule.

Figure 2
Description of stakeholders and their support and reference relationships in Mental Health in PHC

A tool developed for implementation research titled ImPres-BR was adopted for the SMAPS evaluation proposal.(²⁸,²⁹) This tool presents step-by-step instructions for outlining implementation research and can be used to guide the design of research with different theoretical approaches, from those with the objective of describing or outlining the process of translating evidence into practice to those proposing to evaluate the implementation process,(²⁹) as is the case of the present study. The following ImpRes-BR domains were considered: characteristics of implementation research; theories, frameworks and implementation models; implementation outcomes; stakeholder involvement and engagement.

The study complied with national ethical standards for research involving human beings under opinion 6.093.305 (Certificate of Presentation of Ethical Consideration: 12395919.0.0000.0071).

Results

The presentation of the evaluation proposal for the SMAPS implementation process was guided by ImPres-BR considering the domains presented in chart 1.

Chart 1
Description of the proposal to evaluate the implementation process of the Mental Health in PHC project (SMAPS-API)

Characteristics of implementation research	Research objective: evaluate the implementation process of the SMAPS intervention for organizing mental health care in PHC
	SMAPS implementation stage: initial implementation
	Research design: multiple case study with a concomitant mixed methods approach, prioritizing qualitative data (qual + quant)
	Case selection: simple non-equiprobability random sampling to select the Health Regions and simple random draw weighted by the ratio between the number of Family Health Strategy teams per inhabitant, according to the IBGE population size categories to select the municipalities
	Qualitative data collection: individual semi-structured interviews and focus groups Quantitative data collection: consultation of secondary data and application of questionnaires and scales
Theories, frameworks and implementation models	The CFIR was adopted as a conceptual framework in order to enable the identification of the determinants that influence implementation success
Implementation outcomes	Acceptability, adoption, appropriateness, feasibility, fidelity, penetration and sustainability potential were adopted as the implementation outcomes. Possible unintended consequences of SMAPS were also considered
Stakeholder involvement and engagement	Stakeholders were all actors involved in the project. Users and health counselors were considered indirect recipients of the intervention and invited to contribute to the evaluation of the intervention. For the analysis of decision makers, it is proposed to use the Power/Interest Matrix

• SMAPS - Mental Health in Primary Health Care; PHC - Primary Health Care; IBGE - Brazilian Institute of Geography and Statistics; CFIR - Consolidated Framework for Implementation Research

With regard to the domain “Characteristics of implementation research” of ImPres-BR, it was identified that the SMAPS intervention is currently in the process of proposing, executing and managing changes, incorporating data systems and instruments, and starting improvement cycles. In this sense, the SMAPS-API was developed in the context of the initial implementation stage of the SMAPS. Still with regard to this domain of ImPres-BR, the objective of the study was defined in a way that allowed the evaluation of the implementation process and the identification of barriers, facilitators and adaptations at multiple levels of management and care. Therefore, it is expected to identify which strategies should be improved and/or adapted to the reality of each region or municipality, investigating how to sustain the intervention. The non-equiprobability simple random sampling selection method was used to select the participating regions and municipalities by means of a draw of three out of the six Health Regions eligible for the SMAPS-API, one per state participating in the intervention.

One municipality was selected from each category of the Brazilian Institute of Geography and Statistics (IBGE) through a simple random draw weighted by the ratio between the number of Family Health Strategy teams per inhabitant, resulting in the selection of 11 out of the 34 eligible municipalities. In each municipality, the PHC service that acts as the SMAPS laboratory in the municipalities and the decision makers involved were adopted as part of the research sample. Data collection in the SMAPS-API was systematized considering individual semi-structured interviews (online and/or in person) and focus groups with strategic decision makers from the implementation team, comprising the proponent group, managers (state, regional and municipal levels) and PHC care professionals. Municipal health advisors and users, identified as potential beneficiaries of the intervention, were also invited to participate in the SMAPS-API.

In parallel, quantitative data collection occurred by consulting secondary data in electronic citizen record reports from the participating PHC service and monitoring indicators for the implementation of the intervention through a management system (e-Planifica; https://planificasus.com.br/). Furthermore, after completing the interviews, a questionnaire with dichotomous questions was applied to measure the implementation outcomes and the acceptability, appropriateness and feasibility measurement scales. With regard to the domain of theories, structures and implementation models, the SMAPS-API considered the Consolidated Framework for Implementation Research (CFIR) for systematizing the data collection and analysis process in order to enable the identification of barriers and facilitators that have dialogic potential with the implementation outcomes of interest. The CFIR contributed to systematizing the determinants of implementation, which represented mediating agents in the implementation process from planning to evaluation. Attributes of the five domains foreseen in the CFIR were considered: innovation, individuals; inner setting; outer setting and implementation process. In addition to the CFIR, in order to clearly systematize the elements of SMAPS and SMAPS-API and establish relationships between them, the SMAPS-API Logical Model was developed (Figure 3) in joint construction by strategic actors from the implementation and research teams.

Figure 3
Logical model of the Evaluation of the implementation process of the Mental Health in PHC

The SMAPS-API implementation outcomes (Chart 1) considered the effects of implementation, serving as indicators of process success. As the evaluation of the implementation process occurs in the initial phase of implementation, there is no intention to measure the sustainability of the intervention at this moment. However, we seek to understand its sustainability potential after the end of SMAPS. Possible unintended consequences of SMAPS are also considered.

The semi-structured interview scripts used in SMAPS-API were prepared based on the CFIR and implementation outcomes. Regarding operational perspectives, SMAPS-API is interested in identifying SMAPS stakeholders. The proponents of the intervention are characterized as strategic actors, whose role includes mobilizing and/or neutralizing other decision makers according to their potential to support or oppose to institutional interests in the implementation process. In addition to proponents, the implementation team comprises actors who directly support the implementation of SMAPS and are linked to the executor institution. The group of decision makers included in the implementation team who are also receivers of the SMAPS comprises management and care professionals linked to state and municipal health departments with a protagonist role in the planning, operationalization and monitoring of SMAPS in their respective operating contexts (state, regional, municipal and service of PHC). Note that the involvement and engagement of decision makers occurs in different ways in the SMAPS. For the analysis of decision makers in SMAPS-API, the Power/Interest Matrix was applied, based on a specific method developed for this application.

Discussion

The ImPres-BR structure(²⁸,²⁹) was used in this study to describe strategies for evaluating the process of implementing an intervention to organize mental health care in PHC. Implementation Science tools already available in the literature were used and adapted, such as the CFIR, the systematization of the description of implementation strategies, the Power/Interest Matrix, the logical model and scales for measuring outcomes.(¹⁴,²⁷-²⁹,³⁰-³⁴)

Considering the gap in relation to the evaluation of the process and dynamics of implementation, as well as the systematization of the evaluation approach, the present study demonstrates the potential of using IS as one of the mechanisms for evaluating actions, programs, policies and/or projects in the health field in Brazil, contributing to generating evidence in the real world and translating knowledge into practice, providing opportunities for evidence-based decision making.(³⁴)

The SMAPS can be considered a complex intervention because of its multifaceted components, the different decision makers and the dynamics of interaction between them, coordinating the needs of the territories and the available evidence.(³⁵) Therefore, the CFIR was adopted to evaluate the factors that can influence the success of the implementation of the SMAPS.(¹⁴) It is quite widespread and used in the field of IS, consisting of a structure that systematizes the analysis of barriers and facilitators from five domains (innovation, internal context, external context, individuals and process), enabling a broader view of the elements composing the implementation.

As proposed by the ImpRes-BR, the engagement of stakeholders must be considered when designing implementation proposals. However, a considerable part of the efforts has been led by the academic community,(³⁴,³⁵) and there are few experiences that consider the involvement of health professionals at the care or management level in the development of interventions or design of studies, acting beyond the role of intervention recipients. In this sense, the present proposal is innovative in incorporating the SMAPS coordination as part of the research team and designing the study with a key representative of the implementation team.

By considering the different interactions and roles of those involved in designing the SMAPS-API, the research proposal could be informed by important elements such as the implementation stage, the recommended strategies and questions of interest for the implementers. Furthermore, the choice to involve different decision makers(²⁸,²⁹) and use the Power/Interest Matrix(³¹) results from the understanding that everyone is relevant to the intervention, although with variable engagement in the SMAPS. Greater opportunities to approach decision makers should be promising in order to maximize the application of results in the real scenario of health services.(³³)

Regarding implementation outcomes, the proposal was guided by the framework by Proctor et al.(³⁴) and considered the initial implementation stage of the SMAPS. The sustainability outcome, for example, refers to as how much an intervention is maintained or institutionalized in the service. It could not be captured at this stage, as the evaluation of its sustainability demands a certain period of time after the completion of the intervention. This is even described in the literature as one of the least investigated implementation outcomes.(³⁴)

However, given the relevance of measuring this variable and its relationship with other outcomes, considering that sustainability is the final phase of intervention feasibility and good capillarity also contributes to sustainability,(³⁴) it may be possible to investigate the “sustainability potential” in this proposal by investigating impressions collected in semi-structured interviews with the various decision makers during the initial implementation stage of the SMAPS.

Thus, by presenting a proposal that seeks to understand the determinants of the success of the SMAPS based on the experience of different actors and to evaluate the outcomes of its implementation, the present study adds to the advances achieved in the field of IS, currently concentrated on the international scene,(³²) contributing to expanding the debate in the Brazilian context. Finally, the exercise carried out reinforces the power of using the ImpRes-BR tool by actors with varying levels of IS expertise in design of research, since its application provides essential elements to direct and guide implementation efforts.

Among the limitations of the present study, is the recognition of other specific frameworks, models and theories that can also be applied in the evaluation of implementation processes. Furthermore, considering the initial stage of implementation, the SMAPS-API does not predict user and service outcomes, focusing on evaluating implementation outcomes. Future studies including other outcomes and sustainability assessments are suggested.

As strengths, the proposal of a methodological synthesis for evaluating implementation processes in the field of mental health in PHC in the light of IS stands out. The SMAPS-API proposal covers different regions of Brazil, enabling the translation and generation of knowledge in different contexts. In this sense, it is expected that the SMAPS-API will be able to provide subsidies for the maintenance and improvement of SMAPS and for future implementation in other Brazilian states.

Conclusion

The present study demonstrates the path to defining variables and tools adopted to evaluate the implementation of an intervention to organize mental health care in Primary Health Care. It reinforces the potential of IS as an approach to evaluation, in addition to planning and executing interventions so that they generate satisfactory results for care management. Considering implementation science as a relatively new field in Brazil and the diversity of tools and terminologies available in the literature, the present study can contribute to the debate and the design of future research to evaluate the implementation process in the national context.

Acknowledgements

The authors acknowledge the financing support by the SUS Institutional Development Support Project (law number 12.101 of November 27, 2009) through ordinance number 3.362 of December 8, 2017 –Initial Technical Opinion Recommendation for Technical and Financial Analysis of Project within the Scope of PROADI-SUS nº21/2021-CGMAD/DAPES/SAPS/MS25000.036837/2021-51.

Referências

• ¹
  GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Psychiatry. 2022;9(2):137-50.
• ²
  Patel V, Saxena S, Lund C, Thornicroft G, Baingana F, Bolton P, et al. The Lancet Commission on global mental health and sustainable development. Lancet. 2018;392(10157):1553-98. Erratum in: Lancet. 2018;392(10157):1553-98.
• ³
  Thornicroft G. Evidence-based mental health care and implementation science in low- and middle-income countries. Epidemiol Psychiatr Sci. 2012; 21(3):241-4.
• ⁴
  Wenceslau LD, Ortega F. Mental health within primary health care and Global Mental Health: international perspectives and Brazilian context. Interface (Botucatu). 2015;19(55):1121-32.
• ⁵
  Almeida PA. Consultas de enfermagem em saúde mental na atenção primária: análise das competências para enfermeiro de prática avançada [tese]. São Paulo: Faculdade Israelita de Ciências da Saúde Albert Einstein; 2023.
• ⁶
  Silva Filho JA, Bezerra AM. Acolhimento em saúde mental na Atenção Primária à Saúde: revisão integrativa. Id on Line Rev Mult Psic. 2018;12(40):613-27.
• ⁷
  Chazan LF, Fortes SL, Camargo Júnior KR. Apoio matricial em saúde mental: revisão narrativa do uso dos conceitos horizontalidade e supervisão e suas implicações nas práticas. Cien Saúde Colet. 2020;25(8):3251-60.
• ⁸
  Sampaio TC, Silva EC. Potencialidades do matriciamento em saúde mental: revisão narrativa. Cadernos ESP. 2022;16(3):62-74.
• ⁹
  Baptista JA, Camatta MW, Filippon PG, Schneider JF. Singular therapeutic project in mental health: an integrative review. Rev Bras Enferm. 2020;73(2):e20180508.
• ¹⁰
  Souza AP, Rezende KT, Marin MJ, Tonhom SF, Damaceno DG. Mental health promotion and protection actions aimed at the elderly in the context of primary health care: an integrative review. Cien Saude Colet. 2022;27(5):1741-52.
• ¹¹
  Kigozi-Male NG, Heunis JC, Engelbrecht MC. Primary health care nurses' mental health knowledge and attitudes towards patients and mental health care in a South African metropolitan municipality. BMC Nurs. 2023;22(1):25.
• ¹²
  MI-mhGAP Manual de Intervenções para transtornos mentais, neurológicos e por uso de álcool e outras drogas na rede de atenção básica à saúde. Versão 2.0. Brasília, DF: Organização Pan-Americana da Saúde; Organização Mundial da Saúde; Programa de Ação para Reduzir as Lacunas em Saúde Mental; 2018 [citado 2024 Mar 26]. Disponível em: https://iris.paho.org/bitstream/handle/10665.2/49096/9789275719572-por.pdf?sequence=1&isAllowed=y
  » https://iris.paho.org/bitstream/handle/10665.2/49096/9789275719572-por.pdf?sequence=1&isAllowed=y
• ¹³
  Keynejad R, Spagnolo J, Thornicroft G. WHO mental health gap action programme (mhGAP) intervention guide: updated systematic review on evidence and impact. Evid Based Ment Health. 2021;24(3):124-30.
• ¹⁴
  Damschroder LJ, Reardon CM, Widerquist MA, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):75.
• ¹⁵
  Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13:95.
• ¹⁶
  Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3:32.
• ¹⁷
  Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence-based practice: a conceptual framework. Qual Health Care. 1998;7(3):149-58.
• ¹⁸
  Dixon LB, Patel SR. The application of implementation science to community mental health. World Psychiatry. 2020;19(2):173-4.
• ¹⁹
  Treichel CA, Silva MC, Presotto RF, Leme KE, Reis MF, Amorim SG, et al. Teoria da mudança para implementação de apoio matricial em saúde mental. Acta Paul Enferm. 2023;36(s01):eAPESPE022473.
• ²⁰
  Silva ES, Primo CC, Gimbel S, Almeida MV, Oliveira NS, Lima EF. Elaboration and implementation of a protocol for the Golden Hour of premature newborns using an Implementation Science lens. Rev Latino-Am Enfermagem. 2023;31:e3957.
• ²¹
  Andrade KR, Pereira MG. Tradução do conhecimento na realidade da saúde pública brasileira. Rev Saude Publica. 2020;54:72.
• ²²
  Fernandes BD, Foppa AA, Ayres LR, Chemello C. Implementation of Medication Reconciliation conducted by hospital pharmacists: A case study guided by the Consolidated Framework for Implementation Research. Res Social Adm Pharm. 2022;18(9):3631-37.
• ²³
  Castral TC, Bueno M, Carvalho JC, Warnock F, Sousa JC, Ribeiro LM, et al. Implementation of a knowledge translation and exchange intervention for pain management in neonates. Acta Paul Enferm. 2023;36(s01):eAPESPE024073.
• ²⁴
  Demo P. Pesquisa e construção de conhecimento: metodologia científica no caminho de Habermas. 3a ed: Rio de Janeiro: Tempo Brasileiro; 1997.
• ²⁵
  Shimocomaqui GB, Masuda ET, Souza VG, Gadelha AK, Eshriqui I. Atenção ambulatorial especializada à saúde materno-infantil em regiões do PlanificaSUS. Rev Saude Publica. 2023;57: Suppl 3:3s.
• ²⁶
  Mendonça JM, Eshriqui I, Almeida LY, Gomes-Filho VV, Schunk L, Sousa AA, et al. Conhecimento de profissionais da atenção primária em saúde mental: diagnóstico pelo mhGAP. Rev Saude Publica. 2023;57:Suppl 3:3s.
• ²⁷
  Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013; 8:139.
• ²⁸
  Hull L, Goulding L, Khadjesari Z, Davis R, Healey A, Bakolis I, et al. Designing high-quality implementation research: development, application, feasibility and preliminary evaluation of the implementation science research development (ImpRes) tool and guide. Implement Sci. 2019;14(1):80.
• ²⁹
  King's Improvement Science. Laboratório de Interfaces. Ferramenta de desenvolvimento para pesquisa de implementação. Um guia prático para utilização da ferramenta (ImpRes-BR). ImpResBR; 2021 [citado 2024 Mar 26]. Disponível em: https://kingsimprovementscience.org/cms-data/resources/GUIA%20-%20ImpRes.pdf
  » https://kingsimprovementscience.org/cms-data/resources/GUIA%20-%20ImpRes.pdf
• ³⁰
  Creswell JW. Projeto de pesquisa: métodos qualitativo, quantitativo e misto. 3ª ed. Porto Alegre (RS): Artmed; 2010.
• ³¹
  Johnson G, Scholes K, Wittington R. Exploring Corporate Strategy. 8th ed. England: Pearson Education; 2008.
• ³²
  Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.
• ³³
  Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84.
• ³⁴
  Proctor EK, Bunger AC, Lengnick-Hall R, Gerke DR, Martin JK, Phillips RJ, et al. Ten years of implementation outcomes research: a scoping review. Implement Sci. 2023;18(1):31.
• ³⁵
  Lynch EA, Mudge A, Knowles S, Kitson AL, Hunter SC, Harvey G. "There is nothing so practical as a good theory": a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18(1):857.

Edited by

Associate editor: Thiago da Silva Domingos (https://orcid.org/0000-0002-1421-7468) Escola Paulista de Enfermagem, Universidade Federal de São Paulo, São Paulo, SP, Brazil

Publication Dates

• Publication in this collection
  02 Sept 2024
• Date of issue
  July 2023

History

• Received
  14 Aug 2023
• Accepted
  09 May 2024