Policy on Integrative Health Practices of Federal District in Brazil: Evaluability Study

Lemos, Maria Panisson Kaltbach; Luiza, Vera Lucia

doi:10.1590/0103-1104202313708

ABSTRACT

Integrative and Complementary Practices in Health include different health care forms and have been present for more than decades in health services in the Federal District, where the implementation assessment of your policy is not completely defined. To assist in understanding reality, an Evaluability Study was carried out collaboratively, involving discussion with stakeholders, questionnaires and consultation of documents. The policy components were identified, organized and modeled. A simplified-logical-model and one of the operational-logical-models, validated by the participants, are presented in this article. For policy evaluation, the initial aspects of the modeled logical implementation chain were prioritized. It was identified as fundamental the promotion of the actors’ knowledge about the set of resources to the implementation of the Integrative Practices services and about the related intersectoral responsibilities. Formative, qualitative and collaborative assessments are suggested. The modeling carried out in the studies of this policy, shed light on the study of similar ones.

KEYWORDS Health evaluation; Complementary therapies; Health policy; Public health; Health management

RESUMO

As Práticas Integrativas e Complementares em Saúde incluem diversas formas de cuidado de saúde e estão presentes há mais de três décadas nos serviços de saúde no Distrito Federal, onde a avaliação de implantação da sua política não está completamente definida. Para auxiliar na compreensão da realidade, foi realizado Estudo de Avaliabilidade, de forma colaborativa, envolvendo discussão com stakeholders, questionários e consulta a documentos. Foram identificados, organizados e modelizados os componentes da política. Um Modelo-lógico simplificado e um dos modelos-lógico-operacionais, com conteúdo validado pelos participantes, são apresentados neste artigo. Para avaliação da política, foram priorizados aspectos iniciais da cadeia lógica de implementação modelizada. Identificou-se como fundamental o fomento ao conhecimento dos atores sobre o conjunto de recursos necessários para implementação de serviços de práticas integrativas e sobre as responsabilidades intersetoriais relacionadas. Sugerem-se avaliações formativas, qualitativas e colaborativas. A modelização realizada possibilita continuidade nos estudos desta política, lançando luz ao estudo de políticas semelhantes.

PALAVRAS-CHAVES Avaliação em saúde; Terapias complementares; Política de saúde; Saúde pública; Gestão em saúde

Introduction

Traditional and Complementary Medicine (TCM) originate from various parts of the world and have their use recommended by the World Health Organization (WHO)¹. In Brazil, they are called Integrative and Complementary Health Practices (Pics), a term that refers to complex medical systems and other therapeutic resources that contribute to the global promotion of human care, including the expansion of individuals’ co-responsibility for health². Since 2001, the Federal District (DF) has defined Integrative Health Practices (PIS) as the set of technologies that address human health in its multidimensionality, promoting its role as protagonist, co-responsibility, emancipation, freedom and ethical attitude, to promote, maintain and recover one’s health³.

The National Policy on Integrative and Complementary Practices (PNPIC) for the Unified Health System (SUS) in Brazil was published in 2006, guiding the implementation of these practices in the country and sharing responsibilities with state management, but without defining or standardizing its evaluation2. Partial national monitoring takes place through computerized systems⁴, but municipalities are responsible for local monitoring of implementation and results⁵.

In 2015, 527,953 individual consultations were registered in the Brazilian Primary Health Care information system in Pics, in 1,362 municipalities⁶. However, the growth and expansion of Pics in the country does not mean institutionalization and sustainability of the PNPIC, without which there is fragility and instability of supply and difficulty in monitoring and evaluation to contribute to safety and quality⁷.

The multifactorial context of mismatches in the implementation of policies for integrative and complementary practices in health⁷,⁸,⁹,¹⁰,¹¹,¹², added to the lack of definition of ideal mechanisms for evaluating these practices⁴,¹³,¹⁴,¹⁵,¹⁶, makes studies necessary to better understand their operating logic and seek viable strategies for such evaluations.

The Evaluability Study (EA) has the potential to assist planning, contributing to the feasibility of proposed actions in the field of public health, helping to develop realistic objectives and to provide quick and low-cost feedback on implementation. It does not intend to generate evidence on effectiveness, but to identify the logic of the program, in addition to mismatches in the intervention¹⁷.

The EA is of particular value to managers who wish to promote organizational learning, which can serve as an instrument to transfer a program to new leadership, and to anyone planning an evaluation¹⁸. Also, by generating understanding of the mechanisms through which the intervention achieves its results, it may indicate necessary adjustments in objectives, activities or resources, increasing its potential to meet the needs of society¹⁷,¹⁹.

With more than three decades of history since the beginning of the institutionalization of these practices in public health in its territory, the DF can be considered a pioneer in Brazil²⁰. In the last quadrennium of management, 80% of the Primary Care Managements in the DF offered PIS²¹. The District Policy for Integrative Health Practices (PDPIS) – as the object of this study – enabled the translation of a reality that can contribute to the states of the federation.

Due to its strong potential to contribute to the implementation of Pics policies, the EA presented in this article was carried out, with the objective of expressing the PDPIS operating logic and identifying aspects of its implementation for monitoring and evaluation. This publication follows the quality standard suggested by Baratieri et al.²².

Materials and methods

The PDPIS²³ EA was carried out using a mixed research method, using document consultation, research meetings, a policy strategy spreadsheet and a questionnaire, following the steps described in box 1. Box 1 presents the research strategies and resources used in each evaluability stage, adapted from Wholey¹⁹ and Leviton et al.¹⁷. Quali-quantitative data were used for modeling, through frequency analysis and consensus techniques.

Thumbnail

Box 1
Research strategies used in the present study, in stages adapted from Wholey¹⁹ and Leviton et al.¹⁷. Federal District, 2020

Carrying out evaluability with policy stakeholders was a dynamic process, with information being aggregated, validated and adjusted throughout its development¹⁷. EA interactions allowed for adjustments in the logical design and evaluation intentions, based on the repeated use of research resources and the review of information obtained at each stage¹⁷,¹⁹. EA development took place between 2020 and 2021.

In the first stage of the research – focused on involving the interested parties – the context of the study was verified, consulting the available public documents. The DF has Brasília as its only municipality, with around three million inhabitants. The State Department of Health of the Federal District (SES-DF) has central and regionalized management, with no municipal health departments. Each of the seven Health Regions has a Superintendent as the highest authority and it is up to the regional management to act on the implementation of the PIS policy, as well as other health policies. The network has an exclusive PIS basic unit and two Live Pharmacies.

The Management of Integrative Health Practices (GERPIS), central management unit of SES-DF, was responsible for creating and updating the PDPIS. During the study, it was the unit responsible for promoting its implantation and implementation in the SUS-DF, including related educational processes.

The PDPIS, published in 2014 after public consultation, appreciation and approval by the SES-DF Management Collegiate and by the DF Health Council, included: Acupuncture; Art therapy; Self massage; Phytotherapy and Medicinal plants; Hatha Yoga; Homeopathy; Lian Gong in 18 therapies; Anthroposophical Medicine and Therapies; Meditation; music therapy; Reiki; Shantala; Tai Chi Chuan; and Integrative Community Therapy. In 2019, the following were also included in the DF health network: Laya Yoga, Stress Reduction Technique®, Ayurveda³,²⁴. In 2022, Auriculotherapy²⁵.

Each PIS had one or more public servants acting as District Technical Reference (RTD), linked to GERPIS, which contributed to the development of clinical management processes, with the qualification of PIS facilitators and with permanent education²⁶,²⁷. The health units had the figure of the PIS Facilitator, a qualified professional who organizes, supervises, informs, executes and monitors the implementation of the PIS individually or in groups²⁸.

In a meeting, the study and concepts used were presented to the GERPIS team. The research questionnaire was applied and validated, three participating Health Regions were randomly selected, information was obtained for selection and contact with other actors involved, and the groups of study participants were intentionally defined, based on the main people involved in the implementation of this policy²⁹.

Care professionals and central and regional managers of the SES-DF participated in the study, in addition to members of the Health Council of the DF. The selection criteria included professional work with PIS and Managements with the highest number of PIS implemented, prioritizing representativeness of the different PIS and seniority in the SES-DF (box 2).

Thumbnail

Box 2
Number of participants, selection criteria and forms of participation in the Evaluability Study, DF, June 2021

User representatives on the DF Health Council were the only stakeholders external to SES-DF. Stakeholders exclusively linked to secondary or tertiary care were not consulted, as primary care has historically been emphasized in the implementation of PIS¹,²,³.

For the second stage of the research – clarifying the intention of the policy and starting the modelling – the GERPIS group provided documents from the sector. In a meeting, the objectives and guidelines of the PDPIS were identified and aligned, following the views of the participants. The other stakeholders involved answered the survey questionnaire.

Official monitoring reports of the 2016-2019 District Health Plan and regulations, which are publicly accessible, were consulted. Also GERPIS internal documents, from 2020, containing its work processes³⁰ and records of meetings for the preparation of a new text for the PDPIS.

In view of the third stage – exploring the reality of PDPIS execution –, in addition to the documents and questionnaires, GERPIS professionals expressed their views on each of the 72 PDPIS strategies. A spreadsheet was prepared for this purpose, containing: execution, feasibility, expected results, how to measure results, pertinence in the guideline to which they belong and in relation to the policy objectives, clarity, which operational axis they could be part of, among the proposed axes, activities developed and comments.

In the fourth stage of the research – checking plausibility and adjusting the design –, in light of Leviton¹⁷, Thurston and Ramaliu¹⁸, Wholey¹⁹ and Rossi, Lipsey and Freeman³¹, the elements were rearranged and some original texts of the policy were modified, in order to elaborate the first complete version of the models intervention logic.

This process was based on the detailed material obtained. No information was discarded, so the most frequent ones served as the basis for modeling. The consultation of documents and regulations confirmed and facilitated the organization of contributions from the group of people involved.

The responses obtained in the previous steps were organized in an Excel spreadsheet for frequency analysis. The only open question in the questionnaire ratified information obtained. The proposed operational axes were used for classifying and aligning the strategies, as some generated doubts when reading or were too complex.

Some policy strategies integrated the model guidelines, others remained as strategies, and most were shifted as actions in each strategy. Five actions carried out that were not included in the PDPIS text were included, and twelve strategies were discarded by consensus on their non-implementation.

The Simplified Logical Model (MlogS) explained the relationship between the fundamental elements of the PDPIS, while the Operational Logical Models (MlogAt), one for each policy strategy, explained the execution of activities related to its implementation identified during the study (figures 1 and 2).

Figure 1
Simplified logical-model of the PDPIS 2014, from the Evaluability Study of the policy. Federal District, 2021 Purpose of the PDPIS (major objective): to promote, using the PIS, a health culture, with autonomy and empowerment of people and communities for self-care, in the SuS-DF, increasing the satisfaction of health professionals and users, the rationalization of health actions and the resolvability of the system.

Thumbnail

Figure 2
Operational logical model of strategy 4 of the PDPIS 2014, from the Evaluability Study of the policy. Federal District, 2021

The second and third meetings with the GERPIS group allowed for a deeper discussion on the models under construction, which were analyzed and validated by the group, after presenting the results of the questionnaires. There was a consensus suggestion for adjustments to better describe the reality of policy implementation.

Modeling was improved to increase emphasis on three aspects of policy implementation considered by the GERPIS group to be more relevant than represented in the proposed design: educational activities, community participation, and sharing information about PIS.

The fifth and sixth stages of the study – exploring possibilities – were initiated in the last meetings. Priorities and feasibility of monitoring and evaluating the PDPIS were discussed, based on the most frequent responses in the survey questionnaire and the logical models validated by the GERPIS group, as well as on the challenges for implementation and monitoring. All the prepared material was made available so that the participants of this group could register other answers or ideas, closing the stages of the research.

Under registration CAAE 35488720. 3.0000.5240, this research project was approved by the Research Ethics Committees of the Sergio Arouca National School of Public Health and the Health Sciences Teaching and Research Foundation, in the Federal District.

Results

Of the total number of study participants, 38 answered the questionnaire, 25 (65.8%) of whom were female. The complete level of education of the majority was specialization (52.6%), followed by higher education (26.3%), master’s (13.2%) and high school (7.9%). Restricting the analysis only to those who participated in the group of PIS facilitators, 52.9% had a postgraduate degree and 11.8% had no higher education.

The 10 years tenure of 77.1% of the 35 SES participants’ and their diversified professional background help to understand the institutional reality. The three user representatives on the Health Council were participants outside the SES.

GERPIS was made up of administrative technicians, a manager, RTD holders and collaborators of the PIS modalities, a senior civil servant responsible for monitoring and a multiprofessional postgraduate resident. The lead author of this study was part of the team as RTD.

During the stakeholder consultation, there was a lack of clarity in some guidelines and strategies in the policy text, with repetition of ideas. The document referred to the context at the time of its preparation, requiring updates, and presented a structure with little connection between its components, with strategies that were not hierarchical, prioritized, or clearly related to objectives or results.

3 guidelines were outlined in the first modeling with the GERPIS group. Later, one on professional education in PIS was added. The final version of the proposed MlogS of the PDPIS (figure 1) presents four guidelines, which define the lines followed in the operationalization, each one with an objective and two strategies for the construction of action plans. This is the first known modeling of the PDPIS elaborated in 2014. In the dynamics of the model’s operation, the products of the strategies are also feeding and feedback resources of the model and the desired results can exist if there is joint achievement of objectives.

The text of the PDPIS generically predicted the budget guarantee as a strategy for its institutionalization. Funding had been carried out by various sources, but nonspecific and with little definition, which is why MlogS does not make it explicit in the first guideline, translating it as an absent resource in the logic of the policy ’s operation, despite the trend of change in this scenario, which will require updates of the logic-model.

The word strategy came to represent the main sets of actions aimed at certain results, implying a reduction of the 72 in the text to only 08 strategies in the model. While the policy follows its purpose as a major objective, the strategies follow the specific objectives in each guideline. The design of the current functioning of each strategy, with the actions developed, even if partially, irregularly or in an initial phase, and their expected effects was represented in eight MlogAt, exemplified in figure 2.

The text of the policy only mentioned GERPIS for PIS management, care, teaching and research, without making explicit the responsibilities of other stakeholders, which weakens intersectoral actions inside and outside the SES-DF. However, growing movements of decentralization of management focused on PIS, such as standardization, action of supporters at the three levels of care and expansion of a basic unit exclusively for PIS, strengthen its implementation and monitoring.

The partnerships for the management or implementation of the PDPIS were also explicit, when 76% of the respondents in the different groups indicated the existence of, on average, 3.5 partnerships each, in their scope of action, mainly with schools or universities, community groups and other health units.

As for PIS services, they follow the same institutional norms used in the management of other health policies implemented in the DF. However, it did not seem clear to the participants of this study how the PIS were organized in terms of decentralization of management and responsibilities, in order to guarantee intersectoral commitment and the different resources needed to implement the PDPIS.

Regarding monitoring, the District Health Plans (PDS) included PIS offer indicators and the 2019 Annual Management Report indicated 100% achievement of the objective regarding the strengthening of PDPIS actions in the last quadrennium²¹. However, in the opinion of the GERPIS team, such indicators were limited to absolute numbers that do not translate or explore enough aspects of reality and the inadequacy of Pics codes in the SUS information systems leads to underreporting of services³².

Despite the limitations, the team highlighted that the indicators played a fundamental role in mapping SES-DF Network services and allowing the planning of new services. By integrating the PDS, they contributed to discussions on priorities in health management and to the expansion of PIS information, from senior management to the community.

Following it has favored the recording of other important data for monitoring, on the facilitators, number of assistance, intersectoral and educational actions, among others, thus existing a partial qualitative monitoring. Current monitoring is consistent with the proposed MlogS guidelines, despite not sufficiently supervising the expected effects in each of them, as explained by the GERPIS group.

In assessing the evaluability, the plausibility of the program is central to determine its potential to be evaluated, since it indicates whether it is feasible to achieve the proposed objectives with the activities foreseen in the continuous causal chain of the logical model¹⁷,³³,³⁴.

Stakeholders agreed, for the most part, on the main elements of the policy, which allowed modeling, linking them in a logical way. With the first four objectives of the PDPIS original text, 95% or more of the participants agreed totally or partially; with the fifth, 87%. No participant strongly disagreed. All declared themselves interested in implementing the PDPIS.

Another relevant data from the study was that 73.7% of the participants were at that time or had already been users of any of the PIS available by the SUS in the DF. This result corroborates the monitoring data, demonstrating implementation and access to PIS in the service network. The interest in implementing other PIS within its scope of action was registered by 73.6%, in a similar percentage for the different groups consulted, and the lack of interest may be linked, in the opinion of the GERPIS group, to the current implementation management limitations, such as operationalization of services, monitoring and technical follow-up.

Almost 80% of the participants stated that the community helps with the implementation, however 50% agreed totally or partially that there are professionals in their area or workplace who voluntarily make it difficult. Several participants worked in the same places, so this aspect needs to be further explored to be better understood.

The lack of support from managers was emphasized by participants in the questionnaires and meetings, with implications for the availability of resources and the functioning of services. Part of the participants reported never having been a PIS user or practitioner, which can be associated with a lack of knowledge, which is also a necessary resource for the implementation of the policy and can be a factor that hinders its implementation, increasing the importance of the policy.

The implementation of the PDPIS requires the availability of knowledge for the network, which, despite not being so explicit in the original text, not being uniform among the PIS modalities and not reaching all units, was part of the actions developed over the years, favoring the maintenance of activities with PIS in the network, which was confirmed by all data collection mechanisms.

Of the five activities added to the logical models in relation to the text of the PDPIS, all refer to the dissemination of information and knowledge about PIS. The model becomes plausible as these actions are included.

The response options provided for prioritization of evaluation by the participants were grouped into three groups: precursory aspects of the implementation of PIS services; aspects related to the functioning of these services; and aspects related to the expected effects with the implementation of the PDPIS (table 1).

Thumbnail

Table 1
Aspects indicated by the 38 stakeholders as priorities for evaluation in relation to the implementation of the PDPIS DF, 2020

The first set predominated in the general total of answers, with emphasis on the availability of resources and training or professional regulation for PIS. However, this did not have a uniform response among the consulted groups, as it was not prioritized by groups of central managers.

Only for the GERPIS group did the second set of aspects predominate in the answers, but the operationalization and improvement of PIS services was the third most prioritized option for evaluation by the participants in general, registered by all groups.

Apparently, the central management of PIS perceives itself to be in hold of issues that are precursors to the implementation of services that the rest of the stakeholders involved in this study are not. This reinforces the need for education on the use of PIS as care technologies. At the same time, monitoring does not seem to be enough to provide central management with sufficient information about the operation of network services, in order to improve management.

The knowledge and perception of the functioning of the PIS services, the mechanism of action and the rationale for these practices, seems to be a critical point in the implementation of the PDPIS. Without understanding and acting on this node, the implementation and execution of services becomes difficult, without which their purpose cannot be achieved.

It is noteworthy that the development of the study allowed expanding the vision of managers and others involved with the PIS on the policy and its implementation²⁰, as well as its results fostering communication and knowledge for all who access them.

The evaluative questions, the last stage of the EA carried out, must translate the desire and need for understanding of the actors, in addition to being able to show directions that add value to the evaluated object, recognizing its results or illuminating its limitations³⁵. T w o central questions were identified:

Do the actors involved know and exercise their responsibilities in providing the necessary resources for the implementation of PIS in light of the institutional norms of the SES-DF?
Do public servants qualified to provide assistance with PIS and their managers feel that they have the intellectual resources needed to perform the services?

Discussion

The centralized responsibility in GERPIS and the little definition of financial resources are aspects identified for improving the implementation of the PDPIS, which can be favored with this study. However, the implementation has been maintained and expanded over time, with a network of actors internal and external to the SES-DF, including users and those who work in a technical and educational way.

The structure of the PDPIS text included almost all the elements described by the Ministry of Health36 as necessary to fulfill its purpose. This study enabled the recognition and modeling of its constituent elements, advancing in relation to the necessary logical organization for monitoring and evaluation.

The PDPIS guidelines identified in this study coincide approximately with the components described for the Recife Municipal Policy for Integrative Practices³⁷. They “express achievement ideals and guide strategic and priority choices”³⁸⁽¹⁰⁴⁾. They indicate lines of action to be followed or paths of action for the elaboration of plans, programs, projects and activities that will make the policy operational, serving as a foundation for the actions of managers³⁶.

The guideline on guaranteeing access to homeopathic, anthroposophic, medicinal plants and herbal medicines, in the original text, despite the growth and appreciation of the two units of Live Pharmacy implemented, was not feasible, due to the unavailability of resources, technologies and structure, aspects mentioned by Mendes et al.³⁹.

To explore aspects of evaluation, monitoring or improvement of the policy, it is necessary to consider its innovative proposal in the current national context of health services and its implementation limitations, with clear weaknesses in monitoring. Even so, the offer and monitoring of these practices in the DF stand out, considering that the data available in Brazil on Pics is scarce and even the offer is still timid, despite the positive effects for users and services that use them¹⁵.

The logical design constructed and presented here can support other planning instruments, which can be part of or derived from the text of the policy, favoring the existence of parameters, criteria and indicators for planning, monitoring and evaluating its execution, in addition to specifying quantitative and qualitative goals, that may or may not be included in official government plans.

Considering that a policy is a general planning instrument, eliminating the need to quantify objectives, targets and resources³⁶, the use of more targeted and detailed management instruments than the policy itself would allow for easy readjustment and faithful representation of the implementation reality over the years, without the need to adjust the core of the policy, its purpose and its guidelines.

The set of modeled information allows a first look at the implementation reality, but requires periodic updates to represent reality. Likewise, the modeling of educational actions with managers and professionals may be able to assist in decision-making for future adjustments, with the potential to qualify the response of the health system⁴⁰.

The modeled intervention is comprehensive and complex, provoking questions and reflections that require flexibility and adjustments to the particularities of each context involved⁴¹,⁴². In this sense, the generated models were able to logically represent the main components for the operationalization of the policy under study, but could not bring about many advances in relational aspects that interfere with its implementation.

The human aspects need to be explored, as the interprofessional relationships themselves can affect and be positively affected by the implementation of Pics in health units, for a complex of reasons, including the hierarchy of powers in the traditional training of professionals in contrast to the greater horizontality in the dissemination and use of knowledge of Pics¹¹.

Despite the specific Pics policies in the country, the implementation often occurs independently by the interested professionals themselves¹⁰,⁴³, lacking resources for its realization. For this reason, studying Pics management can be a tendency to support the growth and quality of the offer⁴³, since the practice implemented, financed and developed directly by professionals, without management support, would not characterize a State policy, generating instability in the offer⁷.

Even with a good level of education – 65.8% of participants reporting a specialization or master’s degree – specific training for Pics is one of the limitations to the expansion process. There is little national regulation, insertion of the subject in teaching is incipient and experiences in permanent education in Pics are local⁹,¹⁰. Access to inputs is also fundamental for the quality of services and the resolvability of health care¹²,¹⁴.

It is known that there is a political and technical lack of preparation of health professionals to work with Pics in the SUS, being necessary, for its consolidation in the network, educational processes that train professionals in line with the SUS guidelines and with the principles of Collective Health¹⁵,⁴⁴. The training of professionals for Pics in the SUS should receive special attention, so that there is responsibility and understanding of its potentialities and limits, avoiding trivialization or misuse⁴³.

This is not restricted to PIS. Other studies on health promotion identified the need to strengthen formal education and expand articulated intersectoral actions⁴⁵, in addition to the desire to expand material and immaterial resources to consolidate processes⁴⁶.

Thus, with the purpose of answering questions and gaps in knowledge about the intervention, its operation and results, the importance of formative evaluations is highlighted, seeking to improve the program in the course of its execution and with the involvement of the actors⁴²,⁴⁷,⁴⁸.

It is suggested to focus on the set of resources necessary for the operation of PIS services in the health network, so that managers and other actors also appropriate intellectual resources on PIS, obtaining knowledge about the functioning of these services in the network, in order to complement their health care management framework.

Final considerations

Carrying out the study made it possible to clarify and align the elements of the PDPIS in a coherent way, in the current context, with the consensus of the interested parties involved. This makes it possible to explore aspects to be evaluated, monitored or improved.

The participation of those interested in the construction and validation of the modeling carried out facilitates the understanding and execution of elementary aspects of the logical chain of implementation of the policy throughout the network, which can have repercussions on improving its operation in the short, medium and long term.

In addition, the generated logical models can be adjusted whenever there are changes in the scenario, as well as be used for planning, executing, monitoring and improving the implementation of the PDPIS, in a network, to achieve its purpose.

The modeling of the district policy makes it possible to advance towards studies of each PIS and its operationalization in the Health Regions of the DF. Other levels of health care and the context outside the SES – limitations of this study – could also be more easily explored.

It is suggested to encourage the set of resources needed to implement the PDPIS together with conventional health services, with emphasis on knowledge about PIS and related sectoral and intersectoral responsibilities.

The elaboration of action plans derived from the policy can be a good opportunity to agree on these responsibilities, favoring decentralized actions to meet local needs. It is hoped that this article can contribute to the development of PIS in the country.

*
Orcid (Open Researcher and Contributor ID).

Financial support: The publication of this article was financially supported by the Deputy Directorate for Research and Innovation at ENSP/FIOCRuZ with the payment of the submission fee

Acknowledgements

To the State Department of Health of the Federal District and the Oswaldo Cruz Foundation (FIOCRUZ), that made the source research of this article possible, resulting from the Professional Master’s Degree in Public Health, from the National School of Public Health Sergio Arouca/FIOCRUZ.

To the team of the Management of Integrative Health Practices of the Federal District and to the other participants, who collaborated with the study and allowed a look at the reality of the implementation of the PDPIS.

To the Deputy Directorate for Teaching and Innovation at the Sergio Arouca National School of Public Health, who contributed to the publication of the article.

References

¹ World Health Organization. WHO traditional medicine strategy. 2014-2023. Geneva: WHO; 2013. 76 p.
² Brasil. Ministério da Saúde. Política nacional de práticas integrativas e complementares no SUS - PNPIC - SUS. 1. ed. Brasília: Ministério da Saúde; 2006. 92 p.
³ Distrito Federal. Política Distrital de Práticas Integrativas em Saúde: PDPIS. Brasília, DF: Fepecs; 2014. [acesso em 2022 nov 6]. Disponível em: https://www.fcm.unicamp.br/fcm/sites/default/files/2016/page/pdpic_distrito_federal.pdf
» https://www.fcm.unicamp.br/fcm/sites/default/files/2016/page/pdpic_distrito_federal.pdf
⁴ Losso LN, Freitas SFT. Avaliação do grau da implantação das práticas integrativas e complementares na Atenção Básica em Santa Catarina, Brasil. Saúde debate. 2017; 41(esp3):171-87.
⁵ Brasil. Ministério da Saúde. Manual de Implantação de Serviços de Práticas Integrativas e Complementares no SUS. Brasília, DF: Ministério da Saúde; 2018. [acesso em 2019 dez 3]. Disponível em: http://189.28.128.100/dab/docs/portaldab/publicacoes/manual_implantacao_servicos_pics.pdf
» http://189.28.128.100/dab/docs/portaldab/publicacoes/manual_implantacao_servicos_pics.pdf
⁶ Benevides IA. Medicina e terapias antroposóficas em 12 anos da Política Nacional de Práticas Integrativas e Complementares: Conceitos, normativas e resultados. Art. Méd. Ampl. 2018; 38(1):30-4.
⁷ Barbosa FES, Guimarães MBL, Santos CR, et al. Oferta de Práticas Integrativas e Complementares em Saúde na Estratégia Saúde da Família no Brasil. Cad. Saúde Pública. 2020; 36(1):e00208818.
⁸ Contatore OA, Barros NF, Durval MR, et al. Uso, cuidado e política das práticas integrativas e complementares na Atenção Primária à Saúde. Ciênc. saúde coletiva. 2015; 20(10):3263-73.
⁹ Paiva LF. Práticas Integrativas e Complementares Exercidas pelos Profissionais de Saúde no Sistema Único de Saúde: Revisão Sistemática sem Metanálise. [dissertação]. Florianópolis: UFSC; 2016. 177 p.
¹⁰ Tesser CD, Sousa IMC, Nascimento MC. Práticas Integrativas e Complementares na Atenção Primária à Saúde brasileira. Saúde debate. 2018; 42(esp1):174-88.
¹¹ Barros NF, Spadacio C, Costa MV. Trabalho interprofissional e as Práticas Integrativas e Complementares no contexto da Atenção Primária à Saúde: potenciais e desafios. Saúde debate. 2018; 42(esp1):163-73.
¹² Ribeiro AA. Dilemas na construção e efetivação da política nacional e distrital de práticas integrativas e complementares em saúde. [dissertação]. Brasília, DF: Universidade de Brasília; 2015. [acesso em 2019 out 24]. Disponível em: http://repositorio.unb.br/handle/10482/18141
» http://repositorio.unb.br/handle/10482/18141
¹³ Sousa IMC, Hortale VA, Bodstein RCA. Medicina Tradicional Complementar e Integrativa: desafios para construir um modelo de avaliação do cuidado. Ciênc. saúde coletiva. 2018; 23(10):3403-12.
¹⁴ Sousa IMC, Aquino CMF, Bezerra AFB. Custo-efetividade em práticas integrativas e complementares: diferentes paradigmas. J Manag Prim Health Care. 2018; 8(2):343-50.
¹⁵ Ruela LO, Moura CC, Gradim CVC, et al. Implementação, acesso e uso das práticas integrativas e complementares no Sistema Único de Saúde: revisão da literatura. Ciênc. saúde coletiva. 2019; (24):4239-50.
¹⁶ Habimorad PHL, Catarucci FM, Bruno VHT, et al. Potencialidades e fragilidades de implantação da Política Nacional de Práticas Integrativas e Complementares. Ciênc. saúde coletiva. 2020; 25(2):395-405.
¹⁷ Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33.
¹⁸ Thurston WE, Ramaliu A. Evaluability Assessment of Survivors of Torture Program: Lessons Learned. Can J PROGRAM Eval. 2005; 20(2):1-25.
¹⁹ Wholey JS. Exploratory Evaluation. Em: Handbook of Practical Program Evaluation. 3.ed. San Fancisco: Jossey-Bass; 2010. [acesso em 2021 ago 24]. Disponível em: http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf
» http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf
²⁰ Lemos MPK, Luiza VL. Contribuições do estudo de avaliabilidade para a gestão de práticas integrativas. In: Gouveia GDA, organizadoras. Práticas Integrativas em Saúde: uma realidade na atenção primária, especializada e hospitalar. São Paulo: Paco e Littera; 2022. p. 119-39. (Série: Saúde em Dia. v. 4).
²¹ Distrito Federal (Brasil). Relatório Anual de Gestão 2019. Brasília, DF: Secretaria de Estado de Saúde; 2020. [acesso em 2021 ago 9]. Disponível em: https://info.saude.df.gov.br/wp-content/uploads/2021/06/RAG2019_FINAL.pdf
» https://info.saude.df.gov.br/wp-content/uploads/2021/06/RAG2019_FINAL.pdf
²² Baratieri T, Nicolotti CA, Natal S, et al. Aplicação do Estudo de Avaliabilidade na área da saúde: uma revisão integrativa. Saúde debate. 2019; 43(120):240-55.
²³ Lemos MPK. Avaliabilidade da política de práticas integrativas em saúde do Distrito Federal na perspectiva da implementação na atenção primária. [dissertação]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz; 2021.
²⁴ Brasil. Ministério da Mulher, da Família, e dos Direitos Humanos. Portaria nº 371, de 03 de junho de 2019. Inclui Ayurveda, Laya Yoga e Técnica de Redução de Estresse (TRE® - Tension and Trauma Releasing Exercises) à Política Distrital de Práticas Integrativas em Saúde. Diário Oficial da União. 4 Jul 2019.
²⁵ Distrito Federal (Brasil). Secretaria de Estado de Saúde do Distrito Federal. Portaria nº 351, de 06 de maio de 2022. Inclui a Auriculoterapia na Política Distrital de Práticas Integrativas em Saúde. Diário Oficial do Distrito Federal. 6 Maio 2022.
²⁶ Distrito Federal (Brasil). Secretaria de Estado de Saúde do Distrito Federal. Portaria nº 1032, de 17 de setembro de 2018. Institui a Referência Técnica Distrital no âmbito da Subsecretaria de Atenção Integral à Saúde. Diário Oficial do Distrito Federal. 17 Set 2018.
²⁷ Distrito Federal (Brasil). Secretaria de Estado de Saúde Regimento. Decreto nº 39.546, de 19 de dezembro de 2018. Aprova o regimento interno da Secretaria de Estado de Saúde do Distrito Federal. Diário Oficial do Distrito Federal. 19 Dez 2018.
²⁸ Distrito Federal (Brasil). Secretaria de Estado de São Paulo. Portaria nº 175, de 3 de setembro de 2009. Institucionaliza as Práticas Integrativas de Saúde (PIS). Diário Oficial do Distrito Federal. 3 Set 2009.
²⁹ Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR. 1999 [acesso em 2020 fev 22]; 48(RR-11):1-35. Disponível em: https://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf
» https://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf
³⁰ Lemos MPK, Marques PFP. Sistematização dos processos de trabalho na gestão pública de práticas integrativas em saúde: relato de caso. Res Soc Dev. 2023; 12(1):e13312139692.
³¹ Rossi PH, Lipsey MW, Freeman HE. Evaluation: systematic approach. 7. ed. Thousand Oaks: Sage; 2004. 470 p.
³² Losso LN, Freitas SFT. Avaliação do grau da implantação das práticas integrativas e complementares na Atenção Básica em Santa Catarina, Brasil. Saúde debate. 2017; 41(esp3):171-87.
³³ Davies R. Planning Evaluability Assessment- synthesis of the literature with recommendations. 2013. [acesso em 2020 fev 22]. Disponível em: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf
» https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf
³⁴ Medina MG, Silva GAP, Aquino R. Uso de modelos teóricos na avaliação em Saúde: aspectos conceituais e operacionais. In: Hartz ZMA, Silva LMV. Avaliação em Saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Fiocruz; 2005. p. 41-63.
³⁵ Silva RR, Gasparini M, Alquezar E, et al. Avaliação para Negócios de Impacto Social - Guia prático. São Paulo: Artemisia; 2017. [acesso em 2021 ago 8]. Disponível em: https://move.social/wp-content/uploads/2017/11/2017_Guia-Pr%C3%A1tico_Avalia%C3%A7%C3%A3o-para-Neg%C3%B3cios-de-Impacto-Social_Artemisia_ABF_Move.pdf
» https://move.social/wp-content/uploads/2017/11/2017_Guia-Pr%C3%A1tico_Avalia%C3%A7%C3%A3o-para-Neg%C3%B3cios-de-Impacto-Social_Artemisia_ABF_Move.pdf
³⁶ Brasil. Ministério da Saúde. Sistema de planejamento do SUS: uma construção coletiva: formulação de políticas específicas de saúde. Brasília, DF: Ministério da Saúde; 2009. (Série: Textos Básicos de Saúde-Cadernos de Planejamento. v. 7). [acesso em 2021 set 28]. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/sistema_planejamento_sus_v7.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/sistema_planejamento_sus_v7.pdf
³⁷ Cazarin G, Lima SFF, Benevides IA. Avaliabilidade da Política de Práticas Integrativas e Complementares do município de Recife-PE. J Manag Prim Health Care. 2017; 8(2):203-15.
³⁸ Brasil. Ministério da Saúde. Manual de Planejamento no SUS. Brasília, DF: Ministério da Saúde; 2016. 138 p. (Série: Articulação Interfederativa. v. 4).
³⁹ Mendes LV, Campos MR, Chaves GC, et al. Disponibilidade de medicamentos nas unidades básicas de saúde e fatores relacionados: abordagem transversal. Saúde debate. 2014 [acesso em 2014 dez 30]; 38(esp):109-123. Disponível em: https://doi.org/10.5935/0103-1104.2014S009
» https://doi.org/10.5935/0103-1104.2014S009
⁴⁰ Guerra S, Albuquerque AC, Marques P, et al. Construção participativa da modelização das ações educacionais da estratégia de Planificação da Atenção à Saúde: subsídios para avaliação da efetividade. Cad. Saúde Pública. 2022; 38(3):e00115021.
⁴¹ Cardoso GCP, Oliveira EA, Casanova AO, et al. Participação dos atores na avaliação do Projeto QualiSUS-Rede: reflexões sobre experiência de abordagem colaborativa. Saúde debate. 2019; 43(120):54-68.
⁴² Champagne F, Brouselle A, Hartz Z, et al. A avaliação no Campo da Saúde: conceitos e métodos. Em: Brouselle A, Champagne F, Contandriopoulos AP, et al., organizadores. Avaliação - conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 217-38.
⁴³ Aguiar J, Kanan LA, Masiero AV. Práticas Integrativas e Complementares na atenção básica em saúde: estudo bibliométrico da produção brasileira. Saúde debate. 2020; (43):1205-18.
⁴⁴ Azevedo E, Pelicioni MCF. Práticas integrativas e complementares de desafios para a educação. Trab Educ e Saúde. 2011; 9(3):361-78.
⁴⁵ Prado NMBL, Medina MG, Aquino R. Intervenção intersetorial para promoção da saúde em sistemas locais: estudo de avaliabilidade. Saúde debate. 2017; (41):214-27.
⁴⁶ Dias MSA, Oliveira IP, Silva LMS, et al. Política Nacional de Promoção da Saúde: um estudo de avaliabilidade em uma região de saúde no Brasil. Ciênc. saúde coletiva. 2018; 23(1):103-14.
⁴⁷ Vieira-da-Silva LM. Avaliação de políticas e programas de saúde. Rio de Janeiro: Fiocruz; 2014.
⁴⁸ Brasil. Casa Civil da Presidência da República; Instituto de Pesquisa Econômica Aplicada. Avaliação de Políticas Públicas: guia prático de análise ex ante. Brasília, DF: Ipea; 2018. v. 1. 192 p. [acesso em ago 25]. Disponível em: https://www.ipea.gov.br/portal/images/stories/PDFs/livros/livros/180319_avaliacao_de_politicas_publicas.pdf
» https://www.ipea.gov.br/portal/images/stories/PDFs/livros/livros/180319_avaliacao_de_politicas_publicas.pdf

Publication Dates

Publication in this collection
30 June 2023
Date of issue
Apr-Jun 2023

History

Received
06 Oct 2022
Accepted
16 Jan 2023

Este é um artigo publicado em acesso aberto (Open Access) sob a licença Creative Commons Attribution, que permite uso, distribuição e reprodução em qualquer meio, sem restrições desde que o trabalho original seja corretamente citado.

[1] ¹ World Health Organization. WHO traditional medicine strategy. 2014-2023. Geneva: WHO; 2013. 76 p.

[2] ² Brasil. Ministério da Saúde. Política nacional de práticas integrativas e complementares no SUS - PNPIC - SUS. 1. ed. Brasília: Ministério da Saúde; 2006. 92 p.

[3] ³ Distrito Federal. Política Distrital de Práticas Integrativas em Saúde: PDPIS. Brasília, DF: Fepecs; 2014. [acesso em 2022 nov 6]. Disponível em: https://www.fcm.unicamp.br/fcm/sites/default/files/2016/page/pdpic_distrito_federal.pdf
» https://www.fcm.unicamp.br/fcm/sites/default/files/2016/page/pdpic_distrito_federal.pdf

[4] ⁴ Losso LN, Freitas SFT. Avaliação do grau da implantação das práticas integrativas e complementares na Atenção Básica em Santa Catarina, Brasil. Saúde debate. 2017; 41(esp3):171-87.

[5] ⁵ Brasil. Ministério da Saúde. Manual de Implantação de Serviços de Práticas Integrativas e Complementares no SUS. Brasília, DF: Ministério da Saúde; 2018. [acesso em 2019 dez 3]. Disponível em: http://189.28.128.100/dab/docs/portaldab/publicacoes/manual_implantacao_servicos_pics.pdf
» http://189.28.128.100/dab/docs/portaldab/publicacoes/manual_implantacao_servicos_pics.pdf

[6] ⁶ Benevides IA. Medicina e terapias antroposóficas em 12 anos da Política Nacional de Práticas Integrativas e Complementares: Conceitos, normativas e resultados. Art. Méd. Ampl. 2018; 38(1):30-4.

[7] ⁷ Barbosa FES, Guimarães MBL, Santos CR, et al. Oferta de Práticas Integrativas e Complementares em Saúde na Estratégia Saúde da Família no Brasil. Cad. Saúde Pública. 2020; 36(1):e00208818.

[8] ⁸ Contatore OA, Barros NF, Durval MR, et al. Uso, cuidado e política das práticas integrativas e complementares na Atenção Primária à Saúde. Ciênc. saúde coletiva. 2015; 20(10):3263-73.

[9] ⁹ Paiva LF. Práticas Integrativas e Complementares Exercidas pelos Profissionais de Saúde no Sistema Único de Saúde: Revisão Sistemática sem Metanálise. [dissertação]. Florianópolis: UFSC; 2016. 177 p.

[10] ¹⁰ Tesser CD, Sousa IMC, Nascimento MC. Práticas Integrativas e Complementares na Atenção Primária à Saúde brasileira. Saúde debate. 2018; 42(esp1):174-88.

[11] ¹¹ Barros NF, Spadacio C, Costa MV. Trabalho interprofissional e as Práticas Integrativas e Complementares no contexto da Atenção Primária à Saúde: potenciais e desafios. Saúde debate. 2018; 42(esp1):163-73.

[12] ¹² Ribeiro AA. Dilemas na construção e efetivação da política nacional e distrital de práticas integrativas e complementares em saúde. [dissertação]. Brasília, DF: Universidade de Brasília; 2015. [acesso em 2019 out 24]. Disponível em: http://repositorio.unb.br/handle/10482/18141
» http://repositorio.unb.br/handle/10482/18141

[13] ¹³ Sousa IMC, Hortale VA, Bodstein RCA. Medicina Tradicional Complementar e Integrativa: desafios para construir um modelo de avaliação do cuidado. Ciênc. saúde coletiva. 2018; 23(10):3403-12.

[14] ¹⁴ Sousa IMC, Aquino CMF, Bezerra AFB. Custo-efetividade em práticas integrativas e complementares: diferentes paradigmas. J Manag Prim Health Care. 2018; 8(2):343-50.

[15] ¹⁵ Ruela LO, Moura CC, Gradim CVC, et al. Implementação, acesso e uso das práticas integrativas e complementares no Sistema Único de Saúde: revisão da literatura. Ciênc. saúde coletiva. 2019; (24):4239-50.

[16] ¹⁶ Habimorad PHL, Catarucci FM, Bruno VHT, et al. Potencialidades e fragilidades de implantação da Política Nacional de Práticas Integrativas e Complementares. Ciênc. saúde coletiva. 2020; 25(2):395-405.

[17] ¹⁷ Leviton LC, Khan LK, Rog D. Evaluability Assessment to Improve Public Health Policies, Programs, and Practices. Annu Rev Public Health. 2010; 31(1):213-33.

[18] ¹⁸ Thurston WE, Ramaliu A. Evaluability Assessment of Survivors of Torture Program: Lessons Learned. Can J PROGRAM Eval. 2005; 20(2):1-25.

[19] ¹⁹ Wholey JS. Exploratory Evaluation. Em: Handbook of Practical Program Evaluation. 3.ed. San Fancisco: Jossey-Bass; 2010. [acesso em 2021 ago 24]. Disponível em: http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf
» http://www.orange.ngo/wp-content/uploads/2016/09/Handbook-of-Practical-Program-Evaluation.pdf

[20] ²⁰ Lemos MPK, Luiza VL. Contribuições do estudo de avaliabilidade para a gestão de práticas integrativas. In: Gouveia GDA, organizadoras. Práticas Integrativas em Saúde: uma realidade na atenção primária, especializada e hospitalar. São Paulo: Paco e Littera; 2022. p. 119-39. (Série: Saúde em Dia. v. 4).

[21] ²¹ Distrito Federal (Brasil). Relatório Anual de Gestão 2019. Brasília, DF: Secretaria de Estado de Saúde; 2020. [acesso em 2021 ago 9]. Disponível em: https://info.saude.df.gov.br/wp-content/uploads/2021/06/RAG2019_FINAL.pdf
» https://info.saude.df.gov.br/wp-content/uploads/2021/06/RAG2019_FINAL.pdf

[22] ²² Baratieri T, Nicolotti CA, Natal S, et al. Aplicação do Estudo de Avaliabilidade na área da saúde: uma revisão integrativa. Saúde debate. 2019; 43(120):240-55.

[23] ²³ Lemos MPK. Avaliabilidade da política de práticas integrativas em saúde do Distrito Federal na perspectiva da implementação na atenção primária. [dissertação]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz; 2021.

[24] ²⁴ Brasil. Ministério da Mulher, da Família, e dos Direitos Humanos. Portaria nº 371, de 03 de junho de 2019. Inclui Ayurveda, Laya Yoga e Técnica de Redução de Estresse (TRE® - Tension and Trauma Releasing Exercises) à Política Distrital de Práticas Integrativas em Saúde. Diário Oficial da União. 4 Jul 2019.

[25] ²⁵ Distrito Federal (Brasil). Secretaria de Estado de Saúde do Distrito Federal. Portaria nº 351, de 06 de maio de 2022. Inclui a Auriculoterapia na Política Distrital de Práticas Integrativas em Saúde. Diário Oficial do Distrito Federal. 6 Maio 2022.

[26] ²⁶ Distrito Federal (Brasil). Secretaria de Estado de Saúde do Distrito Federal. Portaria nº 1032, de 17 de setembro de 2018. Institui a Referência Técnica Distrital no âmbito da Subsecretaria de Atenção Integral à Saúde. Diário Oficial do Distrito Federal. 17 Set 2018.

[27] ²⁷ Distrito Federal (Brasil). Secretaria de Estado de Saúde Regimento. Decreto nº 39.546, de 19 de dezembro de 2018. Aprova o regimento interno da Secretaria de Estado de Saúde do Distrito Federal. Diário Oficial do Distrito Federal. 19 Dez 2018.

[28] ²⁸ Distrito Federal (Brasil). Secretaria de Estado de São Paulo. Portaria nº 175, de 3 de setembro de 2009. Institucionaliza as Práticas Integrativas de Saúde (PIS). Diário Oficial do Distrito Federal. 3 Set 2009.

[29] ²⁹ Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR. 1999 [acesso em 2020 fev 22]; 48(RR-11):1-35. Disponível em: https://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf
» https://www.cdc.gov/mmwr/PDF/RR/RR4811.pdf

[30] ³⁰ Lemos MPK, Marques PFP. Sistematização dos processos de trabalho na gestão pública de práticas integrativas em saúde: relato de caso. Res Soc Dev. 2023; 12(1):e13312139692.

[31] ³¹ Rossi PH, Lipsey MW, Freeman HE. Evaluation: systematic approach. 7. ed. Thousand Oaks: Sage; 2004. 470 p.

[32] ³² Losso LN, Freitas SFT. Avaliação do grau da implantação das práticas integrativas e complementares na Atenção Básica em Santa Catarina, Brasil. Saúde debate. 2017; 41(esp3):171-87.

[33] ³³ Davies R. Planning Evaluability Assessment- synthesis of the literature with recommendations. 2013. [acesso em 2020 fev 22]. Disponível em: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf
» https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf

[34] ³⁴ Medina MG, Silva GAP, Aquino R. Uso de modelos teóricos na avaliação em Saúde: aspectos conceituais e operacionais. In: Hartz ZMA, Silva LMV. Avaliação em Saúde: dos modelos teóricos à prática na avaliação de programas e sistemas de saúde. Rio de Janeiro: Fiocruz; 2005. p. 41-63.

[35] ³⁵ Silva RR, Gasparini M, Alquezar E, et al. Avaliação para Negócios de Impacto Social - Guia prático. São Paulo: Artemisia; 2017. [acesso em 2021 ago 8]. Disponível em: https://move.social/wp-content/uploads/2017/11/2017_Guia-Pr%C3%A1tico_Avalia%C3%A7%C3%A3o-para-Neg%C3%B3cios-de-Impacto-Social_Artemisia_ABF_Move.pdf
» https://move.social/wp-content/uploads/2017/11/2017_Guia-Pr%C3%A1tico_Avalia%C3%A7%C3%A3o-para-Neg%C3%B3cios-de-Impacto-Social_Artemisia_ABF_Move.pdf

[36] ³⁶ Brasil. Ministério da Saúde. Sistema de planejamento do SUS: uma construção coletiva: formulação de políticas específicas de saúde. Brasília, DF: Ministério da Saúde; 2009. (Série: Textos Básicos de Saúde-Cadernos de Planejamento. v. 7). [acesso em 2021 set 28]. Disponível em: https://bvsms.saude.gov.br/bvs/publicacoes/sistema_planejamento_sus_v7.pdf
» https://bvsms.saude.gov.br/bvs/publicacoes/sistema_planejamento_sus_v7.pdf

[37] ³⁷ Cazarin G, Lima SFF, Benevides IA. Avaliabilidade da Política de Práticas Integrativas e Complementares do município de Recife-PE. J Manag Prim Health Care. 2017; 8(2):203-15.

[38] ³⁸ Brasil. Ministério da Saúde. Manual de Planejamento no SUS. Brasília, DF: Ministério da Saúde; 2016. 138 p. (Série: Articulação Interfederativa. v. 4).

[39] ³⁹ Mendes LV, Campos MR, Chaves GC, et al. Disponibilidade de medicamentos nas unidades básicas de saúde e fatores relacionados: abordagem transversal. Saúde debate. 2014 [acesso em 2014 dez 30]; 38(esp):109-123. Disponível em: https://doi.org/10.5935/0103-1104.2014S009
» https://doi.org/10.5935/0103-1104.2014S009

[40] ⁴⁰ Guerra S, Albuquerque AC, Marques P, et al. Construção participativa da modelização das ações educacionais da estratégia de Planificação da Atenção à Saúde: subsídios para avaliação da efetividade. Cad. Saúde Pública. 2022; 38(3):e00115021.

[41] ⁴¹ Cardoso GCP, Oliveira EA, Casanova AO, et al. Participação dos atores na avaliação do Projeto QualiSUS-Rede: reflexões sobre experiência de abordagem colaborativa. Saúde debate. 2019; 43(120):54-68.

[42] ⁴² Champagne F, Brouselle A, Hartz Z, et al. A avaliação no Campo da Saúde: conceitos e métodos. Em: Brouselle A, Champagne F, Contandriopoulos AP, et al., organizadores. Avaliação - conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 217-38.

[43] ⁴³ Aguiar J, Kanan LA, Masiero AV. Práticas Integrativas e Complementares na atenção básica em saúde: estudo bibliométrico da produção brasileira. Saúde debate. 2020; (43):1205-18.

[44] ⁴⁴ Azevedo E, Pelicioni MCF. Práticas integrativas e complementares de desafios para a educação. Trab Educ e Saúde. 2011; 9(3):361-78.

[45] ⁴⁵ Prado NMBL, Medina MG, Aquino R. Intervenção intersetorial para promoção da saúde em sistemas locais: estudo de avaliabilidade. Saúde debate. 2017; (41):214-27.

[46] ⁴⁶ Dias MSA, Oliveira IP, Silva LMS, et al. Política Nacional de Promoção da Saúde: um estudo de avaliabilidade em uma região de saúde no Brasil. Ciênc. saúde coletiva. 2018; 23(1):103-14.

[47] ⁴⁷ Vieira-da-Silva LM. Avaliação de políticas e programas de saúde. Rio de Janeiro: Fiocruz; 2014.

[48] ⁴⁸ Brasil. Casa Civil da Presidência da República; Instituto de Pesquisa Econômica Aplicada. Avaliação de Políticas Públicas: guia prático de análise ex ante. Brasília, DF: Ipea; 2018. v. 1. 192 p. [acesso em ago 25]. Disponível em: https://www.ipea.gov.br/portal/images/stories/PDFs/livros/livros/180319_avaliacao_de_politicas_publicas.pdf
» https://www.ipea.gov.br/portal/images/stories/PDFs/livros/livros/180319_avaliacao_de_politicas_publicas.pdf

Stages	Aspect addressed	Research strategy	Main resources
Involving stakeholders in the assessment	What are the favorable and unfavorable stakeholders in the assessment whose perspective should be considered. What is the performance of each in relation to the PIS and how to approach it.	Validation of the Matrix of interested parties. Questionnaire validation.	1st Meeting with the GERPIS team. Matrix of interested parties. Questionnaire proposal.
Clarifying policy intent and beginning modeling	Engage stakeholders and determine the scope of the policy – objectives and guidelines. Logical relationship between PDPIS components.	Discussion with technical team. Construction of Logical Model. Collection and analysis of available information.	1st Meeting with the GERPIS team. Document verification. Questionnaires.
Exploring the Reality of Policy Enforcement	How is the implementation of PICS in the DF and the implementation of the policy under study – actions.	Discussion with technical team. Collection and analysis of available information. Construction of Logical Model.	1st Meeting with the GERPIS team. Activities worksheet. Document verification. Questionnaires.
Checking plausibility, validating and adjusting the design	The set of information obtained and organized in a logical way must be plausible and in accordance with the reality encountered.	Analysis, adjustment and validation of the elaborated policy logicmodels.	2nd and 3rd Meeting with the GERPIS team. Available documents. Questionnaires. Logic-models.
Exploring possibilities for evaluation and monitoring	Interest and feasibility of monitoring or evaluation.	Summary presentation of the material. Discussion with technical team. Alignment between documents, opinion of different actors and policy-logic models.	2nd and 3rd Meeting with the GERPIS team. Available documents. Questionnaires. Logic-models.
Checking policy implementation priorities and devel opment points	What aspects should be prioritized in the evaluation/monitoring. What are the other interests and needs in the use of the information obtained.	Summary presentation of the material. Discussion with technical team.	2nd and 3rd Meeting with the GERPIS team. Available documents. Questionnaires. Logic-models.

Participants	Expected No.	Effective No.	Selection criteria and strategy	Form of participation
Central technical group: members of GERPIS	10	13	Professionals who work at GER-PIS, assigned or not.	Face-to-face discussion and consensus meetings; provision of information and internal documents; and self-response in an electronic questionnaire, during a face-to-face meeting.
CSDF user Representatives	03	03	They will be invited into a CSDF meeting, based on their active participation during the meeting.	Assisted response to the questionnaire, in a virtual meeting.
District Secretary of Health	01	00	Holder of the position or person appointed by him.	Did not participate.
SAIS, COAPS, DAEAP	03	03	Holder of the position or person appointed by him.	Self-response in electronic questionnaire, assisted in person.
Superintendents	03	01	The superintendent of the 3 RA randomly chosen.	Self-response in electronic questionnaire.
DIRAPS	03	02	The director of the 3 RA randomly chosen	Self-response in electronic questionnaire
GSAP managers	03	03	1 GSAP in each of the randomly selected ARs, selecting the one with the highest number of PIS implanted.	Self-response in electronic questionnaire.
APS professionals qualified in PIS	15	17	One civil servant trained in each PIS among those assigned to the participating GSAP, even if inactive in practice, with preference for older civil servants at the SES.	Self-response in electronic questionnaire.
TOTALS	41	42

OBJECTIVE: Educar profissionais para implantação e uso qualificado de PIS no SUS do DF.
STRATEGY 4: Qualification of professionals to apply contemporary and traditional scientific knowledge of PIS.
Resources
• People with technical, traditional, educational and PIS management knowledge, responsible for activities • Material, technological and human resources • Establishment of criteria for qualification in PIS • Political and management support and interest
Activities	Product	Results for each action	Measurement of results
Regular offer of qualification courses for health workers in the SES-DF network.	Availability of qualification courses for institutional work with PIS at SES-DF.	Health professionals qualified to work institutionally with PIS. Providing care with PIS by health professionals uniformly qualified by the institution, according to its criteria and needs.	Number of public servants certified in the institution’s parameters, active and inactive. Repressed demands for courses.
Conducting permanent education (PE) for workers who deal with PIS, based on a diagnosis of the network’s needs.	Offer of a permanent education to the network’s public servants to work with PIS in the institutional reality.	Professionals with up-to-date and appropriate knowledge to work with PIS in the institutional reality. Increased resolving capacity of professionals in PIS services. Sharing experiences and expanding self-management in service. Constant updating of knowledge and techniques by the instructors/facilitators. Easier implementation and maintenance of PIS services.	Ability to resolve cases by facilitators for implementation and maintenance of the PIS offer. Skill and technical security of facilitators in conducting PIS services. updating of public servants in relation to institutional norms and technical guidelines for offering PIS on the network.
Carry out technicalscientific partnerships for training and permanent education.	PIS education projects carried out with partner sectors or institutions.	Expansion of education and work strategies with PIS. Increase in the quality of continuing education. Expansion of the number of beneficiaries with technical knowledge. Improvement of the PIS services of the SES-DF network with adherence to successful external trends in similar contexts.	Number of participants in the EP. Satisfaction of professionals with EP. Expansion of the PIS offer (number of appointments, units that offer and types of practices offered). Improved access to PIS. Improvement of work processes with implemented PIS.
Support the participation of professionals working with PIS in technical-scientific events.	Participation of professionals working with PIS in scientific events, with central support.	Dissemination of internal scientific work at external events. Performance with PIS based on scientific knowledge. Valuing local surveys with PIS. Incentive to scientific production in the DF. update of scientific knowledge for the internal network of SES-DF.	Number of technical or scientific events with PIS participation.
Strategic implementation of PIS courses for professionals from other agencies or public and philanthropic institutions.	Availability of PIS courses for public and philanthropic services outside SES-DF.	Professionals from the public or philanthropic sector qualified to work with PIS. Qualified use of PIS in public or philanthropic sectors outside the SES-DF.	Number of events, participants and PIS actions or services carried out in public or philanthropic services outside the SES-DF.

	GROUP OF STAKEHOLDERS
Aspects of the evaluation	Facilitators (17)	PIS managers (9)	Central health managers (3)	Regional health managers (6)	Health Council (3)	TOTAL (38)
Precursors of the implementation of PIS services: availability of necessary resources for execution (18); professional training for PIS or professional regulation for PIS (13); planning, conception, reasoning of the policy (7); scientific knowledge related to PIS (6); budget or financing of PIS-related policy and regulation (4 each); related popular knowledge (2); top management’s understanding of what it is and how important it is (1, in ‘others’)	27	7	5	11	5	55
Functioning of PIS services: operationalization and improvement of the services of each PIS (12); needs/demands of the target audience (10); favorable or unfavorable context and obstacles to implementation (5); feasibility of different PIS services (4); differences between implantation sites, strengths and weaknesses (3); how much each service is sought after or reaches the target audience, possibilities or effectiveness of attempts at improvement and if there was a planned implementation (2 each); support from local leaders (1, in ‘others’).	15	12	4	6	4	41
Effects of implementing the PDPIS: offer and experiences already carried out (6); range of expected or unexpected effects of the policy, in the short, medium or long term (4); effects of the implementation of each PIS (3); cost-benefit of practices (3); and elements necessary for the efectiveness of the Policy or each PIS (3)	8	8	2	1	0	19
TOTAL	50	27	11	18	9	115