Logic models from an evaluability assessment of pharmaceutical services for people living with HIV/AIDS

Modelos lógicos provenientes de estudo de avaliabilidade da assistência farmacêutica para pessoas vivendo com HIV/Aids

TEMAS LIVRES FREE THEMES

Angela Esher^I; Elizabeth Moreira dos Santos^II; Thiago Botelho Azeredo^I; Vera Lucia Luiza^I; Claudia Garcia Serpa Osorio-de-Castro^I; Maria Auxiliadora Oliveira^I

^INúcleo de Assistência Farmacêutica (NAF), Centro Colaborador da OPAS/OMS em Políticas Farmacêuticas, Escola Nacional de Saúde Pública Sérgio Arouca, Fundação Oswaldo Cruz. Rua Leopoldo Bulhões 1480, Manguinhos. 21041-210 Rio de Janeiro RJ. [e-mail]

^IILaboratório de Avaliação de Situações Endêmicas Regionais (Laser), Departamento de Endemias Samuel Pessoa (DEENSP), Escola Nacional de Saúde Pública Sérgio Arouca, Fundação Oswaldo Cruz

ABSTRACT

Brazil was the first developing country to provide people living with HIV/AIDS (PLWA) with comprehensive, universal, free access to antiretroviral medicines (ARV). Pharmaceutical services are considered a strategic action that has the goal of providing access to rational use of quality medicines while also promoting user satisfaction. User satisfaction is a complex concept, and evaluation models for pharmaceutical services for PLWA were not found in the literature. Therefore, an evaluation approach to help assess this issue had to be developed. This article seeks to describe a theoretical evaluation model of user satisfaction with the dispensing of ARV, developed as part of an Evaluability Assessment (EA). It presents a brief review of the EA and user satisfaction and describes the development of models created during the EA. The lessons learned in the process are presented as a conclusion.

Key words: Evaluability assessment, Theoretical model, User satisfaction, HIV/AIDS

RESUMO

O Brasil foi o primeiro país em desenvolvimento a fornecer medicamentos antiretrovirais (ARV) de forma integral, universal e gratuita às pessoas vivendo com HIV/Aids (PVHA). A Assistência Farmacêutica é considerada uma ação estratégica e busca prover acesso a medicamentos de qualidade, com uso racional, promovendo a satisfação dos usuários. Satisfação do usuá-rio é um conceito complexo e modelos para avaliarem serviços farmacêuticos para PVHA não são encontrados na literatura. Este artigo objetiva descrever o desenvolvimento de três modelos criados em um Estudo de Avaliabilidade (EA). É apresentada uma breve revisão dos conceitos de EA e de Satisfação do Usuário. As lições apreendidas no processo são apresentadas como conclusão.

Palavras-Chave: Estudo de avaliabilidade, Modelo teórico, Satisfação dos usuários, HIV/AIDS

Introduction

Brazil was the first developing country to provide people living with HIV/AIDS (PLWA) with comprehensive, universal, free access to antiretroviral medicines. The main objectives of the treatment are to reduce viral load and improve quality of life. It involves a long-term, continuous use of medicines that may cause different adverse effects and types of interactions.

Pharmaceutical services are considered a strategic action consisting of fundamental and interdependent components including selection, forecasting, procurement, storage, distribution, dispensing, and use of medicines^1,2. The operational effectiveness of this set of components depends on the adequate management of each of them, as well as on coordinated planning at the three levels of government administration - federal, state and municipal - in the National Health System. Pharmaceutical services can thus achieve the goals of providing access to quality medicines with rational use while also promoting user satisfaction¹.

Treatment of HIV infection requires frequent contact between users and pharmacy services. Such contact promotes an improvement of the pharmacist-user relationship, which in turn will result in the adequate counseling of clients regarding all aspects of rational use and drug therapy. Therefore, dispensing has an informative and educational role with a strong interactive interface between program staff and users³.

Care for PLWA takes place in a dynamic scenario, in which interactions with subjects reflect the existence of multiple social practices. These practices are permeated by concrete and symbolic social relationships. Patients and providers have different perceptions of issues such as pain management, treatment options, and effectiveness criteria. Such differences in perception may influence user satisfaction.

User satisfaction is a complex concept, and satisfaction evaluation models for any aspect of pharmaceutical services for PLWA were not found in the literature. Therefore, an evaluation outlook to help assess this issue had to be developed. In addition, the concepts of satisfaction proposed in the studies found in the literature were judged to be lacking.

The main purpose of this article is to describe the creation of the three models developed in this Evaluability Assessment (EA) of a National Evaluation of ARV dispensing program. The presentation of models is emphasized due to the need to implement these tools in evaluation practices concerning both user satisfaction and issues related to pharmaceutical services.

The article presents a brief review of EA and user satisfaction and describes the development of models created during the EA. The lessons learned in the process are presented as a conclusion.

Evaluability Assessment

Traditionally viewed as the initial step in an evaluation, the use of EA has expanded, and today it is employed as a planning tool and an effective method for clarifying the purpose of an evaluation^4,5. In the evaluative process cycle, the assessment usually concludes with the implementation of the evaluation. Development of the EA allows the evaluator to acquire important information on the program's theory and goals, as well as on the potential use of the findings by the evaluation's stakeholders. Such findings include the clarification of the program's objectives and operation, its outcomes, and impact indicators. EA may also promote a shared understanding among the stakeholders of the required intervention and evaluation⁶.

Contandriopoulos⁷ emphasizes that an important challenge of evaluation is to incorporate the different expectations and interests of stakeholders into its development, thus providing the evaluators with important and representative elements for a well-balanced judgment.

According to Thurston and Ramaliu⁸, an EA can be viewed as a formative evaluation that uses systematic inquiry and research methods. Thus, evaluability assessments are particularly valuable for managers who wish to promote organizational development, because they foster learning and improvement in institutional life.

According to Trevisan⁴, the technique has been used broadly in government programs, academic studies and scientific research. While the conceptions elaborated by renowned authors are the most widely used, the form in which the EA framework is applied can vary. The most recurring data collection techniques in studies are document reviews, field visits, and interviews. The author stresses the importance of properly documenting and justifying review or changes to the EA process and the addition of new models.

Thurston and Potvin⁹ propose a review of EA in order to turn it into a participative tool to plan evaluations instead of constituting a merely technical procedure applied on a single occasion at the beginning of an evaluation. From this perspective, EA could be applied at any moment of the evaluation planning and review. The authors view EA and evaluation as tools to promote social change. They define six important elements for the execution of EA: a) selection of an evaluability assistant - key stakeholders must be involved in the selection of evaluability assistant(s); b) identification of stakeholders; c) identification and evaluation of critical documents; d) development of the program's logic model and the evaluation plan; e) establishment of agreements to perform the evaluation; f) identification and evaluation of time factors and other required resources.

According to the authors, the desired products of an EA are a detailed description of the program, the main issues to be approached, an evaluation plan, and an established agreement among all stakeholders. Such products may easily become a formal program evaluation proposal. Ideally, they should be developed simultaneously to the program's planning, from beginning to end. Thus, both processes, EA and program planning, should be connected and inform one another⁹.

Object of Evaluation: User Satisfaction

The evaluation of health programs or services from the point of view of their users has been gaining importance, because they can provide health care managers with valuable information on how to improve health care. Users define standards for good quality, and identify the most significant aspects of the care received to be considered in the evaluation. Moreover, user satisfaction evaluations focus on the relationship between the users' system of values, beliefs and experiences, and the health service system. Data on satisfaction might inform whether the relationship pattern established between health care providers and users meets the latter's expectations, which are permeated by subjectivities and concrete health care needs¹⁰.

In the Brazilian context, the incorporation of satisfaction surveys in studies examining the country's public health system has to take into account the recent advances in the system's community participation mechanisms¹⁰. Encouragement of community participation in Brazil is premised on the idea that well-informed users can evaluate, intervene, and propose changes to improve the system. Methodologies that include the user's views are part of a framework that reaffirms the principles of individual and social rights, like those expressed in the concepts of humanization and patient rights¹¹.

Ware et al.¹² view satisfaction as a multidimensional construct in which the attitudes of patients are influenced by different care features. These attitudes relate to providers and services, and each of them has different effects on satisfaction.

Esperidião and Trad¹³ listed four main theoretical approaches for understanding user satisfaction, based on: attitude, in which satisfaction is viewed as the individual's evaluation of certain aspects of the service received; discrepancy, whereby the level of satisfaction is estimated as the difference between expectation and perception of the experience; fulfillment, or the difference between what is desired and what is obtained; and equity, in which individuals base their evaluation of the services in terms of personal input and output, and through comparisons with what is obtained by others, which introduces elements of social comparison.

There are models stating the importance of patient features, such as expectations, age, gender and schooling, on satisfaction. According to such models, these aspects should be described and controlled in the analysis, in order to support the validity of the conclusions¹⁴. Other models stress that certain aspects of care are discernible by users, and that they affect the satisfaction with care received with a certain degree of independence^12,15-17.

According to Ware et al.¹², although satisfaction with health services is influenced by each user's unique combination of characteristics and by other factors pertaining to the service, it is the latter that are mainly responsible for determining satisfaction.

Construction of Models

The need for an evaluation of the work process of medicine dispensing for the treatment of HIV/AIDS was proposed by the STD/AIDS National Program in Brazil. This program has been taking efforts to structure its evaluative activities, including the creation of a specific sector to deal with such issues.

Our work began with the composition of a General Coordination Group (GCG). The group included researchers with significant academic achievements and experience with evaluations in the field of pharmaceutical services, and managers of the STD/AIDS National Program sectors that would run the evaluation, namely Unit of Care and Treatment, and Evaluation Assistance. To support the cooperation efforts of all the actors, focal points were identified to represent the groups involved in the evaluation coordination.

In the initial meetings, the object of the evaluation and the intentions regarding the use of the results were defined. In consonance with the Brazilian organization model for the health system and the guidelines of the National Program, which states that the subjects are active participants, it was determined that the evaluation should contemplate the characteristics of users and suggest recommendations to improve the services provided. In the review of the literature the approach proposed by Ware et al.¹² was considered an adequate framework for the models to be constructed.

In the meetings with the GCG, the Evaluation Operational Group was created. This group was responsible for selecting and analyzing formal documents (ordinances and resolutions) and informal documents (texts from the internet and materials by NGOs) on medicine dispensing, as well as consulting with the professionals in charge of managing and executing medicine dispensing to PLWA. These activities supported the description of pharmaceutical services and of the dispensing component.

In order to raise awareness among managers and users, and to broaden the debate on the evaluation and its purposes, the project was presented to representatives of state-level administrations and to activists selected by these stakeholders, through a videoconference organized by the STD/AIDS National Program.

Concurrently, a web-based discussion list with leading actors of social movements fighting against AIDS was used to promote our proposal and gather valuable information about concrete daily situations experienced by people who use the dispensing units. This allowed the inclusion of views and values independent from the official perspective and provided a baseline for user satisfaction issues.

After defining the activities that comprised the dispensing process and constructing the dimensions structuring user satisfaction, the materials were once again presented for discussion with the stakeholders.

The development of the first model was meant to outline explicitly the dispensing process and its stages. The second model aimed at understanding the factors potentially involved in the phenomenon of user satisfaction with health services. The third model had the purpose of integrating both approaches, signaling aspects to be investigated operationally. The models and the steps taken to construct them are presented in the following sections.

Models

Faced with the challenge of evaluating user satisfaction with the dispensing of ARV, we sought to address the question through the development of three models, namely: a) Dispensing Logic Model; b) Theoretical Satisfaction Model; and c) Theoretical Evaluation Model of User Satisfaction with dispensing.

We have tried to address two different issues, that is, first to clarify our evaluand object and second to look for a plausible evaluation model. To clarify our object, we looked at two main points. First, we described the intervention including the chain of planned activities. Then we built a theoretical satisfaction model to account in the dispensing logic model, for mediators that express interactions which contribute to explain whether or not the program achieves the expected effects. Logic models generally emphasize the technical components of a program, and theoretical models emphasize presumed factors that might influence the magnitude of the effects. For example, to assess the dimensions of satisfaction with availability, opportunity and accommodation, mediators such as gender, race, engagement in HIV/Aids activism and the decision-making flow need to be addressed.

Finally, the development of a theoretical evaluation model of user satisfaction with dispensing detached from the two previous models aims to emphasize the hybridization of the program's technical description with the theory of the evaluand subject, as well as the need to add a causal path highlighting context moderators to the evaluation approach.

Dispensing Logic Model (DLM)

The Dispensing Logic Model (DLM) is a useful tool to clarify the intervention's technical components characterizing the flow among these components, which allows the program to function. It is a fundamental step in the construction of program evaluation models¹⁸.

In the context of our investigation, discussions with stakeholders and consultation of documents showed that the dispensing process was not formally defined. The GCG decided to use the publication "Assistência Farmacêutica para Gerentes Municipais" (Pharmaceutical Services for Municipal Health Managers) as a reference for this model, since this book has been often employed for training purposes.

The DLM (Figure 1) presents the technical components required for achieving the main goals of dispensing: promotion of rational use of medicines and enforcement of compliance with the prescribed regimen. The components considered in the model, which will be taken into account regarding satisfaction, are: welcoming, prescription check, separation and preparation of medicines, delivery of medicines, counseling. For each technical component, the DLM also considered its required input and activities.

For the purposes of this article, activities are understood as the organized work developed to achieve any planned objective. Input is defined as any resources used to make the planned action occur.

Welcoming begins at the moment the user arrives at the pharmacy, including his/her interaction with the pharmacy worker. This stage is influenced by factors related to the quality of the waiting area, particularly by the attitudes and behaviors of the provider when approaching users. Previous descriptions of dispensing activities do not include welcoming as one of the components of dispensing, but it was added to the model following suggestions given by users.

When checking the prescription, the pharmacy worker is expected to confirm whether the medical prescription is written in ink, in full, legibly, using the official nomenclature and system of weights and measures. The check also verifies that the prescription contains proper directions, the date, and legal requirement such as the health professional's signature, office or home address, and registration number in the respective professional board. Regarding nomenclature, the use of acronyms and abbreviations for the name of the medicine does not exempt the prescription from including the name of the medicine in full¹⁹.

Separation and preparation of medicines involves the proper identification of what is specified in the prescription regarding both the drug and its preparation. Depending on the routine established by the service, the pharmacy worker performing the activities may have greater or smaller decision-making power.

Delivery of medicines refers to these specific activities: rechecking and recording product specifications according to what is being prescribed, reporting and interacting with the prescriber. The recording process can vary according to the type of product, as certain medicines might be subject to special control or under a protocol. The interaction between the dispenser and the prescriber aims to clarify any doubts that may arise in relation to the product, or to information on its use.

The last technical component in dispensing involves counseling, which includes providing information on the use of medicines, explaining interactions with food and/or other medicines, recommending strict compliance to dose schedules, and describing adverse effects.

In addition to visually depicting the components related to satisfaction, the DLM is a tool for the integration of management activities and professional practices. Moreover, the DLM facilitates the establishment of more adequate monitoring systems, since certain activities that can improve the work process and increase user satisfaction if well executed are made explicit in the model.

Theoretical Satisfaction Model (TSM)

The following step yielded the Theoretical Satisfaction Model, providing an exploratory literature review on satisfaction and its indicators (for patients, clients, users, or consumers). The period reviewed was from 1980 to 2005, and the databases consulted were Medline (National Library of Medicine) and Lilacs (Latin American and Caribbean Health Sciences Literature). The WHO (World Health Organization) website was also used to obtain documents related to the concepts of interest. The keywords used in the review were: patient satisfaction, user satisfaction, pharmacy, pharmacy services, evaluation, health services, responsiveness, and humanization, in English and Portuguese. The references included in the reviewed papers were also used to locate interesting articles for discussion. A review of several instruments previously used to measure satisfaction was also performed^{12,14,16,17,20-21}.

This study accepts that satisfaction is related not only to a direct reaction of users to the services provided, but also to their history and expectations regarding the solution to problems, including subjective and objective aspects of their judgment of the service's performance¹¹. The TSM considers that previous experiences with health services create or modify the users' expectations towards new experiences (Figure 2). The perception that emerges from the current experience, permeated by these expectations, will influence development of the users' opinions and their resulting evaluation (judgment) of the care received. Also according to this model, other characteristics of users, such as their preferences, culture, and values, participate in the development of their opinions and attitudes, including their degree of satisfaction. This implies that, although undergoing the same experience, different individuals may express diverse opinions in terms of their satisfaction¹⁴. This understanding has led to the need to assess aspects related to age, gender, ethnicity, the perceived role of the health system, and socioeconomic factors as possible confounders in the measurement of user satisfaction. Once the effects of such factors are under control, it may be possible to include the level of satisfaction related to the experience with care in the evaluation.

In addition to allowing control over intervening variables, the organization of this model helps visualize the diversity of the intermediate aspects that qualify and contextualize the relationship between users and services. For example, it becomes possible to investigate how clients with different connections to the dispensing units will judge the dimensions that influence satisfaction. In Brazil, some PLWA may be treated by private physicians and only come to public centers to pick up their medicines, while others receive the entire range of care they need in the public clinics.

Theoretical Evaluation Model (TEM)

To develop the third model, the Theoretical Evaluation Model (TEM) of User Satisfaction with Dispensing, the evaluation team and the stakeholders agreed upon the evaluation question, the focus of the evaluation, and the evaluation approach. The DLM and TSM were used as reference for the construction of this model, guiding the investigation of how each main aspect - previous experience, expectations and perceptions - of the TSM related to each dispensing activity.

The TEM specified what intervention components would be evaluated, and described the dimensions and sub-dimensions applied to allow subjects to judge the interventions. Evaluation dimensions can be previously defined cognitive criteria or value categories applied to program components. The matrix associating dimensions, sub-dimensions and operational components of the program provides a useful map to depict the degree of user satisfaction (Figure 3).

The model shows that evaluating satisfaction involves the way in which the users judge dispensing, through components such as opportunity, availability and accommodation. The captions in the cells identify the activity in the dispensing process where the satisfaction component could or should be evaluated by the user. In this way, it is possible to assess user satisfaction globally, by considering the six sub-dimensions, and by measuring the evaluation according to each component.

The specific focus of this evaluation is on how users judge the activities comprising the intervention considering their current experience rather than the intervention's effect on the target population. This means that what is being evaluated is the work process of the dispensing operation and not the outcome of dispensing. For this reason, the selected dimensions used to structure user satisfaction were related to 'access'²² rather than 'quality'²³. Both, access and quality, are multi-dimensional categories. Access constitutes a useful category to summarize process evaluations, especially the operational aspects of actions that should precede evaluations of the cause-and-effect relationship, as discussed by Santos and Natal²⁴. Quality, according to Donabedian²³, involves operational program dimensions (optimization, acceptability, legitimacy) and program effects (efficacy, effectiveness, efficiency, and equity).

The concept of access to health services used here relates to the adjustment between clients' and system's features, that is, it refers to the relationship between obstacles encountered in the search for care (resistance) and the corresponding population capacity to overcome them, i.e., its "utilization power"^22,25. Thus, access is understood as a mediating category, capable of approaching and measuring the ability to create and offer services (or products) and their actual consumption²⁶.

The access and satisfaction categories represent heavily intertwined concepts^27-29. According to Penchansky and Thomas²², problems with any of the access dimensions influence client and system fitness in three measurable ways: (1) utilization of services; (2) shifts in user satisfaction; and (3) patterns in the organization of care, including less time devoted to each client and distortion of the use of emergency care. Thus, access as proposed by these authors²² and modified by WHO/MSH³⁰ supports operational evaluations and contributes to the establishment of some of the necessary theoretical approaches to user satisfaction.

The combination of the operational components of the program and the evaluation dimensions extended by means of sub-dimensions opened the evaluation model for inclusion of other judgment categories, such as humanization and responsiveness, which are critical in the context of HIV/AIDS treatment^11,31.

In Figure 3, three dimensions from the access category (opportunity, availability and accommodation) are used to organize the structure for the evaluation of user satisfaction³². These value dimensions define theoretically the domain for the operational indicators of the intervention's evaluation, allowing one to grasp user satisfaction or dissatisfaction with the service provided. Availability is defined by the relationship between the type and quantity of products and services needed and the type and quantity of products and services offered; opportunity refers to the existence of products and services at the time and place in which they are required technically in face of the users' health conditions; and accommodation refers to the fitness between the characteristics of the products and services and the expectations and needs of their users.

Access also considers acceptability, which refers to positive attitudes of both providers and clients concerning procedures and practices²². As discussed previously, accommodation is an important judgment dimension because it encompasses all aspects related to interpersonal relationships and ambiance. In this model, acceptability is diffused across the accommodation sub-dimension.

Affordability, another dimension of access which relates to costs, service prices, and the clients' ability to pay, was not included in this model, since in Brazil access is universal and free for all persons with HIV who need treatment, as provided under Law 9.313/96³³.

The dimensions chosen for the model were divided into sub-dimensions (Figure 3). The sub-dimensions were selected based on the literature covering access and user satisfaction, as well as the instruments used to measure user/patient satisfaction^{5,12,17,34,35}. Such sub-dimensions are: (a) convenience, which refers to the distance/ease of geographic access, organization of the service, amount of time spent by the user accessing service or different health units; (b) availability of a specific medicine in the pharmacy in the amount prescribed; (c) technical quality of dispensing, referring to the providers' competence and adherence to the essential components of dispensing; (d) technical quality of medicines, including the packaging and labeling of the medicines, adverse reactions, and resolution of symptoms; (e) ambiance, relating to the characteristics of the setting where care is provided (facilities should be consistent with minimum conditions including shelter, cleanliness, signage and clean drinking water); (f) interpersonal aspects, relating to the dispensers' attitude towards users and including respect for autonomy, for dignity, and for confidentiality of information.

Among the several sub-dimensions covered by the accommodation dimension, user perception of the quality of the dispensing process and of the medicines provided was assessed. Evidence observed after the application of the model suggests that these sub-dimensions should be reviewed, since the various components did not provide the accuracy required to explore the differences in the perceptions of users and providers regarding technical quality.

Lessons Learned

The literature showed that few studies have been published on user satisfaction with ARV dispensing. Therefore, there was an urgent need to develop and test models that contribute to the evaluation of medicine dispensing, including user satisfaction with this activity.

Program evaluations that use theory as a methodological strategy employ the construction of logic or theoretical models as a necessary part of the evaluative strategy^18,36,37. However, the construction of evaluation models has received little attention from the theoretical point of view in the literature³⁷. Articles on evaluation studies, especially in the field of health care, follow the structure of traditional research, rarely, if ever, prioritizing evaluability assessments, model construction, or reflection on criteria and value judgments of the evaluation process. Some authors^18,36,38,39 have shown that the use of theoretical/logic models is relatively simple and can be a useful tool in evaluation, allowing the program to be treated in depth, valuing its plurality, and explicitly specifying its effects. In the current study, the visual representations promoted the integration of different points of view in the evaluation process, allowing a broader understanding of the program to be evaluated and the immediate incorporation of the results into the decisions, supporting the improvement of the program.

The choice to use the concept of access, and especially its relational user/service aspect, allowed focusing on satisfaction regarding the program's function, i.e., dispensing activities. It should be noted that the dimensions proposed by Penchansky and Thomas²² did not include the subjects' expectations and wishes, but allowed for the inclusion of relevant aspects of the concept of satisfaction found in the literature. The inclusion of elements related to non-technical aspects of care, such as autonomy, privacy, dignity, confidentiality and setting for the care provided, extended the scope of the individual model of expectations and wishes to "socially legitimized expectations"⁴⁰.

The third model addressed the evaluation dimensions applied to the program components, intending to promote the use of the evaluation's findings. The evaluation dimensions in the model are presented as a comprehensive matrix in order to clarify the judgment categories, the program components and the tools used to collect data needed for the evaluation process.

This EA involved different stakeholders from the Brazilian STD/AIDS National Program and allowed activists in the fight against AIDS to understand the role dispensing plays in comprehensive pharmaceutical services provided to PLWA. User participation in discussions on the aspects to be evaluated contributed to the inclusion of elements to the initial conceptual discussion. One important example is the inclusion of welcoming as one of the components in the dispensing structure. According to users, experiences are extremely diverse, ranging from situations in which persons with HIV are greeted warmly to remarks that express discrimination or discomfort by the pharmacy workers assisting them. In any case, for the participants welcoming is clearly crucial for the relationship established in care and for the expectations generated from that point on.

Moreover, the interface with the actors taking part in the EA presented issues related the diversity of connections established between users and health systems and services. In the Brazilian context: a) persons who already used the public system to treat other chronic diseases, in which case the difficulties with previous experiences mean that they approach the AIDS program with low expectations, and, when they see that the program works and treatment dispensing functions regularly, their degree of satisfaction may increase; b) PLWA without any previous experience with the public health system who approach the system to receive ARV medicines free of charge, in which case the expectations are very particular in face of the absence of experiences with this type of service; c) users known for their participation as activists in movements against AIDS, in which case the relationships established with the service professionals often gives them more power to speak out and negotiate. Considering such diversity of connections and previous expectations, we can see the challenges and complexities inherent to the proposal of constructing models to represent the theory and the functioning of a program.

The proposal to create models that allow to periodically evaluate user satisfaction seeks to contribute to critical evaluative judgment, which is crucial for the emergence of strategies that encourage learning, reflection and overall improvement not only of dispensing but also of evaluation. The inclusion of the step-by-step construction of the models in the EA supports the elaboration of more robust evaluation models as well as the production of more legitimate and useful evidence from evaluation practices.

Collaborators

A Esher and EM dos Santos worked at all stages of drawing up the article (conception and elaboration). Azeredo TB participated in the development of models and in the writing. Luiza VL, CGS Osorio-de-Castro and MA Oliveira participated in the writing. All authors participated in the revision of the article.

Artigo apresentado em 19/06/2009

Aprovado em 23/01/2010

Versão final apresentada em 05/02/2010

• 1. Oliveira MA, Esher AFSC, Santos EM, Cosendey MAE, Luiza VL, Bermudez JAZ. Avaliação da Assistência Farmacêutica às Pessoas Vivendo com HIV/Aids no Município do Rio de Janeiro. Cad Saude Publica 2002; 18(5):1429-1439.
• 2. Marin N, Luiza VL, Osorio-de-Castro CGS, Machado-dos-Santos S. Assistência Farmacêutica para Gerentes Municipais Rio de Janeiro: OPAS/OMS; 2003.
• 3. Pepe VLE, Osorio-de-Castro CGS. A Interação entre Prescritores, Dispensadores e Pacientes: informação compartilhada como possível benefício terapêutico. Cad Saude Publica 2000; 13(3):815-822.
• 4. Trevisan MS. Evaluability assessment from 1986 to 2006. American Journal of Evalluation 2007; 28(3): 290-303.
• 5. Worthen BR, Sanders JR, Fitzpatrick JL. Avaliação de Programas: concepções e práticas. São Paulo: Editora Gente; 2004.
• 6. Trevisan MS, Huang YM. Evaluability assessment: a primer. Practical Assessment, Research & Evaluation 2003; 8(20):2-9.
• 7. Contandriopoulos A-P. Avaliando a Institucionalização da Avaliação. Cien Saude Colet 2006; 11(3):705-711.
• 8. Thurston WE, Ramaliu A. Evaluability assessment of a survivors of torture program: lessons learned. Canadian Journal of Program Evaluation 2005; 20(2): 1-25.
• 9. Thurston WE, Potvin L. Evaluability assessment: a tool for incorporating evaluation in social change programmes. Thousand Oaks and New Delhi: SAGE plublications 2003; 9(4):453-469S.
• 10. Oliveira CG. Breve análise da categoria de satisfação na obra de Donabedian [dissertação]. Rio de Janeiro: Universidade do Estado do Rio de Janeiro (UERJ); 1991.
• 11. Vaitsman J, Andrade GRBD. Satisfação e responsividade: formas de medir a qualidade e a humanização da assistência à saúde. Cien Saude Colet 2005; 10(3):599-613.
• 12. Ware JE, Snyder MK, Wright WR, Davies AR. Defining and measuring patient satisfaction with medical care. Evaluation and Program Planning 1983; 6(3-4):247-263.
• 13. Esperidião MA, Trad LAB. Avaliação de satisfação de usuários: considerações teórico-conceituais. Cad Saude Publica 2006; 22(6):1267-1276.
• 14. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Soc. Sci. Med 1997; 45(12):1829-1843.
• 15. Ware Jr JE, Snyder MK. Dimensions of Patient Attitudes Regarding Doctors and Medical Care Services. Medical Care 1975; 13(8):669-682.
• 16. Pascoe GC, Atkisson CC. The evaluation ranking scale: a new methodology for assessing satisfaction. Evaluation and Program Planning 1983; 6(3-4):335-347.
• 17. Mackeigan LD, Larson LN. Development and Validation of an Instrument to Measure Patient Satisfaction with Pharmacy Services. Medical Care 1989; 27(5):522-536.
• 18. Santos EM, Natal S. Dimensão técnico-operacional - Modelo lógico de programa Rio de Janeiro: Abrasco, Fiocruz, Programa Nacional de DST/Aids, Centers for Control and Prevention, Tulane University; 2005.
• 19. Brasil. Lei Federal nº. 5.991. Dispõe sobre o controle sanitário do comércio de drogas, medicamentos, insumos farmacêuticos e correlatos, e dá outras providências. Brasília: Diário Oficial da União 1973; 19 dez.
• 20. Bernhart MH, Wiadnyana IGP, Wihardjo H, Pohan I. Patient satisfaction in developing countries. Soc. Sci. Med 1999; 48(8):989-996.
• 21. Atkisson CC, Roberts RE, Pascoe GC. The Evaluation Ranking Scale: Clarification of methodological and procedural issues. Evaluation and Program Planning 1983; 6(3-4):349-358.
• 22. Penchansky DR, Thomas JW. The Concept of Access - Definition and Relationship to Consumer Satisfaction. Med Care 1981; 19(2):127-140.
• 23. Donabedian A. The seven pillars of quality. Arch Pathol Lab Med 1990; 114:1115-1118.
• 24. Santos EM, Natal S. Avaliação dos programas de controle da malária, dengue, tuberculose e hanseníase: estudo de casos municipais/estaduais na Amazônia legal Rio de Janeiro: DENSP, Escola Nacional de Saúde Pública (ENSP), Fiocruz; 2004.
• 25. Frenk J. El Concepto y la Medición de Acessibilidad. In: White KL, editor. Investigaciones sobre servicios de salud Washington D. C.: Organización Panamericana de la Salud(OPAS); 1992. p. 929-943.
• 26. Luiza VL, Bermudez JZ. Acesso a medicamentos: conceitos e polêmicas. In: Bermudez JZ, Oliveira MA, Esher A, editors. Acceso a medicamentos: Derecho fundamental, papel del Estado. Rio de Janeiro: ENSP; 2004. p. 45-66.
• 27. Blazer DG, Landerman LR, Fillenbaum G, Horner R. Health Services Access and Use among Older Adults in North Carolina: Urban vs Rural Residents. American Journal of Public Health 1995; 85(10):1384-1390.
• 28. Cunningham WE, Hays RD, Williams KW, Beck KC, Dixon WJ, Shapiro MF. Access to medical care and health-related quality of life for low income persons with symptomatic human immunodeficiency virus. Medical Care 1995; 33(7):739-754.
• 29. Mor V. Developing Indicators of Access to Care: The Case for HIV Disease. In: Millman M, editor. Access to Health Care in America Washington DC: The National Academy of Sciences; 1993. p. 199-218.
• ³⁰
  World Health Organization (WHO) Management Sciences for Health (MSH). Defining and Measuring Access To Essential Drugs, Vaccines, and Health Commodities Ferney, Voltaire, France: Health Organization (WHO), Management Sciences for Health (MSH); 2000.
• 31. Silva A. A framework for measuring responsiveness Geneva: World Health Organization (WHO);1999. (Global Programme on Evidence for Health Policy Discussion Paper Series: No. 32. EIP/GPE/EBD).
• 32. Esher A, Azeredo TB, Luiza VL, Oliveira MA, Santos EMd. Satisfação do usuário e responsividade do programa nacional de dispensação de medicamentos para o tratamento da aids no Brasil: discutindo importantes dimensões. In: Anais do III Congresso Brasileiro de Ciências Sociais e Humanas em Saúde; 2005; Florianópolis. p. 9-13.
• ³³
  Brasil. Lei nº 9313 de 13 de novembro de 1996. Dispõe sobre a distribuição gratuita de medicamentos aos portadores do HIV e doentes de Aids. Diário Oficial da União 1996; 14 nov.
• 34. Linder-Pelz S. Toward a Theory of Patient Satisfaction. Soc Sci Med 1982; 16(5):577-582.
• 35. Trad LAB, Bastos ACdS, Santana EM, Nunes MO. Estudo Etnográfico da Satisfação do Usuário do Programa de Saúde da Família (PSF) na Bahia. Cien Saude Colet 2002; 7(3):581-589.
• 36. Hartz ZMA. Avaliação dos programas de saúde: perspectivas teórico-metodológicas e político-institucionais. Cien Saude Colet 1999; 4(2):341-353.
• 37. Medina MG, Silva GAP, Aquino R, Hartz ZMA. Uso de modelos teóricos na avaliação em saúde: aspectos conceituais e operacionais. In: Hartz ZMA, Silva LMV, editors. Avaliação em saúde - Dos modelos teóricos à prática na avaliação de programas e sistemas de saúde Salvador, Rio de Janeiro: EDUFBA, Fiocruz; 2005. p. 41-63.
• 38. Hartz ZMA, Champagne F, Contandrioupolos A-P, Leal MdC. Avaliação do Programa Materno-Infantil: análise de implantação em sistemas locais de saúde no Nordeste do Brasil. In: Hartz ZMA, editor. Avaliação em Saúde: dos modelos conceituais à prática na análise da implantação de programas Rio de Janeiro: Fiocruz; 1997. p. 89-131.
• 39. Santos EM, Natal S. Dimensão técnico-operacional - Modelo teórico da avaliação Rio de Janeiro: Abrasco, Fiocruz, Programa Nacional de DST/Aids, Centers for Disease Control and Prevention, Tulane University; 2006.
• 40. Valentine N, Silva Ad, Kawabata K, Darby C, Murray CJL, Evans DB. Health System Responsiveness: concepts, domains and operationalization. In: Christopher J.L. Murray, Evans DB, editors. Health Systems Performance Assessment: debates, methods and empiricism Geneva:World Health Organization (WHO); 2003.

Logic models from an evaluability assessment of pharmaceutical services for people living with HIV/AIDS

Modelos lógicos provenientes de estudo de avaliabilidade da assistência farmacêutica para pessoas vivendo com HIV/Aids

Publication Dates

• Publication in this collection
  12 Dec 2011
• Date of issue
  Dec 2011

History

• Accepted
  23 Jan 2010
• Received
  19 June 2009