DISCUSSANTS DEBATEDORES
Concepts and approaches in the evaluation of health promotion. Dilemmas in the evaluation of health promotion: how should Brazilian scientific output be oriented?
Concepções e abordagens na avaliação em promoção da saúde. Dilemas da avaliação em promoção da saúde: como orientar a produção científica brasileira?
Márcia Faria Westphal
School of Public Health, University of São Paulo, Department of Public Health Practice, Health Promotion Thematic Area. Center for Studies, Research, and Documentation on Healthy Cities (CEPEDOC), marciafw@usp.br
The authors of the article "Concepts and Approaches in the Evaluation of Health Promotion", from the National School of Public Health, have made an important contribution to Health Promotion evaluation in various ways, which will be specified below.
Their article critically analyzes the trend to overvalue evaluation as a methodological instrument producing signs scientific evidence of the effectiveness of Health Promotion programs. This problem has been faced by evaluators, generally academics, together with managers and local agents that are urged to prove to funding agents and society at large that this new approach to health work is efficient.
In contrast to this trend, the authors present various arguments, including paradigmatic, epistemological, and methodological ones, related to the conceptual field of Health Promotion. They call attention to analogies and especially to the divergences with the field of medical care practices that have been increasingly oriented by scientific evidence and identified with the biomedical model. The authors show that the evaluative practices and evidence of effectiveness constructed with this biomedical model are simple, direct, and easy to demonstrate: an individual with type-1 diabetes takes insulin and his blood sugar drops, thus demonstrating, through the result of the blood test, the evidence that the medication is effective for such occurrences.
In Health Promotion, participatory and inter-sectoral practices attempt to deal with the socioeconomic, cultural, political, and environmental determination of the health/disease process by establishing healthy public policies that transcend the health sector and are oriented by another model (socio-historical, cultural, and humanist), involving a complex view of the problems and their causality and potential resolution, thus requiring contributions by various sciences and as well as different intervention strategies. The change occurring in the view and the strategies, which encompass various dimensions of social life, requiring multidisciplinary and inter-sectoral practices, has a decisive impact on the evaluation models needed to construct such evidence. The fact that one cannot establish a direct and immediate relationship between the cause (the intervention) and its effects in the resolution of such complex problems (e.g., the influence of social inequality on infant mortality rates, mediated by other conditions and factors) hinders the gathering of evidence of effectiveness in multi-factor interventions and places in a relative light the value of evaluations performed to prove the importance of Health Promotion actions as an essential component of human development actions in an integrated and sustainable development model.
A lesser issue in this debate, but one that needs to be clarified nevertheless, is a conclusive argument related to the paradigmatic discussion: The issue is thus to view the health sector and the population's health as a fundamental economic investment for human and social development. This statement actually contradicts another, which purports to place Health Promotion within the sphere of the national development model and policies, where the subordination of the economic to the social is pursued. The authors no doubt meant to call attention to Brazil's development model, which basically values only the economic sphere, relegating human and social development to a secondary status. Health Promotion for the population does not occur only through economic investments, but the paradigm transforms the decision-making process concerning where and how to conduct sectoral investments in such a way as to produce greater gains for the population's health. This means defining health and quality of life as a fundamental concern for all sectors of government, that is, health should be included as one of the decision-making criteria within all specific sectors. For example, by viewing health as an investment, before making investment decisions the São Paulo Municipal Secretariat of Education should ask whether installations involving inter-sectoral action, such as one of the so-called Unified Educational Centers (CEUs), has greater potential for producing health and quality of life for the population it serves than ten traditional schools, which cost less than such a Center.
Returning to the previous question and to the conclusions on the issue of evidence, we wish to conclude by commenting that although the arguments demonstrating that efforts to establish evidence in Health Promotion (in relation to the time and effort needed to obtain such evidence) are not proportional to the scientific and practical advances in Health Promotion achieved through the evaluations performed, the studies conducted in this direction have had the merit of allowing and encouraging various key actors involved in the programs (and their financers) to move towards achieving consensuses, based on some criteria defined on the basis of these evaluation results concerning the importance of certain actions and methodological approaches in Health Promotion interventions. The arguments in favor of this conclusion, which are well-grounded in the literature, were an important contribution by this article. The authors have demystified the issue of evidence, while contradictorily and simultaneously demonstrating the importance of evaluation and the search for new methodologies and procedures to deal with the issue of Health Promotion, especially in dealing with the theory and practice of evaluating Health Promotion programs.
The authors' proposal on the theory and practice of evaluating Health Promotion programs, based on "theories of change" and aimed at an in-depth discussion of the relationship between the theoretical constructs and the results, does not correspond to the customary practice of researchers from other countries, still heavily influenced by the rationalist paradigm. Potvin and Richard (2001), in an article discussing Health Promotion evaluation in the community, present the four categories of work found in the literature and the frequency with which they are found: comprehensive evaluative designs, which are the least frequent; articles presenting results of evaluation of mid-term processes or results, which are the most frequent; articles presenting the final results of programs; and finally articles discussing methodological issues related to the three previous types (also quite infrequent). Articles presenting the final results of Health Promotion programs are generally disappointing, because their conclusions are generally that the programs are not effective: either they have failed to meet with the complexities of Health Promotion issues, or their evaluation has failed to grasp this complexity. The authors comment that the evaluative designs adopted are generally experimental or quasi-experimental and fail to grasp the complexity of Health Promotion interventions, and that this probably explains why the results obtained fail to demonstrate the programs' success.
Researchers dedicated to evaluating Health Promotion actions according to the concept defended by this article and who propose to analyze complex interventions utilizing such evaluation approaches as theory-driven evaluation (TDE) and realist analysis, as the article proposes, defend the use of models and therefore logical criteria for evaluation other than traditional scientifically rigorous criteria, thus of the positivist line. There can be two logical criteria based on previously elaborated complex conceptual models, according to Potvin & Richard (2001): either transparency in the decision-making process or critical implementation of the multiple methodological procedures, which tends to be criticized by the traditional methodologists and runs the risk of not have the resulting articles approved for publication.
Despite the risks related to publication and the possible underrating of the resulting research work due to the gap vis-à-vis the hegemonic positivism and rationalism of the capitalist world, Health Promotion evaluation as conceptualized in this article has already become a common practice in some academic institutions in Brazil. The theme of intersectorality linked with quality of life is one of the lines of investigation in Schools of Public Health in several Brazilian universities and related institutions such as the National School of Public Health (ENSP-FIOCRUZ), the home institution of the authors of the current debate article, and the Center for Studies, Research, and Documentation on Healthy Cities (CEPEDOC Healthy Cities), affiliated with the School of Public Health at the University of São Paulo, focused on the follow-up and implementation of experiences with integrated and participatory public management in Brazilian cities, as well as the State University in Campinas (UNICAMP) and others. Various theses and documents have been produced within this line of research, utilizing methodologies similar to that proposed in this article, combining methodologies and associating partners and communities involved in the production of evidence of effectiveness for Health Promotion programs, related much more to the process than to the results in terms of changes in coefficients and indicators.
Investment in Health Promotion programs from a broad and critical perspective and the use of logical evaluation models (Dwyer & Makin, 1997), with the combined use of different methodologies, reflects an alignment with many professionals working in this area in different parts of the world, but it is not a hegemonic position. It means a commitment to a truth, a view of the world and society, but it can involve problems and conflicts with individuals and institutions where ideas associated with behaviorism, positivism, and or classical epidemiology prevail.
References
Dwyer JJM & Makin M 1997. Using a Program Logic Model that focuses on performance measurement to develop a program. Canadian Journal of Public Health 88(6):421-425.
Potvin L, Haddad S & Frolich 2001. Beyond process and outcome evaluation: a comprehensive approach for evaluating health promotion programmes In IG Rootman et al. (org.). Evaluation in Health Promotion: principles and practices.
Potvin L & Richard, L 2001. Evaluating Community Health Promotion Programmes, pp. 213-240. In IG Rootman et al. (org.). Evaluation in Health Promotion: principles and practices.
Publication Dates
-
Publication in this collection
20 Oct 2004 -
Date of issue
Sept 2004
