DEBATE DEBATE
Gastão Wagner de Sousa Campos
Debate sobre o artigo de Zulmira M. A. Hartz
Debate on the paper by Zulmira M. A. Hartz
Evaluation: from soliloquy to dialogue
There was a time when I considered Planning humankind's last and greatest rationalist delirium. I was completely wrong. Policy and Program Evaluation has reemerged at the turn of this century, promising effects very similar to those that Planning methods claimed to achieve twenty or thirty years back. The utopia of making life in society rational! The vain promise of orienting policy according to a rationale above the interests of human groups! The real-socialist countries turned this fantasy into a reality principle and left millions of people in the lurch. I believe that to a certain extent the Institutionalization of Evaluation Systems has come to occupy the void left behind by the relative failure of Planning. Evaluating results would be representing "new approaches to governance that are changing the basic thinking of modern politics", as stated by Jorjani, quoted on page 3 of Hartz' article.
Evaluation is changing the thinking and practice of modern politics! Planners in the 1960s thought the same. But time taught them how wrong they were: political struggle, in both the Old and New Worlds, dared to keep following its old rules: the motor force of desire, interests, and needs worked (oftentimes), rowing upstream against plans; subjects, classes, movements, and ideologies always went beyond the boundaries of what was considered rational; and power, conflict, struggles, and negotiations, followed by the defeats and triumphs of widely diverse protagonists belied the beautifully painstaking technical plans. Life proved to be far more vast than the best and most democratically drafted plan: grandes sertões e veredas! [Translator's note: The discussant has made a play on words by exclaiming, "The vast hinterlands and pathways!" in an allusion to the novel of the same name by Brazilian author Guimarães Rosa, as if to say, "The long and winding road of Planning!"]
Nevertheless, Planning continues to be practiced. Far be it from me to suggest anything other than an Evaluation Policy for the Brazilian Unified Health System (SUS). On the contrary, I laud the pioneering and sagacious effort by Zulmira Hartz. However, although I trust in her academic and ethical qualifications, I am suspicious (in principle) of any Evaluation Method, especially any Institutionalized Evaluation System. One would have to take the rationale resulting from such studies with a grain of salt, even when they are based on more flexible methods and combine statistical logic with that of such others as sociology and anthropology. Just as Mário Testa deconstructed his own line of activity by demonstrating the limits of the Planning he himself helped to invent, Professor Zulmira also blasts the totalitarian pretenses of some of the more prudish schools of Evaluation. She thus points to the inevitable link between evaluation processes and Values that always skew them in defined directions. The author even explicitly assumes the idea that the criterion of reduction of iniquities should underlie any and all social policies and thus provide the reference for any evaluation method. As quoted in the article, it is not a matter of replacing policy-making with statistical logic. Fine! Nevertheless, even considering the broad frame of reference Zulmira employs to analyze evaluation policies, a few comments are still worthwhile on the limits of evaluative methods in general.
Based on international experience, Zulmira suggests that Sectorial Evaluation is preferable to centralized governmental nationwide evaluation. This no doubt has to do with the advantages resulting from decentralization, such as possibilities for adjusting the method to local or regional reality, but it also appears to be related to another characteristic of evaluation. Social processes result from a complex set of determinants, in which there are nearly always objective variables (from the external world), subjective variables (action by the Subject), and social variables (actions between Subjects). In the production of phenomena, such levels emerge mixed, constantly penetrating and influencing each other. Still, from a practical and operational point of view, it is possible to identify processes with a predominance of this or that form of determination. Thus, in more objective programs, with slight influence from subjectivity and the social and historical context, the result of Evaluation might represent judgement that we could consider closer to the truth. Concerning the efficacy of cervical cancer prevention or polio vaccination programs, such measures might reach safe conclusions. More solid certainties would be produced whenever the Method is evaluating aspects in which subjectivity and politics are not heavily involved. Not that Subjective and Social questions cannot be investigated (there are surveys on satisfaction, opinion trends, ethnographic observations, etc.) or that it is more difficult to perform evaluations of Social Policies or Programs. I am referring to another type of difficulty, namely, that these results would always tend to produce very relative truths, truths more dated and situated than those resulting from more objective problems. Thus, a vaccine's efficacy tends not to vary according to the subjective and historical context. The same could never be said of an Agrarian Reform Program. For social programs, one could almost use the old adage from Clinical Medicine: each case is a case apart. One would thus have to take great care when generalizing the results of evaluating social programs. Hundreds of studies have attested to the inefficiency of public services in dozens of places, such as the former USSR, England, and Brazil, but it would still not be proper to conclude that the public sector is structurally and generically incapable of ensuring equity and social justice. Contexts have to be compared, variables have to be cross-analyzed, and one always has to ask, under different circumstances would public services not have greater potential? One has to try new arrangements and not generalize, as has become frequent in contemporary Globalized discourse. Successive negative evaluations of social programs are used politically against social development. Yet the feasibility, acceptance, efficacy, and efficiency of social programs never come ready-made; rather, they are built over the course of their very effort to counteract what had been considered possible until then. Such is the essence of macro- and micro-policy: a wager on building the future, a wager against previous evidence, against warnings that the proposal will never work. Both the right and the left invent their policies and programs arguing against the absolute value that the results of past Evaluations tend to acquire.
In short, yes, let use evaluate, as long as the Evaluation Systems have neither the first nor the last word on policy decisions. Let them act as a backdrop, as a critical conscience, which this or that social actor can employ to argue against and defeat contrary positions. Evaluation may even have the first word, I admit, as a social warning. A warning to be re-drafted by this or that subject group. Technique does not replace policy, and policy should not pertain exclusively to the Administration (Executive Branch) and Congress. Administrations and Legislatures go about their work in what appears to be a suicidal fashion, against statistical evidence. Equity depends on the radical democratization of political life at both the national and internal institutional levels. Transparency of information is just one aspect of this necessary democratization.
Publication Dates
-
Publication in this collection
18 Mar 2003 -
Date of issue
Apr 1999
