Two concepts of evidence-based policyAbstract
The notion that public policy should be evidence-based has gained prominence in recent years, attracting both supporters and critics. This article argues that a significant part of this debate stems from the coexistence of two distinct conceptions of what it means to base policy decisions on evidence: evidence as a restriction, which operates as a necessary condition for the admissibility of options; and evidence as a preference criterion, which guides the selection among options already legitimized by political or normative criteria. The article defines these two concepts, analyzes their respective advantages and limitations, and demonstrates that, under plausible assumptions, each approach can be applied in isolation, but hardly simultaneously without practical contradictions. The key contributions offer a normative typology that allows for diagnosing tensions in the use of evidence, positions this distinction in relation to classical typologies in the literature, and guides the institutional design of scientific advisory systems in democratic contexts.
Keywords:
evidence-based policy; evidence-informed policy; decision theory; public choice theory; policy evaluation
Resumo
A noção de que políticas públicas deveriam ser baseadas em evidências tem crescido em popularidade nos últimos anos, atraindo apoiadores e críticos. Neste artigo, argumento que parte significativa desse debate decorre da coexistência de duas concepções distintas do que significa basear decisões políticas em evidências: a evidência como restrição, que opera como condição necessária para a admissibilidade de opções; e a evidência como critério de preferência, que orienta a escolha entre opções já legitimadas por critérios políticos ou normativos. O artigo define esses dois conceitos, analisa suas respectivas vantagens e limitações e demonstra que, sob premissas plausíveis, cada abordagem pode ser aplicada isoladamente, mas dificilmente de forma simultânea sem contradições práticas. As contribuições-chave oferecem uma tipologia normativa que permite diagnosticar tensões no uso da evidência, posicionar a distinção em relação a tipologias clássicas da literatura e orientar o desenho institucional de sistemas de assessoramento científico em contextos democráticos.
Palavras-chave:
políticas públicas baseadas em evidências; políticas públicas informadas por evidências; teoria da decisão; teoria da escolha pública; avaliação de políticas públicas
Resumen
La noción de que las políticas públicas deben basarse en evidencias ha ganado popularidad en los últimos años, concitando tanto defensores como críticos. Este artículo sostiene que una parte significativa de este debate se debe a la coexistencia de dos concepciones distintas de lo que significa fundamentar las decisiones políticas en evidencias: la evidencia como restricción, que opera como condición necesaria para la admisibilidad de las opciones; y la evidencia como criterio de preferencia, que orienta la elección entre opciones ya legitimadas por criterios políticos o normativos. El artículo define estos dos conceptos, analiza sus respectivas ventajas y limitaciones, y demuestra que, bajo supuestos plausibles, cada enfoque puede aplicarse de manera aislada, pero difícilmente de forma simultánea sin contradicciones prácticas. Las contribuciones clave ofrecen una tipología normativa que permite diagnosticar tensiones en el uso de la evidencia, sitúa esta distinción en relación con las tipologías clásicas de la literatura y orienta el diseño institucional de los sistemas de asesoramiento científico en contextos democráticos.
Palabras clave:
políticas públicas basadas en evidencias; políticas públicas informadas por evidencias; teoría de la decisión; teoría de la elección pública; evaluación de políticas públicas
1. INTRODUCTION
In recent years, Evidence-Based Policy (EBP) has become an influential watchword in governmental and academic circles. Broadly speaking, EBP represents an explicit commitment to applying rigorous scientific methods to the formulation of public policy, in contrast to approaches guided solely by ideology or common sense (Cartwright & Cowen, 2024).
Originating in the fields of medicine and public health, the evidence-based policy movement emphasizes experimental research, especially randomized controlled trials (RCTs), and systematic literature reviews to determine “what works” in public policy (Baron, 2018). Its spread across governments and international organizations responded to demands for greater efficiency, transparency, and accountability in public administration (Khosrowi, 2021).
Although intuitive at first glance, this agenda has drawn criticism from philosophers, methodologists, and social scientists. Critics argue that the narrow emphasis on “evidence of what works” creates practical and conceptual problems: difficulty in extrapolating results beyond the contexts in which they were obtained; neglect of political values and objectives not captured by the metrics used in studies (Khosrowi, 2021); and the risk that evidence may be selectively mobilized to legitimize decisions already made, a phenomenon known as policy-based evidence (Strassheim & Kettunen, 2014). Greenhalgh and Russell (2009) observe that a rigidly positivist conception of evidence-based policy cannot address crucial elements of the public decision-making process, since policies are decided in contextual and complex ways, shaped by values and disputes among interest groups.
In this article, I argue that an important part of the controversy stems from different understandings of what it means to “base” policies on evidence. I identify two distinct conceptions, often implicit in the arguments advanced by each side of the debate. The first treats evidence as a constraint on the set of policy choices: evidentiary considerations serve to limit and filter which policies can be considered admissible. The second treats evidence as a preference criterion for policy choice: evidence guides the selection of the best option within a delimited set of viable alternatives, predefined by political priorities or contextual values.
The contribution of this article is twofold. First, I offer a conceptual typology that clarifies the structural differences between these approaches and their relationship to existing debates in the literature. Second, I show that, under reasonable assumptions, the two approaches tend to produce divergent outcomes, with important implications for the institutional design of evidence governance systems.
By making this distinction explicit, the article seeks to avoid both inflated promises and imprecise criticisms of EBP, fostering a more constructive debate about how and to what extent evidence should influence collective choices in a democratic society. In particular, I show how the two types of EBP discussed here carry implications for the institutional governance of evidence use within public administration.
The article is organized into nine sections, including this introduction. Section 2 reconstructs the context of the EBP agenda, its central promises, and the objections that motivate the distinction proposed here. Section 3 describes the analytical strategy, grounded in the construction of Weberian ideal types. Sections 4 and 5 develop, respectively, the conceptions of evidence as constraint and as preference criterion, characterizing their decision-making mechanisms, institutional examples, and the advantages and risks associated with each. Section 6 presents a synthetic comparative framework of the two conceptions. Section 7 situates this typology in relation to the existing literature (Head, 2016; Parkhurst, 2017), while Section 8 examines the tensions between the two conceptions, showing their tendency toward practical incompatibility and drawing out implications for the institutional design of more robust systems of scientific advice for public decision-making. The article closes with a concluding section that points toward a future research agenda.
2. THE EVIDENCE-BASED POLICY AGENDA: PREMISES AND PROMISES
Before outlining the two main conceptions, it is helpful to review the context and premises of the evidence-based policy agenda. The contemporary Evidence-Based Policy (EBP) movement has its intellectual roots in the Evidence-Based Medicine (EBM) movement of the 1990s, which held that clinical decisions should rely on the best available research evidence rather than solely on physicians’ experience or on untested conventions (Baron, 2018).
By analogy, EBP emerged with the proposal that public policies should be formulated and evaluated on the basis of rigorous scientific evidence about what produces desirable outcomes (for example, reductions in crime, improvements in education, or economic growth). This idea spread across governments and international organizations in response to demands for greater efficiency, transparency, and accountability in public administration (Khosrowi, 2021).
Proponents of EBP frequently emphasize two central promises of this approach (Khosrowi, 2021). The first is the creation of evidence libraries or knowledge bases that compile reliable studies of successful policies, providing a repository of tested, “ready-to-use” options for policymakers.
In areas such as health, education, or public safety, systematic reviews and meta-analyses of experiments can identify, for example, which social programs effectively reduce poverty or improve student learning. Such “what works” repositories help governments select empirically validated interventions rather than rely on unproven solutions. A prominent example is the United Kingdom’s What Works Network (Evaluation Task Force, 2013).
The second promise is that the use of robust evidence will reduce the influence of political interests, ideology, or personal beliefs in decision-making, promoting greater objectivity, transparency, and rationality in the policy process. From this perspective, reliable scientific evidence serves as a counterweight to arbitrariness: decisions made by “following the science” are more impartial and accountable because they are anchored in documented, verifiable evidence rather than in opinion. In short, EBP aspires to institutionalize a form of technical rationality in governance, in which the question “which policy should be adopted?” is answered, at least in part, by the best available evidence on the results of similar policies.
Critics, however, have noted that these promises often do not materialize so straightforwardly. The first set of challenges is methodological: determining which evidence counts as valid and relevant for a given policy decision. “Strict” EBP tends to privilege certain types of studies, most notably RCTs and systematic reviews, as the gold standard of evidence.
Yet several authors argue that even RCTs, despite their credibility, have important limitations: they rely on assumptions that are not always satisfied, and their results do not always transport to different contexts. Deaton and Cartwright (2018), for example, show that the effects measured by an RCT depend on specific conditions; contextual factors and local causal interactions may cause an intervention that “worked” in a pilot to fail when implemented in another region or at another scale.
This is the classic problem of external validity, or extrapolation: evidence obtained in one place and at one time does not always “travel” well to other settings. The hope of building universal “what works” libraries runs into the fact that what worked in one context may not work in another. Successful policies cannot simply be transplanted without adaptation.
The second set of challenges is evaluative and political. Public policies do not aim solely at technical effectiveness; they also embody values and normative choices. Parkhurst (2017) argues that good governance of evidence use requires recognizing that different types of evidence may be appropriate depending on the normative objectives at stake, giving weight, for example, to equity, political feasibility, or public acceptability, and not only to average effectiveness as measured in studies. One well-documented effective solution for mitigating urban congestion, for instance, is congestion pricing; but it is equally well documented that proposing such a policy reduces the popularity of the politicians who advocate it, at least until the charges take effect and the public adapts to the new traffic conditions (Selmoune et al., 2020; Xun, 2021).
Proponents of EBP sometimes suggest that following the evidence will lead to the “best” policy, but this underestimates the fact that what counts as best depends on value judgments about which objectives matter and which trade-offs are acceptable. As Greenhalgh and Russell (2009) note, a purely empiricist framework tends to obscure the fact that policymaking is inherently political and “value-laden,” involving negotiation among diverse interests and competing interpretations of the public good. Cairney and Oliver (2017) make a similar argument but identify the root of the problem more precisely: a lack of information for decision-making creates uncertainty, but disputes over how to frame the problem create ambiguities that cannot be resolved simply by gathering more information.
Recognizing these challenges does not mean discarding the idea of evidence-based policymaking, but rather refining what it means to be “based on evidence.” Some authors suggest replacing rigid universal hierarchies of evidence (in which only RCTs and experiments count) with more pluralistic, context-sensitive approaches. Head (2016), for instance, proposes the term evidence-informed policy in place of policy that is “strictly based” on evidence, emphasizing that scientific evidence should inform, but not mechanically dictate, decisions.
Similarly, Parkhurst and Abeysinghe (2016) argue that the adequacy of evidence should be assessed in relation to the decision-making context, rather than through blind adherence to quality hierarchies that abstract away from specific circumstances. Such perspectives recognize that multiple types of evidence are relevant (quantitative, qualitative, experimental, pragmatic, and so on), and that the “appropriate” use of science in policy requires prudent judgment about which evidence matters for a given problem and how it should be weighed against other criteria.
It is against this background of differing understandings of the role of evidence in policy that the central distinction of this article emerges: evidence as constraint versus evidence as preference criterion. The following sections examine each conception in detail, illustrating their respective logics and differences.
3. ANALYTICAL STRATEGY AND METHODOLOGICAL POSITION
This article adopts an approach of conceptual analysis and rational reconstruction, commonly used in analytical political theory and applied philosophy of science (Khosrowi, 2021). The method consists of drawing out the logics implicit in governance practices and in the literature on Evidence-Based Policy (EBP) and distilling them into “ideal types” in the Weberian sense (Weber, 1949): theoretical constructs that highlight and systematize characteristic features of social phenomena for comparative analysis.
The two ideal types proposed here, evidence as constraint and evidence as preference criterion, do not describe pure empirical cases. They provide reference points against which real cases can be contrasted. The criteria for their construction are: (i) conceptual clarity, requiring each type to be internally coherent and distinguishable from the other; (ii) explanatory scope, requiring the types to capture the underlying logics of the principal positions in the EBP debate; and (iii) normative utility, requiring the types to generate practical implications for institutional design.
The validity of this work derives not from the collection of statistical data but from the internal coherence and explanatory power of the proposed concepts in illuminating observable tensions in debates on EBP. The analysis synthesizes contributions from Nutley et al. (2007) on typologies of evidence use, Head (2016) on the distinction between evidence-based and evidence-informed policy, and Parkhurst (2017) on the governance of evidence, reorganizing these perspectives into a new analytical framework focused on the normative function of evidence in the architecture of public choice. The emphasis is not on the type or quantity of evidence, but on when and how it operates within the decision-making process.
4. EVIDENCE AS CONSTRAINT IN POLICY CHOICE
In the conception of EBP as constraint, “basing policy on evidence” means restricting the range of possible policy choices by treating empirical knowledge as a mandatory filter. Evidence functions as an eliminative criterion, an admissibility filter: policies lacking evidence of effectiveness are ruled out a priori from the set of options under consideration. Evidence thus operates as a necessary condition: if there is no evidence that a policy works, it should not even be considered.
In concrete terms, applying evidence as constraint requires assessing the “level of evidence” available for each intervention before deliberating on its adoption. Suppose a government has $1 billion available in its budget. Under the constraint criterion, any policy lacking systematic studies of effectiveness is excluded at the outset. Those with an empirical basis may then be ranked according to political preferences, values, costs, or other criteria. The key point is that the first stage of deliberation becomes a screening procedure: only policies that clear a minimum evidentiary threshold remain on the “menu.”
This conception aligns with what Khosrowi (2021) calls the “strict” version of EBP. It also echoes weaker forms of the precautionary principle (Powell, 2010): better not to act than to act without evidence and risk wasting public resources or harming individuals. It approximates, too, a notion of “doing policy as one does science”: just as a researcher should not assert a hypothesis without evidence, a policymaker should not implement a policy without evidence that it works.
4.1 Institutional Examples
The most successful institutional example of this approach is found in health regulatory agencies such as the Food and Drug Administration (FDA) in the United States and the Brazilian Health Regulatory Agency (ANVISA) in Brazil. These agencies do not dictate which medicines should be developed; they act as filters that deny authorization to technologies that fail to demonstrate robust evidence of efficacy and safety. Their existence is rarely seen as a threat to democracy; on the contrary, countries lacking such regulatory oversight are hardly regarded as more liberal in matters of health. Transparent technical criteria replace arbitrary or convenience-based criteria, not democratic ones.
Civil aviation regulators operate under a similarly explicit commitment to facts, evidence, and causal mechanisms of safety. The idea of changing aviation technologies or protocols without evidence that such changes are safe would sound irresponsible, not anti-democratic. In both cases, the success of the constraint model rests on a specific condition: broad normative consensus about the ends pursued (public health, air transport safety), combined with the catastrophic cost of error. In such contexts, society accepts that evidence should constrain the menu of political options.
In the Brazilian context, the National Committee for Health Technology Incorporation (Conitec) within the Unified Health System (SUS) offers another relevant example. Conitec is charged with assessing evidence of efficacy, safety, and cost-effectiveness for medicines and procedures before their incorporation into the public health system. It operates according to the logic of constraint: technologies that fail to meet the required evidentiary threshold are excluded - or at least should be excluded - from the list of procedures financed by SUS, regardless of political or commercial pressures. The National Civil Aviation Agency (ANAC) performs an analogous restrictive function, conditioning aircraft certification and the approval of changes in operational protocols on demonstrated compliance with evidence-based safety standards. These examples show that the constraint model already operates in well-established Brazilian institutions and enjoys broad social acceptance precisely because it functions in domains marked by strong normative consensus and the potential for catastrophic harm.
4.2 Limits and Risks
This perspective faces significant tensions. First, policies are not repeatable experiments: they involve open contexts, values, and multidimensional consequences (Cartwright, 2009). The rigid application of constraint may be blind to practical or circumstantial forms of knowledge that, though not derived from RCTs, are highly relevant, disregarding them simply because they do not meet the standard of “best evidence” (Nutley et al., 2013).
Second, there is a risk of decision-making paralysis: taken literally, no action would be taken without first clearing a strict empirical test, even when the social problem is urgent. The moral cost of this can be high (Khosrowi & Reiss, 2019). Third, there is the risk of opaque technocracy: when evidence becomes a mandatory filter, fundamentally evaluative choices about what counts as good evidence are concealed beneath a technical-scientific veneer. Who sets the admissibility threshold? On what criteria? This issue requires sound institutional governance; it cannot be resolved merely by appealing to “science.”
Finally, Strassheim and Kettunen (2014) warn of situations in which EBP is instrumentalized to legitimize decisions already made. When evidence is presented as an absolute constraint, political disputes can be disguised as purely technical matters.
5. EVIDENCE AS PREFERENCE IN POLICY CHOICE
The conception of evidence as preference criterion understands “being evidence-based” not as an exclusionary filter, but as a criterion to be weighed in the decision-making process. The decision-maker already has a policy area or a set of alternatives delimited by political considerations and values. The function of evidence is to guide the selection of the best alternative within that scope. It operates as a compass: it points in a promising direction but does not determine where the decision-maker should go; the destination is set by the agent’s purpose.
Suppose, for example, that a government has decided to allocate additional resources to basic education, driven by political priorities or social demands. There are many ways to distribute this investment: building new schools, training teachers, implementing tutoring programs, installing air conditioning in hot regions, and so on. To operate under the preference model is to draw on evidence to assess which of these options is most effective in producing educational gains.
If studies indicate that equipping schools in hot regions with air conditioning improves performance more than new construction or other alternatives, the decision-maker chooses that course of action not because the others are invalid, but because the data provide the strongest evidence of relative effectiveness within the previously defined scope.
Within this framework, no option is excluded a priori for lack of evidence: when the decision was made to invest in education, options in health or public safety were already excluded on other grounds. Evidence enters only after the options have been defined politically. The decision-maker also weighs other considerations (costs, equity, urgency, alignment with policy guidelines), and evidence functions as one informational input among several.
5.1 Advantages over evidence as constraint
This model has the advantage of being more consistent with actual policy practice. Research on how governments use knowledge shows that scientific evidence is only one influence among many (Head, 2010). Moreover, this conception reduces the risk of paralysis in the absence of perfect evidence and allows action under uncertainty, using the best available information as guidance while explicitly acknowledging its limits.
5.2 Risks and Limits
One important criticism is that this approach can dilute the role of evidence. Because political intention determines the initial menu of options, a sufficiently restrictive intention can justify almost any decision on weak evidence. A public agent might define the objective not as “investing in health,” a broad menu within which evidence would play a meaningful role, but as “investing in the renovation of high-complexity units in rural areas,” a menu so narrow that the evidentiary criterion becomes irrelevant.
For a more extreme case: if the goal is defined, on whatever political grounds, as promoting practices such as “ozone therapy” or “family constellations,” specialists in those practices can debate which procedures have more or less evidence, and evidence as preference formally operates as a criterion. Yet in the broader scientific debate, none of these practices has produced even minimal evidence of effectiveness. The counterintuitive implication - that such practices could be presented as “evidence-based” - is a built-in risk of the preference model. Under the constraint model, they would be excluded a priori.
Authors such as Parkhurst (2017) stress the need to institutionalize deliberative processes in which evidence is presented and scrutinized transparently. Jasanoff (2003) proposes “technologies of humility”: institutional arrangements that bring evidence and values into open dialogue, acknowledging uncertainties rather than concealing political choices behind technocratic facades.
6. COMPARATIVE TABLE: SYNTHESIS OF THE TWO CONCEPTIONS
Table 1 presents a comparative synthesis of the two conceptions of EBP, summarizing their defining features, mechanisms of operation, advantages, associated risks, and the contexts in which each is most appropriately applied.
7. RELATIONS WITH EXISTING TYPOLOGIES IN THE LITERATURE
The typology proposed here does not emerge in a vacuum; it engages with and reorganizes classic contributions in the field. This section makes explicit how the concepts of constraint and preference relate to influential distinctions in the literature, showing what this new framing adds, refines, or clarifies.
7.1 Nutley, Walter, and Davies: instrumental, conceptual, and symbolic
Nutley et al. (2007), building on a research tradition initiated by Caplan (1979) and Weiss (1979), distinguish three main types of evidence use: instrumental use (research directly guides a specific decision); conceptual or enlightenment use (evidence diffusely shapes decision-makers’ cognitive frameworks and problem definitions without leading to immediate action); and symbolic or tactical use (evidence is mobilized to legitimize decisions already made, serving as ammunition in political disputes). The book also identifies a fourth type, process use, in which the mere fact of commissioning or conducting research is used to postpone or evade decisions, regardless of its substantive content. This type is not central to the argument developed here and is therefore omitted from the synthesis.
That typology operates primarily as a descriptive device. The authors themselves note the fluid and overlapping character of the categories, which serve primarily to classify what happens to evidence during political decision-making. The constraint/preference distinction, by contrast, is normative and procedural: it classifies how the decision rule ought to be structured in order to prevent symbolic use and promote instrumental use.
Both the constraint model and the preference model are forms of instrumental use; both employ evidence to guide action. But they represent opposing instrumental mechanisms: the constraint model is a “hard,” negative form of instrumental use (veto), while the preference model is a “soft,” positive form (adjustment). This level of granularity, absent from Nutley et al. (2007), helps explain why certain attempts at instrumental use fail: when advocates of evidence try to impose a constraint model in political environments that accept only the preference model, the result is rejection and conflict. The new typology offers precision about the mechanics of instrumental use that the authors’ original scheme does not capture.
7.2 Head: evidence-based vs. evidence-informed
Brian Head (2016) proposed the distinction between evidence-based and evidence-informed as a way of recognizing that evidence can rarely “ground” decisions deterministically. “Informed” implies that scientific evidence is only one of the “three lenses” of policy formulation, alongside professional/practical knowledge and political/value-based knowledge (Head, 2010).
The typology developed here formalizes this intuition in terms of decision rules. Evidence-based policy operates under the constraint logic (evidence is the basis; without it, there is no policy), whereas evidence-informed policy (EIP) operates under the preference logic (evidence informs choice within a field of possibilities shaped by values). The concept of “evidence as preference” shows that EIP is not merely a matter of “listening to science” but of integrating it as an optimizing vector within a complex utility function. This shields EIP from the charge of being “anti-scientific” or merely rhetorical, provided it is implemented with procedural rigor.
The typology also exposes a weakness in Head’s concept: “informed” remains vague about the weight of evidence in a decision. A decision-maker may claim to have been “informed” by evidence when, in practice, the evidence was ignored. The constraint/preference distinction reveals that EIP requires institutional mechanisms to prevent the preference model from degenerating into mere symbolic use, a risk Head acknowledges but does not formalize.
7.3 Parkhurst: governance of evidence and the diagnosis of biases
Justin Parkhurst (2017) raised the debate to the institutional level through his concept of the “good governance of evidence.” He identified two structural biases that corrupt the use of science: technical bias (the use of evidence that does not meet the best scientific standards, as in cherry-picking or statistical misinterpretation) and issue bias (the distortion of the policy agenda through insistence on certain types of evidence, such as RCTs, which privilege measurable problems amenable to deliberate experimentation at the expense of issues that cannot feasibly be tested in that way).
The typology developed here functions as a diagnostic tool for institutional risk that complements Parkhurst’s analysis. The model of evidence as constraint is structurally prone to issue bias. By raising the admissibility threshold (for example, by requiring RCTs), it systematically excludes complex or hard-to-measure social issues from the policy agenda, depoliticizing priorities under the cloak of technical rigor.
The model of evidence as preference criterion is structurally vulnerable to technical bias. By allowing policy actors to define the scope of the menu of options freely, it leaves room for policymakers to use weak or biased evidence to justify predetermined choices. At the limit, it can degenerate into what Strassheim and Kettunen (2014) call policy-based evidence.
This connection has practical implications for accountability and the evaluation of public decisions. If an agency adopts the constraint model, auditors should monitor issue bias (what are we failing to do?); if it adopts the preference model, auditors should monitor technical bias (are we using evidence correctly?). This responds to calls for more sophisticated accountability mechanisms in the use of evidence in public administration.
8. TENSIONS BETWEEN THE TWO CONCEPTIONS AND THE IMPOSSIBILITY OF FULL COMPATIBILITY
Having characterized the two conceptions, one question arises: can they be combined within a single decision-making process? I argue that, under reasonable premises, the two approaches tend to come into conflict, leading to divergent practical choices.
8.1 The logic of incompatibility between the two concepts of Evidence-Based Policy
The two maxims can be stated in simplified form:
-
Evidence-based policy as constraint: “Do not adopt a policy unless there is high-quality evidence that it can work in the relevant context.”
-
Evidence-based policy as preference: “Given the chosen objective, prioritize, among the available options consistent with that objective, those for which there is better evidence of effectiveness in achieving it.”
The constraint maxim precedes political choice and may exclude the very family of policies the preference maxim would take as its starting point. The preference-model maxim preserves the chosen family and uses evidence to rank its options, even when alternatives outside that scope are supported by stronger evidence.
When the policy with the strongest empirical support does not coincide with the one preferred for external reasons (values, strategy, democratic commitments), following both maxims strictly leads to practical contradiction: either the political plan is abandoned in deference to the evidence (constraint), or the plan is kept and evidence is used only to calibrate it (preference).
8.2 Three typical scenarios of incompatibility
Scenario 1: conflict of scope. A government decides to invest in education to improve the population’s average income. Under the constraint criterion, options lacking robust evidence are eliminated. It may turn out that no strong evidence links educational investments to short-term income gains, whereas solid evidence shows that income transfers have an immediate impact. The constraint criterion would lead the decision-maker to reconsider the original plan and prioritize transfers. Under the preference criterion, by contrast, the government would retain its educational objective for political or evaluative reasons, using evidence only to select the best type of educational intervention, even if stronger evidence exists for policies outside that domain.
Scenario 2: trade-off between evidence and feasibility. Urban tolls (congestion pricing) are supported by robust evidence of their effectiveness in reducing congestion and emissions, as documented in cities such as London, Stockholm, and Singapore (Selmoune et al., 2020). Under the logic of constraint, the policy passes the evidentiary filter and is admitted. There is also, however, robust evidence of its short-term political unpopularity: mayors who propose it frequently face rejection (Xun, 2021). Under the preference logic, the policy would be set aside in favor of alternatives with weaker evidence of technical effectiveness but greater political feasibility. The two conceptions thus lead to opposite conclusions.
Scenario 3: evidentiary vacuum under urgency. In March 2020, governments faced urgent decisions about lockdowns, school closures, and mask use in the absence of randomized controlled trials (RCTs) or high-quality evidence specific to SARS-CoV-2. A strict constraint model would have implied inaction while evidence was being generated - clearly unacceptable. In practice, governments acted under the preference model, combining partial evidence from other contexts, epidemiological modeling, the precautionary principle, and the early experiences of other countries. This historical case shows that, in domains where decisions cannot wait, evidence as constraint can become unworkable, and the preference model becomes the necessary operational mode.
The Brazilian experience during the COVID-19 pandemic is particularly illustrative of this tension. On the one hand, ANVISA maintained its constraint logic in vaccine approval, requiring evidence of efficacy and safety before granting registration or emergency use authorization, and rejecting applications that failed to meet the established technical criteria. On the other hand, decisions on lockdowns, school closures, and treatment protocols were made by governors and mayors under the preference logic, combining partial evidence, political pressures, local health-system capacity, and public demands.
The two models coexisted within the same crisis, each operating in a distinct domain of the same problem. This shows how the two conceptions can coexist in institutional practice without merging into a unified approach. At the same time, the preference model in such contexts requires that considerable weight be given to the best available evidence, since catastrophic risk is at stake if decisions go wrong or if actions supported by weak evidence are permitted.
These three scenarios show that the incompatibility is not merely theoretical. It recurs in real-world decision-making, where policymakers face genuine tensions between strictly following the available evidence and preserving evaluative commitments or responding to urgent needs.
8.3 Conditions of compatibility
The two approaches converge only when two conditions are met: (a) the political scope already coincides with the set of alternatives that clear the minimum evidentiary threshold; and (b) the other criteria the agent values do not reverse the ordering the evidence suggests. In most cases, these two conditions do not hold simultaneously.
In practice, any “combination” of the two approaches takes the form of a lexical priority: either the constraint is applied first and evidence then guides a choice among the remaining options, or the political scope is fixed first and a minimum epistemic threshold is imposed afterward. In both arrangements, one conception is a first-order criterion and the other a second-order criterion. Full convergence requires a rare alignment of conditions.
8.4 Implications for Institutional Design
This distinction has direct implications for the design of scientific advisory systems. Procedurally, it makes an ex ante disclosure criterion attractive in the design of institutional architectures for ‘evidence-based’ decision-making.
Institutions that use evidence as constraint should be designed as safety and regulatory agencies, with a high degree of technical autonomy, narrow mandates, transparent procedures regarding the evidentiary standard required, and mechanisms for revision as new evidence emerges. They are appropriate for areas in which consensus about ends is high and the cost of error is catastrophic, such as public health, product safety, monetary stability, and aviation safety.
Under these conditions, it is reasonable for society to accept a stronger use of evidence as constraint in order to reduce the probability that catastrophic risks will materialize. Democratic debate over which measures to adopt then takes place within a pre-filtered menu, ensuring that none of the available options lead to an outcome a majority of society would regard as disastrous - an uncontrolled epidemic, widespread contamination, large-scale accidents, hyperinflation, and the like.
Institutions that use evidence as preference criterion should be designed as intelligence and advisory centers, with proximity to the executive branch, capacity for translation and brokerage, permeability to social values, and a consultative function without veto power. They should explicitly declare the political scope and the normative objectives that justify it, along with exception mechanisms that authorize revision of that scope when significant evidence indicates the objective is not being achieved. They are appropriate for areas marked by high normative contestation, such as education, social assistance, public security, and urban planning.
One institutional mechanism that clearly illustrates the preference logic is the public hearing, an instrument increasingly present in Brazilian public administration, particularly within the framework of Regulatory Impact Analysis (Análise de Impacto Regulatório, AIR), institutionalized by Law No. 13.874/2019 and Decree No. 10.411/2020. Public hearings invite interested parties to present their evidence and arguments, functioning as a hybrid space in which values and evidence are openly confronted. This format is compatible with the conception of evidence as preference: the evidence brought by participants informs and guides the decision but competes with other criteria, and the public decision-maker is not obliged to follow the option with the strongest empirical support.
For the good governance of evidence, however, it is important to recognize that public hearings are rarely capable of performing the function of evidence as constraint. In domains where the underlying problem involves broad normative consensus and potentially catastrophic risk, open discussion among interested parties may be insufficient to filter options competently against rigorous evidentiary criteria.
The risk is that the underlying technical issue - for example, product safety or the effectiveness of a public health intervention - is diluted amid a dispute over values and partial evidence, often advanced in self-interested ways, which the public decision-maker cannot evaluate competently without a specialized technical body. The governance of evidence then requires not only deliberative spaces but also institutions with the technical capacity and mandate to apply restrictive criteria; otherwise, the evidentiary filter becomes a rhetorical facade.
Attempting to turn an education council into an “FDA of Pedagogy” (the constraint model) would be difficult and potentially counterproductive. Educational evaluations do not exhibit the kind of normative consensus, causal robustness, or clearly catastrophic risk that characterize pharmacology and create social pressure to avoid negative outcomes at all costs. Conversely, operating health agencies as mere advisory centers (the preference model) would compromise public safety in situations capable of causing grave and irreversible harm to millions of people.
As a consequence of this differentiation between the two concepts of evidence use, there is no single model of an “Evidence Office” applicable across all domains. Transparency about the relevant concept of evidence-based policy and its auxiliary criteria helps reduce two risks: opaque technocracy (when epistemic constraint silently replaces deliberation about ends) and mere evidentiary rhetoric (when the language of science is mobilized only as a veneer to legitimize ends fixed in advance).
9. CONCLUSION
In this article, I have distinguished two competing conceptions of evidence-based public policy - evidence as constraint and evidence as preference criterion - and shown how each assigns a distinct role to scientific knowledge in the decision-making process.
Under the conception of evidence as constraint, a technical orientation prevails: only empirically validated policies should be considered, with the aim of minimizing arbitrariness and risks of ineffectiveness.
Under the conception of evidence as preference criterion, by contrast, a more flexible orientation prevails: evidence informs choices within politically established parameters and is used to reconcile knowledge with values. Although both approaches have merits, deep tensions between them preclude their simultaneous adoption, and this incompatibility is not merely theoretical but appears in real decision-making situations.
Clarifying this distinction benefits both academic debate and governmental practice. First, it helps dispel misunderstandings. Recurring disagreements about evidence-based public policy often stem from different expectations about what it means to be “based on” evidence. Advocates of evidence-based policy may have in mind the idea of improving decisions through scientific information (the preference model), whereas critics attack the vision of rigid technical determinism (the constraint model). Recognizing that both sides focus on distinct aspects, and on problems that are partly inescapable, can open the way to a more constructive dialogue.
Second, it guards against inflated promises and unfair disillusionment. A reformer who promotes the idea that “science will solve our policy problems” may have in mind either of the two criteria, or neither. When contexts arise, as they will, in which evidence-supported solutions are hard to implement or fall short of expected standards of rigor, opponents may declare the entire evidence-based policy agenda a failure. The real failure is often not having recognized the limits built into the stricter approach.
Third, it offers a tool for institutional diagnosis: the association between constraint and Parkhurst’s issue bias and between preference and technical bias, enables auditors, researchers, and public managers to identify more precisely which pathology is compromising the use of evidence in a given context.
Finally, the analysis highlights the importance of reflexive evidence governance. The challenge is not simply to “use more evidence” or to “follow the science,” but to integrate scientific knowledge into a decision-making process that can be evaluated and re-evaluated transparently, by both the broader public and specialists. In some contexts, it is desirable to adopt evidence almost as a constraint (clinical protocols, pharmaceutical regulation). In others, the preference model is favored (innovative social policies). What matters is that the choice be made consciously, explicitly, and communicated appropriately to the parties involved.
9.1 Limitations and research agenda
Like any conceptual analysis, this study has limitations. The two ideal types are deliberate simplifications: concrete practices of evidence use often combine elements of both conceptions in hybrid, context-dependent ways. The typology is intended to clarify underlying tendencies and logics, not to describe exhaustively every possible arrangement.
The article also does not offer a systematic empirical analysis of which arrangements work best in which contexts; that is a task for future research. Comparative studies of scientific advisory institutions in different countries could empirically examine which ones predominantly adopt the constraint model versus the preference model, and with what results. In the Brazilian context, analyses of the practices of ANVISA, the Federal Court of Accounts (Tribunal de Contas da União), and public policy councils could test and refine the concepts proposed here.
Future research could also examine the role of organizations such as the Institute for Applied Economic Research (Ipea) in producing and mediating evidence for federal policies, to see how far its work approximates the preference model (advisory support and evidence translation) or incorporates elements of constraint (evidentiary baselines for government programs). The experiences of state education secretariats with performance-evaluation programs and evidence-based resource allocation, such as those in Ceará and Pernambuco, offer further empirical ground for testing the typology in subnational contexts marked by intense value disputes and limited resources.
In sum, evidence-based public policy is not a monolithic idea but a label covering at least two different conceptions of the role of scientific knowledge in public choice. The debate, and the policies that follow from it, improve when we recognize what is actually at issue: not whether evidence should be used, but how and to what extent it should shape complex collective choices in a democratic society.
ACKNOWLEDGMENTS
I thank the editorial board of Brazilian Journal of Public Administration and its reviewers for the constructive criticism and comments that enabled substantial improvements to this article. Any remaining errors and omissions are entirely the author’s responsibility.
References
-
Baron, J. (2018). A brief history of evidence-based policy. The ANNALS of the American Academy of Political and Social Science, 678(1), 40-50. https://doi.org/10.1177/0002716218763128
» https://doi.org/10.1177/0002716218763128 - Cairney, P., & Oliver, K. (2017). Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy? Health Research Policy and Systems, 15(35), 1-11.
- Caplan, N. (1979). The two-communities theory and knowledge utilization. American Behavioral Scientist, 22(3), 459-470.
- Cartwright, N. (2009). Evidence-based policy: what’s to be done about relevance? Philosophical Studies, 143, 127-136.
- Cartwright, N., & Cowen, N. (2024). Disagreement about evidence-based policy. In M. Baghramian, J. A. Carter & R. Cosker-Rowland (Eds.), The Routledge Handbook of Philosophy of Disagreement (pp. 486-496). Routledge.
- Deaton, A., & Cartwright, N. (2018). Reflections on randomized control trials. Social Science & Medicine, 210, 86-90.
-
Evaluation Task Force. (2013, June 28). What Works Network GOV.UK. https://www.gov.uk/guidance/what-works-network
» https://www.gov.uk/guidance/what-works-network - Greenhalgh, T., & Russell, J. (2009). Evidence-based policymaking: a critique. Perspectives in Biology and Medicine, 52(2), 304-318.
- Head, B. W. (2010). Reconsidering evidence-based policy: key issues and challenges. Policy and Society, 29(2), 77-94.
- Head, B. W. (2016). Toward more “evidence-informed” policy making? Public Administration Review, 76(3), 472-484.
- Jasanoff, S. (2003). Technologies of humility: citizen participation in governing science. Minerva, 41(3), 223-244.
- Khosrowi, D. (2021). Evidence-based policy. In C. Mäki (Ed.), The Routledge handbook of the philosophy of economics (pp. 370-384). Routledge.
- Khosrowi, D., & Reiss, J. (2019). Evidence-based policy: the tension between the epistemic and the normative. Critical Review, 31(2), 179-197.
- Nutley, S. M., Walter, I., & Davies, H. T. O. (2007). Using evidence: How research can inform public services Bristol University Press.
- Nutley, S. M., Powell, A. E., & Davies, H. T. O. (2013). What counts as good evidence? Provocation paper for the alliance for useful evidence Alliance for Useful Evidence.
- Parkhurst, J. O. (2017). The politics of evidence: from evidence-based policy to the good governance of evidence Routledge.
- Parkhurst, J. O., & Abeysinghe, S. (2016). What constitutes “good” evidence for public health and social policy-making? From hierarchies to appropriateness. Social Epistemology, 30(5/6), 665-679.
- Powell, R. (2010). What’s the harm? An evolutionary theoretical critique of the precautionary principle. Kennedy Institute of Ethics Journal, 20(2), 181-206.
- Selmoune, A., Cheng, Q., Wang, L., & Liu, Z. (2020). Influencing factors in congestion pricing acceptability: a literature review. Journal of Advanced Transportation, (1), 4242964.
- Strassheim, H., & Kettunen, P. (2014). When does evidence-based policy turn into policy-based evidence? Configurations, contexts and mechanisms. Evidence & Policy, 10(2), 259-277.
- Weber, M. (1949). The methodology of the social sciences (E. A. Shils & H. A. Finch, Trans.). Free Press.
- Weiss, C. H. (1979). The many meanings of research utilization. Public Administration Review, 39(5), 426-431.
- Xun, F. (2021). Urban traffic congestion pricing: Literature review and real-world applications Master’s thesis. University of Washington, Seattle, United States of America.
-
RESEARCH DATA AVAILABILITY
This article is theoretical and conceptual in nature. The sources used are cited throughout the text, and most are publicly accessible.
-
ARTIFICIAL INTELLIGENCE USAGE
The artificial intelligence tools ChatGPT and Claude were employed to provide critiques of intermediate versions of this article.
-
Reviewers:
Sónia Paula da Silva Nogueira, Instituto Politécnico de Bragança, Bragança, Portugal https://orcid.org/0000-0002-8675-6102
-
Reviewers:
Ricardo Ceneviva, Universidade Federal do ABC, São Bernardo do Campo, SP, Brazil https://orcid.org/0000-0001-5464-1119
-
Reviewers:
One reviewer did not authorize the disclosure of their identity.
-
Peer review report:
The peer review report is available at https://periodicos.fgv.br/rap/article/view/97069/90457
Edited by
This article is theoretical and conceptual in nature. The sources used are cited throughout the text, and most are publicly accessible.
