Cross-sector learning among researchers and policy-makers: the search for new strategies to enable use of research results

El aprendizaje intersectorial entre investigadores y creadores de políticas: la búsqueda de nuevas estrategias que permitan el uso de los resultados de investigación

ARTIGO ARTICLE

Cross-sector learning among researchers and policy-makers: the search for new strategies to enable use of research results

El aprendizaje intersectorial entre investigadores y creadores de políticas: la búsqueda de nuevas estrategias que permitan el uso de los resultados de investigación

Patricia Pittman^I; Celia Almeida^II

^IAcademyHealth, Washington DC, U.S.A.

^IIEscola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz, Rio de Janeiro, Brasil

^{Correspondence}

ABSTRACT

This paper assesses the preliminary results of a research funding strategy that alters the structure and process of research by requiring interaction between researchers and policy-makers. The five research teams focused on different aspects of expanding social protection in health in Latin America and the Caribbean. Preliminary results revealed negotiation of the research questions at the start of the process, influencing not only the project design, but the decision-makers' ways of thinking about the problem as well. As the projects advanced, turnover among government officials on four of the teams impaired the process. However, for the one team that escaped re-composition, the interaction has led to use of data in decision-making, as well as a clear recognition by both parties that different kinds of evidence were at play. The process highlighted the importance of stimulating systems of learning in which multiple kinds of knowledge interact. This interaction may be a more realistic expectation of such initiatives than the original goal of "transferring" research knowledge to policy and practice.

Research Financing; Research Personnel; Strategies

RESUMEN

Este trabajo establece los resultados preliminares de una investigación sobre estrategia de financiamiento que altera la estructura y proceso de la misma, requiriendo la interacción entre los investigadores y los creadores de políticas. Los cinco equipos de investigación se centraron en diversos aspectos para la ampliación de la protección social en salud en Latinoamérica y Caribe. Los resultados preliminares revelaron la negociación de las preguntas de la investigación al comienzo del proceso, influenciando no solamente en el diseño del proyecto, sino también en las maneras de pensar de los tomadores de decisión acerca de la problemática planteada. A medida que los proyectos avanzaron, los cambios de responsables del gobierno en cuatro de los equipos deterioró el proceso. A pesar de ello, la interacción ha llevado – en el único equipo que logró escapar de la recomposición – al uso de datos en la toma de decisión, así como a un reconocimiento claro por ambas partes que diversos tipos de evidencia estaban en juego. El proceso destacó la importancia de estimular sistemas de aprendizaje en los que múltiples tipos de conocimiento interactúan. Esta interacción puede ser una expectativa más realista de este género de iniciativas que la meta original de la "transferencia" conocimiento desde la investigación a la política y su práctica.

Financiación de la Investigación; Investigadores; Estrategias

Background

Despite the conceptualization of health services research as an applied area of study, the challenge of making research relevant to the policy process continues to preoccupy the field and its funders ^1,2. It is the hypothesis of this paper that this may be, at least in part, a function of the research funding mechanism that is most often employed. Grants typically begin with a researcher-defined proposal, even when the general topic is predefined in a Request for Proposals, and they tend to end with the submission of a paper to a peer-reviewed journal. Most requests for proposals include no funding that would obligate, or even allow, researchers to interact with those whom they wish to influence with their findings.

In this paper we analyze preliminary results of a new research program strategy developed by the International Development Research Centre (IDRC) and the Pan-American Health Organization (PAHO) with the support of AcademyHealth that experiments with the structure and process of research. The initiative's goal is to spur innovation in the expansion of social protection in health in Latin America and the Caribbean. Its strategy was to require partnerships among researchers and decision-makers from the start of the research design. Five projects were funded from a pool of over sixty applicants. The assessments of these partnerships as they evolved during the first and second phases of the projects are included in this supplement ^3,4,5,6,7.

The underlying assumption of the partnership strategy is twofold. First, research results are more likely to be used if there is interaction with decision-makers throughout the design and implementation of a study. Second, the process of negotiation of inquiry itself creates and enriches knowledge for both sides.

The first assumption reflects an instrumental goal and is fueled by the notion that research findings should be "transferred" to decision-makers ⁸. It builds on the idea that research questions are more likely to be relevant to policy if the two groups talk to each other early on and that if interaction continues throughout the research process, a relationship of personal trust will emerge. Trust will, in turn, ensure that the decision-maker is (a) exposed to the findings and (b) is more likely to view them as legitimate.

The second level moves beyond instrumental use and reflects a paradigm shift away from the linear construction of "theory-knowledge-practice". It acknowledges that there is knowledge everywhere and that research is a socially constructed practice like any other ⁹. The objective in this case is to promote interaction between two or more worlds such that learning is advanced for all involved. The content of that learning is spurred by the different perspectives which researchers and decision-makers bring to the table.

The initiative builds on the work of many others that have studied and experimented with ways to increase the impact of research. A well-known idea about how to increase research use focuses on funding post-research completion activities that push findings towards decision-makers. Dissemination efforts, such as policy briefs, workshops, and press conferences, are increasingly included in research grants as additional (but integral) activities to be funded ¹⁰. The IDRC Research Matters program is an example of targeted funding for push activities. Another way to improve uptake is to build the pull, i.e. the capacity and interest of decision-makers to access and understand research, including training programs ^11,12,13. This strategy includes, such as REN, promoted by the Agency for Health Care quality and Research (AHRQ) in the U.S., or the EXTRA, sponsored by the Canadian Health Services Research Foundation, as well as initiatives such as the Pan-American Health Organization's BIREME virtual library for decision-makers.

Research on the use of research, however, suggests that there is another factor that strongly predicts successful uptake: the level of interaction between researchers and decision-makers throughout the research process ^14,15. Landry et al. ¹⁵ describe the interaction as a series of disorderly interactions between researchers and users, rather than a linear sequence that begins with the needs of researchers or the needs of users. Lomas et al. ¹⁴ in Canada call this "linkages" and suggest that, in addition to push and pull activities, there need to be repeated situations of contact (personal or otherwise) from the start ¹⁶. In an idea that reckons back to action science, and to a lesser extent participatory research, they believe that more sustained and intense interaction will result in a greater likelihood of uptake ^17,18.

While there is growing consensus on the importance of linkages, it is an umbrella concept and little is known about how and why it improves research uptake. What are the conditions of successful linkages? Are there always pre-existing political or personal ties that facilitate relations? If so, is it reasonable for funders to try and prompt such interactions? Are there certain kinds of health policy research that cannot or should not engage in linkages? In short, is linkages a sound strategy for research funders to explicitly pursue, and if so, how can it be promoted?

IDRC and PAHO, with assistance from AcademyHealth, proposed to explore this question in the design of their program of research on Social Protection in Health in Latin America and the Caribbean. Social protection in health is defined by PAHO as "society's guarantee, through public authorities, that individuals or groups of individuals can meet their health needs and demands through adequate access to health services of the system or any of the existing sub systems in the country, regardless of their ability to pay" ¹⁹. IDRC has also prioritized the issue through its Program Initiative on Governance, Equity and Health, which supports applied research that will both strengthen and monitor the capacity of governments to ensure equitable financing and delivery of priority public health and health care services, especially to marginalized and underserved populations. The idea was to alter the traditional funding modality from the start, such that the conditions of research (the structure) and the subsequent dynamic of implementation (the process) would be different, producing, we hoped, better chances of research use, as well as unpredictable patterns of learning along the way.

New funding arrangements

The initiative was conceived as a two-part process that began with a call for research proposals that required a formal agreement between the researchers and a decision-maker group that could be either public or private. In Phase I, small grants were provided to nine of the sixty-seven projects submitted. The number of excellent projects that had to be rejected demonstrated what funders in the Region have long known: there is enormous unmet demand for research funding. Repeating past patterns, most of the proposals originated in Argentina, Colombia, Brazil, and Mexico. Less demand was observed in Central America and in the Caribbean, which produced only two proposals each. This first phase of the initiative lasted six months and was used to develop the research protocols in close collaboration with the decision-maker group that was interested in using research results in policies, programs, and practice. Phase II, which began in October 2004, funded the five most successful projects to both implement the research proposal and to foster dissemination and translation activities upon completion.

Three sub-regional research networks served as partners in Phase I: the Network for Health Systems and Services Research in the Southern Cone of Latin America (Redsalud); the Thematic Network on the Economic Evaluation of Health Care Programs and its Application for Decision-Making in Latin America Countries (NEVALAT/Fundación FES), from Colombia; that works in the Andean and Central American region; and the Caribbean Network of Health System Research and Policy, based at the University of West Indies, Jamaica, a group still in formation. The networks were active in the dissemination of the call, formed part of the selection team, and provided technical support to the teams in their respective geographical areas.

Selection criteria emphasized the importance of a solid working agreement with a decision-maker body that could potentially use the research to spur change in the health system. At Phase I, less importance was placed on the clarity of the research question, which we anticipated could be improved during the planning period. Decision-makers were loosely defined as any governmental or non-governmental group that could use the research results to raise awareness or take action that seeks to extend social protection in health.

Several complimentary strategies were also employed to emphasize the importance of the alliances. By requiring teams to submit a new proposal for a Phase II selection, the intent was to provide a strong incentive to strengthen relations with the decision-maker, as well as, of course, to improve the clarity, rigor, and relevance of the proposal. Teams were also asked to produce a report on the process of interaction between the two groups, which was submitted at the end of Phase I with the proposed protocol. This prompted teams to reflect on the dynamic of their relationships, and later provided the basis for participation in the overall assessment of the program strategy. Additionally, while research organizations were the recipients of the grants, decision-makers participated in two regional meetings as equal partners.

We focus in this paper on the five projects approved for Phase II because it was not until after the competition was over that teams were willing to reflect critically on the experience. Additionally, these were the projects that were best able to adhere to the strategy defined by the initiative.

The research projects

The five projects selected for Phase II addressed different issues and research questions, but they shared a common interest in identifying innovative ways to improve access to health care by their countries' most disadvantaged groups. We begin by summarizing their original research questions, and then describe the evolution of each project as the relationship between researchers and decision-makers progressed during Phase I.

Báscolo et al. ³, from Argentina, proposed to evaluate the design and implementation of a public sector maternal and child health insurance in the Province of Buenos Aires. In the original proposal they formulated the following research questions: how do different dynamics of governance at the municipal and provincial levels affect the management and outcome of the Maternal-Child Health Insurance Program?

Viana et al. ⁴ initially proposed to analyze the challenges of social protection in health in the context of inequities in the Brazilian Unified National Health System (Sistema Único de Saúde – SUS), and their research questions focused on the nature of the relationships among different levels of government.

Gordon-Strachan et al. ⁵ proposed to evaluate user fees for preventive care services and to examine impact on health seeking and coping behaviors. They asked what the impact of user fees was for preventive services, as measured by utilization and outcomes. They also proposed to explore alternative financing mechanisms.

Hernández-Bello et al. ⁶ proposed to improve the effectiveness of social protection policies for populations displaced by violence in Bogotá, Colombia. Their central questions were: how can access and quality of healthcare for refugees living in Bogotá be improved, and what is the contribution of a "differential" policy towards the special needs of refugees?

Finally, Cardona et al. ⁷ proposed to analyze alternative organizational and financial designs to guarantee the sustainability of public health insurance for unemployed workers in Medellín, Colombia. They asked: how can municipalities and States work together to create an insurance program for the unemployed? This project included the design, pilot, and evaluation of such an insurance experiment in Medellín.

The range of topics explored under the rubric of social protection in health included the relationship of governance to equity, public insurance targeting priority populations, and barriers to access. Researchers were all based in universities and research institutions. Decision-makers were all governmental, although two were federal, one provincial, and two municipal.

Phase I developments

Consistent with the premise of the program design, we expected that the introduction of this pre-research period of funding (Phase I) would result in protocols that were more relevant and more rigorous than if we had not had the pre-research funding phase. We also expected the decision-makers involved in the project to be more sensitive to the challenges of research and more interested in its potential contributions to policy than they would otherwise be.

So what did happen over the course of the six-month planning period? As with any intervention, there were both expected and unexpected results. Our assessment was based on the teams' self-reported accounts of the interaction between researchers and (a required deliverable of Phase I), and on a questionnaire that we jointly developed at the beginning of Phase II that focused in on some of the common themes that had emerged and would facilitate comparison across the five experiences.

Factors at play during the formation of partnerships

The story began in every case with the researchers learning of the call for proposals through local PAHO offices, regional research networks, or universities. Researchers then sought out decision-makers with whom they wanted to form an alliance. This defined a certain level of ownership by the researchers from the start that may not only be inevitable, but perhaps even positive. The inevitability stems from the fact that it is the researchers, not the decision-makers that will be generating revenues from the research, and so it is, therefore, their "job" to seek out new research opportunities. The potential positive effects concern the balancing of power between the researchers and decision-makers as discussed in the next section.

Regardless of the merits, researchers were in fact the active partners in search of a decision-maker body with whom to form an alliance. In Argentina this choice was a function of prior contacts and relationships ³. In Brazil there was a new Administration at the time, and several of the new authorities had worked with the research team in the past in academic settings ⁴. Interestingly, Argentina and Brazil were the two projects that also began with very broad research topics focusing on governance, and in both cases, despite the intimacy of the relationship between researchers and decision-makers, the research project changed radically during Phase I.

In the three remaining cases - Jamaica ⁵, Bogotá ⁶, and Medellín ⁷ –, prior contacts did not play as important a role in defining the teams. The researchers had pre-determined a topic of investigation that was relatively well defined, although complex, and they quickly identified specific government officials that they hoped had the political will to produce change in that policy arena.

In the case of Jamaica ⁵, the topic of user fees was central to the political agenda, and even though the researchers did not have close relationship with the decision-makers, and there were a number of competing perspectives on this topics, the good institutional relationships between the university and the Ministry of Health is likely to be one of the reasons for the success of the initiative. For this group, the challenge was initiating contact and having access to the policymakers. In addition to requesting formal appointments, researchers employed "chance meetings" (in hallways and elevators), as well as the influence of intermediaries.

In both Colombian cases ^6,7, the research topics were not at the top of the political agenda, but research teams were able to engage and persuade the decision-makers of their projects' importance.

This preliminary distinction - having prior contact or not – did not appear to be clearly advantageous in either case. Argentina identified the fact that they had had a prior relationship with the decision-makers as a factor that had strengthened the interaction during Phase I. In Brazil the researchers reflected on the important process of building trust with the decision-makers based on their new roles in government. The other three project teams, when asked whether their lack of prior relations weakened the interaction, did not believe that it had negatively affected the projects.

Interactions shape research question(s)

In the two cases with broad research topics, Argentina ³ and Brazil ⁴, the interaction with the decision-maker helped to define and narrow the scope of the topic and the research questions, and to include an evaluative component in the research.

Báscolo et al. ³ reported that in Argentina they engaged in exploratory data collection and analysis that served to inform the structure of the Project and to jointly define the research questions. This process allowed them to develop a shared conceptual framework and to define the central themes of the investigation in accordance with the preliminary data. The questions that emerged during Phase I were the following: What is the level of development and evolution of institutional capacities of the eight municipalities being studied, and how has that affected the program? And how can the reform process be sustained over time?

Viana et al. ⁴ reported that in Brazil there was a lengthy negotiation that focused on the parameters of the study itself. The Ministry was firm in its insistence that the study focus on a region that was a high priority for the Administration's agenda: the Amazon. The study questions that evolved were: (a) How are relationships among different levels of government and different sectors evolving in the context of the new Amazon health policies? and (b) What is the effect of the regionalization of health policies on the equitable distribution of resources and health outcomes in comparison to other regions?

In the other three cases, the decision-makers played a similar role in pushing the researchers not only to evaluate policies, but also to propose and test policy alternatives. In the Jamaica study ⁵ a series of interviews with key policy personnel were held. During the interviews they discussed whether the study was really necessary given that numerous international investigations had already examined the impact of user fees. They also discussed what outcomes could be used to measure the effects and whether there were other issues that needed to be considered. Most importantly, the Ministry authorities asked that they explore alternative sources of funding that could replace user fees.

In Bogotá ⁶, the information obtained from different sectors – academia, government, religious and other non-governmental organizations – was triangulated and systematized in matrices. They developed a methodology for prioritizing that led to the definition of a series of problems according to their potential sensitivity to intervention. This process was carried out in the context of intense negotiations with all the parties.

Lastly, Cardona et al. ⁷ in Medellín developed a series of simulations of scenarios that would lead to a financially viable health plan tailored to the needs of the unemployed. These scenarios were subsequently presented to the municipal and provincial authorities to gain buy- in for their ideas. While this project was probably the least altered as a result of interaction during Phase I, ironically it is also the project that is most dependent of decision-maker buy-in in order to have an impact.

When we asked researchers from the five teams whether their perspectives on the research topic had changed as a result of the partnership with decision-makers, they reported that while the topic had not changed, their understanding of it had been enriched and become more focused. In all five cases decision-makers were in effect nudging the researchers closer to questions that would produce clear conclusions that could be used in a policy debate.

In search of clearly delineated roles

All of the teams agreed that over the course of the Phase I the relationships between the two groups had changed. The tensions regarding respective roles, and with that the distribution and use of power, were explicitly addressed in several cases. On the one hand, researchers empathized with decision-makers, yet they also struggled to avoid feeling "captured by the other's logic".

As mentioned above, the relationship began with the researchers having a certain quota of power due to that fact that they had initiated the proposal and "chosen" the decision-makers. As Phase I evolved, however, decision-makers' power was established as a result of the researchers need for their approval. Researchers reported that in their eagerness to obtain buy-in and establish trust, they were "tempted" to reduce their own autonomy and to assume the decision-makers' viewpoints as their own.

This was clearly expressed by the Brazilian ⁴ and Jamaican researchers ⁵. In Brazil, the Ministry officials were firm in their insistence that the study focus on an area of high priority for them. In Jamaica, a committee of outside academic experts and Ministry officials was convened to review the proposal and to ensure its methodological rigor. One of the elements that in the minds of the researchers helped them to maintain their autonomy was the fact that they insisted on "ownership" of the initial (albeit subsequently transformed) idea ^3,6,7.

Other elements also played into the process of delineating the respective roles. The case of Brazil ⁴ was notable in that at the beginning of the relationship, the decision-makers, who had themselves been researchers at one time, wanted to take over and define the research project themselves. Similarly, as the interactions continued and researchers became more familiar with the political challenges being faced by the decision-makers, they also reported having become excited about designing policy. At times they indicated that they felt as though they had overstepped the boundary and tried to advise the decision-makers on how to maneuver politically. After this blurring of roles occurred on both sides, the team was able to discuss what was happening and eventually agreed that it was important to explicitly differentiate roles in order to maximize the usefulness of the alliance. The other teams reported similar dynamics ^3,5,6,7, and all agreed that the differentiation of roles was key to establishing a relationship of trust and mutual respect.

This is interesting both as a difference with participatory research, which includes research subjects and researchers. It is also germane to the age-old debate over researcher as "objective" observer, versus researcher as advocate that often plays out in discussions between researchers in the North and the South ²⁰. In a sense, the kinds of alliances established in these projects fall in a middle ground. The teams recognize the importance of autonomy of researchers without going so far as to affirm that "objectivity" is possible or desirable.

There was also a clear appreciation of the special knowledge and skills of each side. In Jamaica ⁵, feedback from the government officials actually led to a decision that a multiple method approach should be used. In Argentina ³ and Brazil ⁴, the researchers emphasized that only as a result of the partnership with the decision-makers did they have access to and understand the real conditions of health services in remote areas, the nature of the relationships among different government levels and different sectors, and more generally, the kind of evidence, and the timing of evidence that would be useful to the policymaking process. Similarly, they retained a self-image as technicians that brought a specific set of skills to the table that the decision-maker respected and even needed. The notion that they were complementing each other was novel and exciting to both the researchers and the decision-makers.

Additional determinants

In assessing the factors that strengthened or weakened the relationships, during a workshop held at the end of Phase I the teams summarized the factors that appeared to have contributed to and to have detracted from successful partnerships. These factors were as follows:

• Having clearly expressed the needs and role of each side;

• Having clearly defined the place and time of meeting with decision-makers, such that focus was exclusively on the research project;

• Having selected a timely and relevant topic that interests the decision-maker.

Factors that were viewed as having weakened the relationships were:

• Geographic distance between researchers and decision-makers in the case of Brazil ⁴ and Argentina ³;

• The time pressure faced by decision-makers to provide policy answers, especially in Jamaica ⁵ and Argentina ³;

• In some cases, the very preliminary stage of the policy process in which decision-makers were engaged slowed down the researchers, as in Brazil ⁴, Bogotá ⁶, and Medellín ⁷;

• Government turnover of officials in Brazil ⁴, Bogota ⁶, and Medellin ⁷;

• The uncertainty of not knowing if the proposal would be funded in Phase II.

A side note on participatory research

The Bogotá research project merits some specific consideration because it is participatory in its design ⁶. It includes other actors, in addition to the governmental decision-makers, in the planning and the implementation. A committee was established to oversee the project, including leaders from the different refugee groups (who are highly antagonistic towards each other), and representatives from health care provider organizations. They also identified and worked with a second circle of actors, including the Church, NGOs, and international agencies. The researchers, in this case, not only had a direct relationship with the decision-makers, but also played the role of mediators among the different interest groups.

A preliminary reflection on this situation is that it probably altered the balance of power between researchers and decision-makers. For example, the mediation role could elevate the importance of the researchers in the eyes of the decision-makers. On the other hand, the other groups of actors are also decision-makers, since they are able to influence change by altering their own behaviors. This could, of course, play out as opposition to the decision-makers' position, and could even endanger the researchers' relationship with the decision-makers. This has not been the case in Bogotá, but it will be interesting to observe the evolution of the project in the next two years.

The case of Bogotá ⁶ was also notable in that it was the exception. It may be worth asking whether the project design has a built-in disincentive to bring more actors into the discussion of the research objectives. Establishing a strong relationship with the decision-makers is a challenge in itself, and adding further dynamics with groups that have varying interests may introduce additional complexity.

Unexpected results

Some of the projects reported preliminary impacts that went beyond producing a research protocol that had the decision-maker's agreement. Important examples of these unexpected results that came about during Phase I included the following:

• In Medellín ⁷, the team was able to obtain a commitment from both the municipal and departmental levels to contribute with financial resources to the insurance fund;

• This also occurred in Bogotá ⁶, where the provincial secretariat gave both political and financial support to the project. In this case, following a series of meeting with the communities' leaders, the mayor actually changed his views on whether it was necessary to provide refugees with specific health programs that were different from those that other migrant groups in the city receive; he had previously opposed the proposal;

• In the case of Jamaica ⁵, the Minister of Health became interested in assessing alternative financing strategies to replace user fees, and during the first phase asked accountants to begin systematically reporting the administrative costs of collecting such fees;

• In Brazil ⁴, three major research groups from different institutions located in different States established a working relationship for the first time.

Another side effect of the program design was that it brought attention to the issue of research use, and to the importance of communication with potential research users, and generated debate and awareness in multiple sectors. In Argentina ³, the Lazarte Institute introduced debate on the topic with their students in the School of Public Health. In Brazil ⁴, the discussions on research use spilled over into academic institutions, and the team led a series of discussions in different departments of their universities and in the Ministry of Health. Similarly in Medellín, Antioquia ⁷, the principal investigator reports that the project has impacted the organization of the university's research program with alliances with different types of decision-makers becoming an important objective.

Decision-maker involvement

Researchers expressed concern that the slow pace of the selection process and the funding flow, in addition to the time required to carry out the research, could result in a loss of commitment. Indications are that the commitment of the decision-makers to the projects has been strong to date. They have attended two week-long regional meetings and participated actively. They have spent time discussing protocols with researchers. They have allocated funding and in some case altered policy already. Yet by their own account, the decision-makers are atypical in their willingness to do this. They insisted that not all decision-makers are the same and that any consolidation of a program design like this in the future should consider this variation.

Theories of knowledge diffusion are premised on the idea that early adaptors and opinion leaders are strategic to the process of change ²¹. Apparently the researchers in this initiative have selected decision-makers that are early adaptors, and this may well be a concept that would be worth elaborating in a future call for proposals.

Use of funds

A final observation is that the program design allowed considerable flexibility in the use of funds. The five teams did indeed use their funds in many different ways, with one using it exclusively for travel, while another used it for salaries. Some actually began testing pilot instruments and did preliminary research. In future iterations of this design, and based on the lessons learned in this program, it may be important to retain the flexibility, but to tailor the process to each situation more precisely. Budgets could be adjusted to the demonstrated needs of each team. Additionally, Phase I could be shortened to three months, and the range of fundable activities could be more clearly delineated in the call for proposals.

Phase II developments

The implementation phase of these projects began in October 2004 and is still ongoing at the time of this writing. The primary findings of the five experiences tend to validate the model. However, significant problems have emerged as a result of the institutional instability among the decision-makers, the complexity of the research topic in some cases, and the difficulties in guaranteeing a steady funding flow for the projects.

The impact of decision-maker turnover on buy-in

The issue of decision-maker turnover is a reality in all countries, especially in Latin America, and is a likelihood that needs to be considered in the design of such programs. The Brazilian ⁴, the two Colombian ^6,7 and the Jamaican teams ⁵ faced significant difficulties as a result of changes. In all four cases with decision-makers departing, the commitment of the new decision-makers to the research was significantly diminished. Researchers have had to spend time once again trying to obtain decision-maker buy-in, rather than using the project resources to deepen the dialogue with the decision-maker. In all four cases, to their credit, the researchers have, however, managed to keep the projects alive by expanding their contacts to mid-level officials and pursuing relations with the new authorities.

The Argentine team ³, on the other hand, escaped turnover and has been remarkably successful in deepening their relationships. The number of meetings has increased over time, and there has been more in-kind collaboration with the data collection and analysis from the decision-makers. Decision-makers also found that there was a certain prestige in being associated with a university team that has benefited them politically. In short, Báscolo et al. ³ in their paper in this issue report that there has been significant mutual learning along the way, and that preliminary data are already being used by the decision-making team to inform their decisions.

Difficulties in the funding flow

In several cases there was a lag between the funding of Phases I and II, which was problematic for the country teams. This was especially difficult for the Brazilian team, which in effect lost precious time with a group of policymakers that, once the funding finally did come through, were in the process of resigning from the Ministry of Health.

The complexity of topics as an impediment

Bogotá ⁶, Medellín ⁷, and Brazil ⁴ have confronted an additional challenge as a result of the complexity of the social problems they set out to tackle. There are no quick policy fixes to increase access to care for the displaced in Bogotá, for the unemployed in Medellín, or for the poor in the Amazon. Moreover, the legal implications of the two Colombian projects run up against the prevailing social security legislation. Reforming those laws, however, is a highly political endeavor that is clearly beyond the scope of these projects. Changes in government policies also altered the environment and interest of decision-makers in the research in Jamaica ⁵ and Brazil ⁴, situations that again exemplified the critical (and somewhat uncontrollable) function of context in determining the success of these projects.

In synthesis, the object of analysis evolved in these projects not only as the relationship between researchers and decision-makers occurs, but also as contextual developments unfolded. The challenge for the country teams, then, is to adjust to these two levels of change in ways that maintain a focus on producing research results that can be used to inform decisions ³. This requires significant flexibility and a dynamic perspective, competencies that are not developed in the context of formal research training.

Final reflections

There were a number of benefits to the program design that emerged from these five experiences, and while they are not generalizable, they do provide lessons worth sharing and building upon.

The first was that both the program leadership and the participants strongly agreed that the pre-research phase improved the policy relevance of the research proposal. Since relevance is assessed primarily by the decision-makers and they played a role in defining the research question, this comes as no surprise, but it is an important confirmation of this design.

A second benefit was that the decision-makers played a positive role in helping to focus the research question. Researchers reported that they gained detailed operational knowledge through discussions with decision-makers that helped them tighten their methods and sharpen their focus.

They were also pushed to include evaluative components in the research that would not otherwise have been considered. As noted in a recent review of research on health sector reform in Latin America and the Caribbean carried out by PAHO and IDRC, there is a dearth of this kind of public research ²². Creating partnerships with decision-makers, then, appears to be one way to stimulate more evaluative research.

In keeping with this finding, all but one team felt the scientific rigor of the projects had improved as a result of the extended planning and interaction with decision-makers as well. The Argentine team ³ reported that discussion with decision-makers helped them adjust their theoretical framework on governance to specific conditions of municipal governments and health insurance. The Brazilian team ⁴ did not share this view; they argued that scientific rigor is the exclusive responsibility of the researchers. Their perspective may have been a result of the difficulties they faced in differentiating roles, and the need to strongly demarcate the researchers' autonomy. One important concern in the PAHO/IDRC leadership was whether the exclusion of four projects from the second phase would jeopardize the relationships researchers had formed with decision-makers and create a barrier for future efforts. In follow-up discussions with those teams, however, we have learned that three of the four projects moved forward with local funding. Of the five teams that were selected, all indicated that if their project had not been selected for funding, they would have either scaled it back and carried it out with local funding, or sought external funding from elsewhere. Our preliminary conclusion, therefore, is that the decision-makers' involvement in the research design vastly increased the likelihood that the projects be funded by other sources.

In looking across these experiences, the question arose as to what was gained by insisting on differentiated roles and responsibilities. The types of decision-makers and the types of relationships with researchers present in this initiative were far more varied than have thus far been contemplated in the literature. Yet in all of the projects there was a concern that by blurring distinctions, both researchers and decision-makers run the risk of being "captured" by the other group's interests and logic. Each of the teams found it critical to explicitly differentiate roles in order to establish a balanced and effective partnership.

The notion of developing a social contract between the two sides was discussed as a challenge for the future. Teams drew on their experiences to identify the following elements as components of that contract:

• Clearly defined professional roles: (a) explicitly expressing fears and expectations; (b) emphasis on the decision-makers' role in helping define the problem and facilitating access to data; (c) emphasis on the researchers' role in defining research strategy and in retaining autonomy in the analysis of results; and (d) formalizing alliances, such that if there is decision-maker turnover, the project can continue.

• Commitment to the overall principals of the health system, such as equity, participation, efficiency, effectiveness, etc.

• Cooperative attitudes, such as being: (a) reciprocal; (b) respectful; (c) trusting; and (d) transparent.

The analysis of the cases presented in this issue also reconfirms the notion that "windows of opportunity", determined by specific historical and contextual circumstances, are critical determinants of research uptake ²³. Such windows create visibility and interest in gaining new knowledge around a specific topic.

A side commentary on how this experience relates to research action and participatory research is that the above-mentioned attributes identified by the teams do resonate with both methodologies. However, there also seem to be differences. The first is that research action and participatory research are usually focused on the direct beneficiaries of change at the community level (as in the case of Bogotá project ⁶), rather than the policy-makers. The second is that in this experience the researchers and the decision-makers have chosen to clearly differentiate their roles; the decision-makers, not the researchers, are the agents of change, and the decision-makers do not participate in the research, they simply influence certain aspects of it, in particular the framing of the research question.

There were also weaknesses to this program design that emerged from the group analysis. The first involved the difficulties in providing decision-makers with incentives to remain active in the project, when their timelines tend to be more short-term. Although none of the decision-makers withdrew from their alliance of their own accord, both sides reported frustration. A second limitation was that, while Bogotá was able to establish participatory dynamics with nongovernmental actors, most of the projects focused their energies on constructing the relationships with a single decision-maker that was governmental. Expanding alliances beyond government would appear to be a key lesson to protect against the devastating effects of political turnover on a research project.

The real possibility, if not probability, of turnover among high-level decision-makers in governments is a serious threat to this research program funding model. While the Argentine team ³ escaped change in their team composition and, as a result, has been able to consolidate and deepen their partnerships, this was indeed the exception, not the rule. While the other teams have continued to move ahead, they have quite wisely chosen to expand the stakeholders involved and to include mid-level officials that are less likely to be caught up in political strife.

The five projects also reconfirm the notion of scientific knowledge as a source of power ²⁴. "As a power resource, knowledge plays innumerable roles, which change with place, time, and circumstances. Each concrete case also involves different explanatory variables for policy change, and each variable implies that the research has played a specific role. Some variables are particularly important" ²⁵ (p. S11).

The Brazilian case ⁴ illustrated how important research can be to policy-makers as they pursue legitimization of a given policy or program. The research offers visibility for a programmatic area considered a priority for a new Administration. A similar situation exists in Argentina ³, although in that case the program is well underway, while in Brazil it is still in its infancy and its implementation is still somewhat tenuous.

The question of power, then, is closely tied to the uptake of research ^24,26, and it is likely that, as Bowen & Zwi ²⁶ (p. 601), have suggested, "the way in which research evidence is combined with other forms of information is key to understanding the meaning and use of evidence in policy development and practice. (...) A major challenge to contextualizing evidence for policy-making is recognition that a broad information base is required [and] ( ) considering the evidence within the context in which it will be used is critical for effective policymaking and practice^".

Almeida et al. ²⁵, in their discussion of Brown ²⁴, Weiss ²⁷, and Majone ²⁸, emphasize this point as follows: " the potential contribution of research to the decision-making process has less to do with offering definitive solutions to the problematic issues in debate and more with improving the quality of the terms of the debate. Thus, the ability to change the nature of public debate on a given issue is an important form of power, because bringing ideas, proposals, and interests into confrontation is an important force in changing the balance of power among the various contesting groups" ²⁵ (p. S13).

A final assessment of whether (and to what extent) this type of research design will have an impact on innovation in social protection in health will, of course, need to be addressed upon completion of the initiative. The methodological challenges of measuring research impact are well known, and there will undoubtedly be limits to our ability to answer this question. Similarly, a serious assessment of this research funding strategy would evaluate the cost opportunity of the investment. While the amount of the grants could be reduced in the future, it remains a significant additional cost, and a valid question is whether the same amount of resources would be better spent on push or pull strategies. While both the final impact of this strategy on research use and the cost opportunity of Phase I are beyond the scope of this paper, the difficulty in answering such questions points to the need to begin a program of study that will seriously assess the many strategies funders are now using to increase research uptake.

Contributors

The underlying ideas and the structure of this project design were developed jointly by the authors and representatives from partner organizations. P. Pittman wrote the article and revised the final version and C. Almeida provided feedback and additional comments that linked this paper to the introductory article and the case studies published in this special issue, commented on and wrote in each successive version, and assisted in shaping the paper for publication.

Acknowledgements

In addition to the researchers and decision-makers directly involved in this project, the following colleagues contributed to its design and assessment: Roberto Bazzani and Christina Zarowsky from International Development Research Centre; Eduardo Levcovitz, Daniel Lopez Acuña, and Soledad Urrutia from Pan-American Health Organization; and Celia Almeida, Gabriel Carasquilla-Gutiérrez, and Elsie LeFranc from the three regional health systems research networks.

Submitted on 19/Jun/2006

Approved on 27/Jul/2006

References

• 1. Greenberg JN, Choi T. The role of the social sciences in health services research: an overview. In: Choi T, Greenberg JN, editors. Social science approaches to health services research. Ann Arbor: Health Administration Press; 1982. p. 2-20.
• 2. Kralewski JE, Greene BR. Health services research and the evolving health systems. In: Levey S, McCarthy T, editors. Health management for tomorrow. Philadelphia: J. B. Lippincott Co.; 1980. p. 293-307.
• 3. Báscolo E, Yavich N, De León AS. El proceso de interacción investigadores y tomadores de decisiones: un estudio de caso. Cad Saúde Pública 2006; 22 Suppl:S47-56.
• 4. Viana ALd'A, Machado CV, Lima LD, Baptista TWF, Mendonça MHM, Heimann LS. Pesquisa para a tomada de decisão: um estudo de caso no Brasil. Cad Saúde Pública 2006; 22 Suppl:S57-67.
• 5. Gordon-Strachan G, Bailey W, Lalta S, Ward E, Henry-Lee A, LeFranc E. Linking researchers and policy-makers: some challenges and approaches. Cad Saúde Pública 2006; 22 Suppl:S69-76.
• 6. Hernández-Bello A, Vega-Romero R. Más allá de la diseminación: lecciones de la interacción entre investigadores y tomadores de decisiones en un proyecto de investigación en Bogotá, Colombia. Cad Saúde Pública 2006; 22 Suppl:S77-85.
• 7. Cardona A, Nieto E, Restrepo R, Sierra O, Cárdenas CE, Aguirre F. Una propuesta de innovación en políticas de aseguramiento en salud, basada en hallazgos de investigación: el caso de los trabajadores cesantes en Medellín, Colombia. Cad Saúde Pública 2006; 22 Suppl:S87-96.
• 8. Weiss CH. Knowledge creep and decision accretion. Knowledge: Creation, Diffusion, Utilization 1980; 1:381-404.
• 9. Wenger E. Communities of practice: learning, meaning, and identity. Cambridge: Cambridge University Press; 1998.
• 10. International Development Research Centre. Research matters: call for proposals. http://www.research-matters.net/en/ev-59144-201-1-DO_TOPIC.html (accessed on Apr/2004).
• 11. AcademyHealth. AHRQ ULP workshop to strengthen policy analysis skills. Final report: "Ren Retrospective". Washington DC: AcademyHealth; 2003.
• 12. Canadian Health Services Research Foundation. Extra/Forces. http://www.chsrf.ca/extra/index_e.php (accessed on 13/Jun/2006).
• 13. Virtual Health Library. Health information locator. http://lis.bvs.br/xml2html/xmlListT.php? xml%5B%5D=http://lis.bvs.br/lis- Regional/I/define.xml&xsl=http://lis.bvs.br/lis-Regional/home.xsl (accessed on 13/Jun/2006).
• 14. Lomas J. Improving research dissemination and uptake in the health sector: beyond the sound of one hand clapping. Hamilton: McMaster University Centre for Health Economics and Policy Analysis; 1997.
• 15. Landry R, Lamari M, Amara N. Extent and determinants of utilization of university research in government agencies. Public Adm Rev 2000; 63:192-205.
• 16. Lavis J, Ross S, McLeod C, Gildiner A. Measuring the impact of health research. J Health Serv Res Policy 2003; 8:165-70.
• 17. Putname R, Smith D. Action science: concepts, methods and skills for research and intervention. San Francisco: Jossey-Bass; 1985.
• 18. Whyte WF. Participatory action research. New York: Sage Publications; 1991.
• ¹⁹ Pan American Health Organization. Extension of social protection in health: joint initiative of the Pan American Health Organization and the International Labour Organization. http:/www.paho.org/English/gov/csp/csp26-12-e.pdf (accessed on 31/Jul/2006).
• 20. Moynihan R. Using health services research in policy: case studies from nine countries. Washington DC: Milbank Memorial Fund/AcademyHealth; 2004.
• 21. Rogers E. Diffusion of innovations. 4^th Ed. New York: The Free Press; 1995.
• 22. Pittman P, López-Acuña D. Research on health sector reforms in LAC: conceptualizing the research to policy link. In: Almeida C, Bazzani R, Pittman P, editors. Investigación sobre reformas del sector salud en América Latina y el Caribe: reflexiones sobre sus contribuciones al desarrollo de políticas. Washington DC: Organización Panamericana de la Salud; 2001. p. 1-16.
• 23. Trostle J, Bronfman M, Langer A. How do researchers influence decision-makers? Case studies of Mexican policies. Health Policy Plan 1999; 14:103-14.
• 24. Brown L. Knowledge and power: health services research as a political resource. In: Ginzberg E, editor. Health services research: key to health policy. Cambridge: Harvard University Press; 1991. p. 20-45.
• 25. Almeida C, Báscolo E. Use of research results in policy decision-making, formulation, and implementation: a review of the literature. Cad Saúde Pública 2006; 22 Suppl:S7-33.
• 26. Bowen S, Zwi AB. Pathways to "evidence-informed" policy and practice: a framework for action. PLoS Med 2005; 2:600-5.
• 27. Weiss C. The many meanings of research utilization. Public Adm Rev 1979; 39:429-31.
• 28. Majone G. Evidence, argument and persuasion in the policy process. New Haven: Yale University Press; 1989.

Publication Dates

• Publication in this collection
  10 Nov 2006
• Date of issue
  2006

History

• Accepted
  27 July 2006
• Received
  19 June 2006