Framing effects on foreign policy: experimental evidence from emerging countries and the Argentine-Brazilian rivalry¹

Efeitos de enquadramento em política externa: evidência empírica de países emergentes e a rivalidade argentino-brasileira

Efectos de encuadramiento en política externa: Evidencia de países emergentes y la rivalidadargentino-brasilera

Les effets de cadrage sur la politique étrangère: La preuve expérimentale des pays émergents et la rivalité argentino-brésilienne

Abstract

Civil society plays an increasingly important role in the formulation of foreign policy in emerging countries. This article investigates whether public opinion is sensitive to framing effects regarding foreign policy. Data from a survey experiment with a sample of 1,530 students at the Universidad de Buenos Aires and the Universidad Nacional de Avellaneda, we find that participants are sensitive to framing effects on foreign affairs. The interviewees changed their preferences when stimulated by information regarding Brazilian economic growth and military expenditure in comparison with Argentina. In turn, this effect was more pronounced among a) people who tend to stay less informed regarding foreign affairs and b) individuals who are more nationalistic.

framing effects; survey experiment; foreign policy; Argentina; Brazil

Resumo

A sociedade civil desempenha um papel cada vez mais importante na formulação da política externa nos países emergentes. Este artigo investiga se a opinião pública é sensível a efeitos de framing no que diz respeito à política externa. Através de um survey experimental aplicado a uma amostra de 1530 estudantes na Universidad de Buenos Aires e na Universidad Nacional de Avellaneda, descobrimos que os participantes são sensíveis a efeitos de framing em temas de política exterior. Os entrevistados mudaram suas preferências quando foram estimulados com informações sobre o crescimento econômico brasileiro e os gastos militares de forma comparada com a Argentina. Por sua vez, esse efeito foi ainda mais pronunciado entre a) pessoas que tendem a se informar menos sobre assuntos internacionais e b) entre indivíduos mais nacionalistas.

efeitos de framing; experimento de survey; política externa; Argentina; Brasil

Resumen

La sociedad civil desempeña un papel cada vez más importante en la formulación de la política externa de países emergentes. Este artículo indaga si la opinión pública es sensible a efectos de framing en lo que concierne a política externa. A través de un survey experimental, aplicado a una muestra de 1530 alumnos en la Universidad de Buenos Aires y en la Universidad Nacional de Avellaneda, encontramos que los participantes fueron sensibles a efectos de framing en temas de política exterior. Los entrevistados cambiaron sus preferencias cuando recibieron información respecto al crecimiento económico brasilero y sus gastos militares de forma comparada con la Argentina. A su vez, este efecto fue más pronunciado entre: a) personas que tienden a informarse menos sobre asuntos internacionales y b) individuos más nacionalistas.

efectos de framing; survey experimental; politica externa; Argentina; Brasil

Résumé

La société civile joue un rôle de plus en plus important dans la formulation de la politique étrangère dans les pays émergents. À travers une enquête auprès de 1530 élèves de l'Université de Buenos Aires et de l'Universidad Nacional de Avellaneda, nous avons constaté que les participants étaient sensibles aux effets de cadrage concernant les affaires étrangères. En effet, les préférences des personnes interviewées changeaient lorsqu'elles étaient stimulées par des informations concernant la croissance économique et les dépenses militaires brésiliennes, en comparaison avec l'Argentine. De plus, cet effet était plus prononcé chez a) les gens qui ont moins tendance à se tenir informés des affaires étrangères et b) les gens plus nationalistes.

effets de cadrage; enquete; politique étrangère; Argentine; Brésil

Introduction

Public opinion has a specific influence on the formulation of public policy (Howlett, 2000) in democratic regimes where rulers rely on newer forms of public consultation, are accountable to voters, and need legitimacy (Echegaray, 2001). Thus, although foreign policy cannot be considered a "traditional" public policy (Lentner, 2006), the perception that public opinion has regarding this subject has become a matter of concern for academics and policymakers (Faria, 2008).

Historically, it was thought that due to the average citizen's lack of interest in international affairs (Lippmann, 1932; Mueller, 2002), public opinion had little influence over foreign policy formulation (Jacobs and Page, 2005). Nevertheless, over time certain issues of a greater sensitivity, such as the number of deaths in military operations (Aldrich et al., 2006, p, p. 478), have captured the public's attention and repeatedly risen to importance on the electoral agenda (Holsti, 1992). As a consequence, this has increased the influence of public opinion on foreign policy formulation in developed countries (Foyle, 2004).

With regard to emerging countries, processes such as economic liberalization have meant that civil society is increasingly taken into account in discussions about foreign policy. A clear example is the Brazilian case, in which just after the end of the military dictatorship and the subsequent democratization of the public sphere, foreign policy formulation has become increasingly complex, now including civil society actors such as trade unions, NGOs, business associations, and academics (Milani and Pinheiro, 2013). In this regard, it has been noted that the average Brazilian voter has begun to associate his/her economic welfare with the results achieved by the government in foreign affairs (Lopes and Faria, 2014), which has led to the creation of an agenda in public opinion regarding foreign policy (Lopes, 2011).

Our article aims at contributing to the public opinion literature surrounding perceptions of foreign policy issues in emerging countries. More particularly, this study investigates whether these perceptions are sensitive to framing effects. The main objective of our work is to understand the sensitivity of Argentines to framing about Brazil's growing power. To this end, we applied an experimental survey in Argentina to a convenience sample composed of 1,530 students belonging to the Universidad de Buenos Aires (59% of the sample) and the Universidad Nacional de Avellaneda (41%). We subsequently looked at whether new information about Brazil's economic growth and military spending changed respondents' opinions on the best strategy Argentina might adopt in relation to Brazil: align itself with its neighbor or seek new regional partners to counterbalance Brazil's power.

The objective of this work is also based on the literature analyzing (a) the historic rivalry between the two countries (Selcher, 1985; Russell and Tokatlian, 2011; Merke, 2015), (b) the leadership role that Brazil has assumed recently (Malamud, 2011; Schenoni, 2012; Mouron and Onuki, 2015), and (c) the possible reactions of middle powers to the regional hegemon's growth (Kaufman, 1992; Schroeder, 1994; Waltz, 2000).

Our findings show that when respondents receive information about Brazil with no point of comparison with Argentina (see Appendix I), the impact on their opinions is null. However, when Brazil's level of growth and military spending are contrasted with Argentina's (see Appendix II), the interviewees prove to be sensitive to framing effects, with an inclination toward thinking that Argentina should seek new partners to counterbalance Brazilian power. In turn, this effect is even greater among people who reported that they did not stay regularly informed about international affairs and among those who believed that Argentina has a greater role in the international arena. This finding shows that survey experiments can generate effects that are only observable among particular subgroups of our samples (Barabas and Jerit, 2010, p. 226).

Our article is structured as follows. First, we contextualize our research problem, presenting a historical evolution of the relationship between Argentina and Brazil and pouring through the public opinion research aimed at capturing Argentines perceptions of their neighbor. Then we delve into the theoretical framework, addressing the debate about framing effects and drawing on a few survey experiments on foreign policy issues conducted in the United States. Next, we explain our methodology and present our survey results without considering the different framings. We then go on to discuss the results of our experimental research and, finally, our conclusions, providing suggestions for a future research agenda.

From rivalry to regional hegemony

Brazil is the most important country in Argentina's contemporary foreign policy (Russell and Tokatlian, 2011). However, throughout their history, the two countries—the largest in South America—have gone through phases of rivalries and alliances, only stabilizing their relationship in 1985 with the rapprochement between presidents Alfonsin and Sarney in Foz do Iguaçu. Since then, Brazil has been seen as Argentina's closest ally and vice-versa (Selcher, 1985; Hirst, 1987; Bandeira, 1987).

In economic terms, bilateral trade grew exponentially between the two with the creation of Mercosur in 1991. As Figure 1 shows, Argentina enjoyed a trade boom during the 90s, with the trend reversing after the deep crisis of 2001, when Brazil started to set the pace for the countries' commercial relationship. Today, even though 20% of Argentine exports go to the Brazilian market, Argentina absorbs only 8% of its neighbor's exports, which enjoy both a higher aggregate value and a higher level of industrialization (Bernal-Meza, 2008).

Figure 1
Evolution of bilateral trade

On the other hand, analyzing the historical evolution of both countries using the National Material Capabilities Index², we can see that the power gap between the two has grown steadily since 1956 when the Kubitschek administration came into power (1956-1961) (Sikkink and Wolfson, 1993) (see Figure 2).

Figure 2
Share of global power (%)

Taking as a reference the theoretical framework of the Realist school, when it comes to foreign policy, regional middle powers like Argentina have two choices regarding regional hegemonic power: stand with it or counterbalance its power. While the first strategy consists in following the regional hegemon's policies, counterbalancing can be approached in two ways: (a) externally, by combining material capabilities with countries in the same situation; or (b) internally, by mobilizing its own resources to more effectively resist the hegemonic country.

Historically, Argentina tried to counterbalance Brazilian power following strategy (b). Nevertheless, by the mid 1980s, when Brazil achieved a clear military superiority, Argentina once again was forced to choose between two foreign policy strategies regarding its neighbor: align itself with Brazil to achieve greater relevance in the international arena, or align itself with other regional countries in order to counterbalance Brazilian power.

Among the research that has systematically tried to capture the perceptions of the Argentine public regarding foreign policy, the efforts made by the Argentine Council for International Relations through the project "La opinión pública argentina sobre política exterior y defensa" should be noted³. In the 2002⁴, 2006⁵, 2010⁶, and 2015⁷ editions, one of the questions included asked with which country Argentina should establish closest relations. If we focus only on "Brazil" as an answer (see Figure 3), we can see that public opinion has remained generally stable over time, reaching a peak in positive responses of 18% in 2006 and 2015, but remaining behind options such as "Europe" in 2002 and 2006 (24% and 27%, respectively) and the "United States" in 2010 and 2015 (47% and 21%, respectively). Meanwhile, policy-makers put Brazil as the second most chosen option in the 2002 and 2006 editions, both times behind the United States. However, this configuration changed in the 2010 edition, with Brazil taking first place with 42% of the valid responses and China securing second place with 18%. Brazil's relevance decreased to 29% in 2015, but it has still remained the most chosen response.

Figure 3
Perception about Brazilian relevance (%)

At this point, it is worth asking ourselves how sensitive Argentines' perceptions on foreign policy are to framing effects. Drawing on the literature on framing effects, in the next section we present our own theoretical framework together with some experiments applied in the United States that focus their attention on specific foreign policy issues.

Theoretical framework: framing effects and foreign policy

Research on framing effects can be found throughout the Social Sciences, as well as in various subfields of the Political Science field (Druckman, 2001, p. 226). In fact, the high interest in the topic has often been accompanied by a lack of clarity in the definition of what exactly framing effects are and how they influence public opinion (Nelson, Oxley and Clawson, 1997). To account for this conceptual vagueness, this article defines framing effects as "small changes in the presentation of an event or topic, but which have large effects on the perception that the public has regarding these issues" (Chong and Druckman, 2007, p. 104).

Since its inception, the specialized literature has shown that the way in which a question is worded has the ability to influence respondents' answers (Rugg and Cantril, 1942, 1944; Payne, 1951; Schuman and Presser, 1977, 19, 1981). However, with the passing of time and advances in the area, framing effects have come to be classified into two main groups: (a) those stimuli that, while presenting the same data (but in a different way), alter interviewees' preferences; (b) those stimuli that focus their attention on certain topics and cause interviewees to focus on particular issues at a moment at which they are forming opinions⁸ (Druckman, 2001, p, p. 228-230). Our work is in line with the first type of research, which tries to analyze how the same piece of information, when presented differently, can affect an interviewee's opinions.

Among the classic works that use this approach, the foundational article by Tversky and Kahneman (1981) is certainly among the most noteworthy. Through an experiment involving 307 students from the University of British Columbia, the authors show that when deaths resulting from a given health program were presented relative to the number of patients, respondents were more likely to support the initiative than when they were stimulated simply by the information in absolute terms. In contrast, when the given information focused on the lives that could be saved, the framing effect reversed, and the students showed themselves to be more supportive in relation to the program than when they were stimulated by the information in absolute terms. Drawing on this research, a great number of studies since have tried to analyze how framing effects can affect public opinion perceptions on foreign policy issues⁹.

Undoubtedly the topic that has been discussed most among foreign policy issues, mainly in the American literature, has been war and the impact of casualties on public opinion perceptions (Gartner, 2008). Boettcher and Cobb found that when combat casualties are presented relative to enemy casualties, the negative effect of the latter information is activated and support for the war increases (Boettcher and Cobb, 2006, p. 849). However, other recent research into framing effects related to this topic has presented different results. Americans are concerned not only with the proportion of casualties, but also with their distribution across society (Kriner and Shen, 2013). In turn, these casualties are tolerated more if a future victory is perceived as likely (Gelpi, Feaver and Reifler, 2009). Thus, we can affirm that framing effects are not homogeneous and may be influenced by individual, as well as situational, factors (Boettcher and Cobb, 2009, p. 692).

On the other hand, other authors have focused their attention on issues such as international trade and the propensity of citizens to support economic integration processes. Hiscox (2006) observed for the American case that those people stimulated by framing that included potential loss of jobs were less likely to support trade with other countries than those who did not receive the job loss framing. The author found that less educated people are more sensitive to framing effects, demonstrating again that these effects are not homogeneous and that it is always important to take into consideration heterogeneous effects across different subgroups of the population (Hiscox, 2006). In a related work, after replicating Hiscox's experimental survey in Argentina, Ardanaz, Murillo and Pinto (2013) found that individuals' material conditions factor into explaining both their perceptions on trade policy and, therefore, their sensitivity to framing effects.

Given what we have analyzed so far, we can conclude that: (a) the question regarding the degree of sensitivity of public opinion in relation to framing effects on foreign policy issues is still a matter of discussion; (b) there are studies that have addressed specific foreign policy issues, but few have investigated matters of bilateral relations; (c) almost all of the existing literature has focused its attention on public opinion in developed countries, mainly especially the United States.

The present work furthers this research agenda by providing evidence from an emerging country, Argentina, on how public opinion there reacts to information regarding the growth and military spending of the country's main ally, Brazil. Ultimately our goal is to see if these stimuli affect respondents' perceptions. If so, this would reveal public opinion's sensitivity to framing effects related to foreign policy.

Methodological design

The use of experimental methods has recently expanded into the fields of Political Science and International Relations (Hyde, 2010, p. 73). In this regard, although there is a significant history of experimental work in this area (Morton and Williams, 2010, p. 9), the discipline, as a whole, has seen the resurgence of this methodology in recent years, becoming a widely accepted approach (Druckman et al., 2006, p. 634). Nevertheless, as pointed out by Turgeon and Rennó, this type of methodology is still underused in Brazil and Latin America (Turgeon and Rennó, 2010, p. 147).

In summary, the great advantage of experimental research is that it allows us to manipulate our independent variable, while controlling for the remaining factors that could influence our dependent variable (Neuman and Robson, 2004, p. 205; Babbie, 2013, p. 238). Thus, in contrast with observational research such as cross-sectional and panel surveys (Gaines, Kuklinski and Quirck, 2007, p. 1-2), experimental studies allow us to isolate the effect of the variable we want to analyze and, therefore, are more suitable for answering some specific research questions, particularly those involving causal mechanisms (Morton and Williams, 2010, p. 12).

Our experimental data comes from a survey applied at the Universidad de Buenos Aires (UBA) and the Universidad Nacional de Avellaneda (UNdAv) between April 7^th and May 30^th, 2014¹⁰ to a convenience sample composed of 1,530 students in the Human Sciences¹¹. We chose this type of sample because, despite having some limitations, it has been shown to behave in the same way as general population samples (Druckman and Kam, 2009). Moreover, given the ease of recruiting students, using this type of sample offered an efficient way to test our initial hypotheses before generalizing our theory to the general population (Dasgupta and Hunsinger, 2008, p. 94).

With 308,000 students¹², UBA is the oldest and most prestigious university in Argentina¹³. Before entering the university, students must pass a common course that lasts at least one year and has a 40% dropout rate. In turn, professors and the system as a whole are very demanding compared to other universities, especially for those students who have financial limitations and who must work. Thus, only 23% of UBA students graduate.

On the other hand, located in Avellaneda, on the outskirts of the city of Buenos Aires, UNdAv was founded in 2011 and saw its first graduates in 2015. With 9,311 students—most of them first-generation college attendees¹⁴—this university seeks to fulfill a social commitment, serving as a tool for the reintegration of young people who were excluded from the university system, preparing them to enter the labor market. In this sense, the courses offered are quite diverse, responding to a social environment different from that of traditional universities such as the UBA (see Table 1).

Table 1
Comparison between universities

	Observations	Visited Brazil (%)	Family members	Monthly family income (AR$)	Average age	Born in Buenos Aires (%)
UBA	893	55	4,15	18000	23,8	82
UNDAV	623	28	4,23	12500	25,5	91

• Source: Elaborated by the authors.

With the aim of obtaining a more accurate representation of Argentine society, we conceived of our sample including students from both universities, rather than simply students from either an elite institution (UBA) or a public university with a specific profile such as UNdAv.

To conduct the survey, we came into contact with professors in the Human Sciences at both universities with the help of the academic staff. With their approval, we scheduled times at which we could interrupt their lessons and carry out the survey. We avoided giving participants prior knowledge about the research in order to prevent them from actively seeking information about international affairs before taking the survey. Students were simply invited to participate in a public opinion poll. To incentivize participation, students were offered two vouchers (worth AR $1,000) for books. As suggested by McDermott (2014), in order to maintain anonymity, participants' personal information was collected on a separate sheet of paper and kept apart from the answers.

After answering five questions about foreign policy, Argentina, and its relevance in the international arena, participants were randomly assigned to one of the two experimental treatments or to the control group¹⁵. Both treatments provided real information about Brazilian GDP growth over the last ten years, the sum of money spent by Brasilia on military expenditures, and the number of troops the country has. The only difference was that one of the texts presented the information in an absolute way (see Appendix I), while the other did so relative to Argentina (see Appendix II). As suggested by Boettcher and Cobb (2009, p. 684), a third of our sample did not receive any stimulus, thus serving as the control group necessary to estimate the exact impact of each type of framing. The following procedure was used to assign participants to the three different groups: the absolute treatment was coded as 1, the relative as 2, and the control group as 0. Then we randomized the distribution of these 3 numbers among 900 (UBA) and 650 (UNdAv) cases and mixed the questionnaires according to this random distribution.

Immediately after these stimuli, respondents were questioned about which strategy would be more suitable for Argentina: (a) align with Brazil or (b) seek new regional partners to counterbalance Brazilian power. Also, to more broadly capture Argentinian public opinion perceptions regarding international issues, participants had to tell us which they felt were the three most relevant countries in the international arena, the three countries they believed would be the most important in the next ten years. We also asked respondents to tell us if they believed Brazil was a leader in South America and if they think Argentina should imitate its behavior.

Moreover, we included a number of questions in order to control for political variables: (a) political ideology¹⁶; (b) government approval¹⁷; (c) Argentinian relevance in the international arena as proxy of nationalism¹⁸; (d) the frequency with which respondents inform themselves about foreign affairs¹⁹. Finally, classic socioeconomic questions such as those inquiring about age, sex, gender, and income were also presented. To compare the subsamples, we used simple tests of hypotheses. To incorporate the controls, we used Probit logistic regressions.

General results: align with a future global power and a regional leader

Next, we will present the results of the survey without considering the treatment effects. First, as we can see in Table 2, more than 77% of those interviewed believe that Argentina should align with Brazil and not seek regional partners in order to counterbalance its power.

Table 2
Should Argentina align with or counterbalance Brazil?

Strategy	Frequency	Percentage
Align	1141	77,7
Counterbalance	329	22,3

• Source: Elaborated by the authors.

Second, we can see that Argentinians consider there to be only two major global powers today. When consulted on which are the three most relevant countries in the international arena²⁰ (see Table 3), the United States and China stand out well above the others, with 92% and 73% of the responses respectively. It is noteworthy that Russia was the third most selected country with 33%; we believe this result can be explained by the proximity of the implementation of the survey to events happening in Ukraine and to the annexation of Crimea to the Russian Federation²¹. Finally, only 24% of respondents mentioned Brazil as a current global power, taking fifth place, behind Japan.

Table 3
Current powers (%)

Country	Percentage
USA	92.3
China	73.4
Russia	33.4
Japan	32.7
Brazil	24.7
UK	21.6
Other	6.4
Argentina	6.4
France	4.6
India	4.4

• Source: Elaborated by the authors.

When asked which would be the three most powerful countries in the next ten years²², the answers appear to be more evenly distributed (see Table 4). China retains almost the same percentage (68%) but now is the most chosen country, slightly above the United States, which falls to 61%. For its part, Brazil attains 51% of the preferences, more than double the rate of responses pertaining to the present, making Brazil the third most selected country on the list. This reflects Argentinians' optimism about the future of its neighbor.

Table 4
Future powers (%)

Country	Percentage
China	68.3
USA	60.9
Brazil	51.3
Japan	31.2
Russia	26.8
India	19.7
Argentina	16.2
UK	11.4
Other	11.0
France	3.2

• Source: Elaborated by the authors.

Moving forward with questions that tried to capture Argentine perceptions on international affairs, we note again that there is a strong consensus about Brazil being the most important South American country. Generally, almost 65% of respondents believed that Brazil is the regional leader (Table 5), although this perception is inversely proportional to the perception of Argentina's relevance in the international arena (see Figure 4).

Table 5
Do you think that Brazil is a regional leader?

	Frequency	Percentage
No	523	35
Yes	973	65

• Source: Elaborated by the authors.

Figure 4
Plotted probability of Brazil being seen as a regional leader

Source: Elaborated by the authors using Clarify (Tomz et al., 2003)

To calculate this last probability, we took the marginal probabilities of a logistic regression and plotted them to facilitate their interpretation (see Figure 4). Thus, we see that the probability of Brazil being considered a regional leader falls from 80% to 40% as we move from respondents who believe that Argentina is irrelevant to the international arena (value 0), to those who think it is most relevant (value 10).

Finally, 44% of the respondents appeared to believe that Argentina should hold Brazil up as a model, although for this question the variation of responses among different subgroups was considerable. Those participants who had posited themselves on the left of the ideological spectrum tended to believe to a lesser extent that the Argentinian government should follow Brazil's example (see Figure 5). On the other hand, compared to those who declared support for Kirchner's policies, those with a negative view of the Kirchner administration tended to believe more strongly that Argentina should imitate its neighbor. Like Figure 4, Figure 5 is a visual representation of the marginal probabilities calculated using logistic regression.

Figure 5
Plotted probability of considering Brazil to be a good example

Source: Elaborated by the authors.

This is probably one of the most important findings of this section, as it confirms Russel and Tokatlian's hypothesis, which suggest that the Brazilian image has been historically used in Argentinian domestic debates as an "inverted mirror," featuring Brazil as a rising power and Argentina as a country in decline (Russell and Tokatlian, 2011, p. 258). This does not necessarily mean that people who used to support Kirchner's administrations had a negative view of Brazil, but it does suggest that those who were opponents of Kirchner's government used Brazil's success in order to criticize the government in power.

Framing effects

Analyzing the results of the survey with the differences between each treatment group and the control group in consideration, we can see that framing effects indeed changed participants' perceptions, albeit in a heterogeneous way.

Argentines who received absolute information regarding Brazilian growth, its military spending, and its number of soldiers proved to have perceptions similar to those who received no stimulus (control group). Meanwhile, those who received information comparing Argentina to its neighbor tended toward the idea that Argentina should seek new regional partners to counterbalance Brazilian power (26% vs. 18%) (see Table 6). These findings support the theory that in order to fully grasp what large numbers mean, it is necessary to have a reference point closer to our reality (Tversky and Kahneman, 1981).

Table 6
Difference between treatments and control groups

		Argentina should counterbalance Brazil
Treatment	Control	Absolute	Relative
Average	0.18	0.22	0.26**
Observations	476	524	470

• Source: Elaborated by the authors.
• Note: Two tailed T Test. Statistical significance: * p < 0.05; ** p < 0.01; *** p < 0.001.

As a robustness measure, we compared the treatments, including controls, with individual characteristics through a logistic regression. As we can see in Figure 6, the previous findings hold even after including controls. These variables are: the ideology of the person (a categorical variable ranging from 1 to 10, with10 representing the extreme right), how often the person informs themselves about foreign affairs (ordinal variable, ranging from 0 to 3), a variable about the person's opinion on Argentina's role in the world (categorical variable ranging from 1 to 10, 10 being "very important"), opinion about the current government (categorical variable ranging from 1 to 10, 10 being "very favorable"), family income (ordinal variable), university degree (fixed effects for careers), the person being nationalist or not (dummy variable based on self-declared perception), gender (dummy, 1 being "man"), and a dummy for the university (UBA or Undav).

Figure 6
Treatment effects with control variables

Source: Elaborated by the authors.

A second empirical strategy consisted of analyzing whether the treatment effect was more or less significant for certain subgroups of our sample. What we observe is that among people who usually get little information about international affairs, the relative treatment was 64% more effective than the one we had previously observed. That is, 46% of uninformed people chose to counterbalance Brazilian power after receiving the comparative treatment. In turn, among nationalists this effect was subtly higher than the one for the general population—just 4.6% higher than the 26% associated with the general sample (see Table 7).

Table 7
Treatment effects by subsamples

	Degree of nationalism		Political ideology		Degree of information		Supports kirchners
	Nationalist	Non-nationalist	Left	Right	Very informed	Little informed	No	Yes
Absolute treatment	0.252	0.121	0.176	0.206	0.203	0.172	0.14	0.271
	(1.83)	(0.86)	(1.63)	(0.88)	(1.57)	(1.13)	(1.11)	(1.74)
Relative treatment	0.272*	0.275	0.257*	0.26	0.134	0.467**	0.249	0.277
	(1.99)	(1.88)	(2.32)	(1.12)	(1.04)	(2.98)	(1.92)	(1.78)
Controls	Si	Si	Si	Si	Si	Si	Si	Si
Observations	650	632	1039	243	761	521	753	529

• Source: Elaborated by the authors.
• Note: T test between parentheses. Jacknife Standard Errors. Statistical significance: * p < 0,05; ** p < 0,01; *** p < 0,001.

Thus, this finding reinforces the idea that while some framings can have no effect on public opinion perceptions, others may have an effect that is conditional on or amplified by respondents' characteristics (Knoll, Redlawsk and Sanborn, 2011). Donovan and Jalleh (1999) suggest that the degree of involvement of individuals with regard to the issue addressed may alter their sensitivity toward framing. To summarize the authors' argument, while negative framing would be more effective than positive framing among people with a high degree of involvement, the relationship reverses when respondents' interest is low, making positive framings more effective.

Conclusions

Is public opinion sensitive to framing effects on foreign policy issues? It depends. Drawing on novel experimental data, we arrive at some findings that contribute to an ongoing discussion in the literature and present empirical evidence from emerging countries that until now have received little attention.

As other authors have pointed out (Flachaire and Hollard, 2008; Barabas and Jerit, 2010, p. 226), survey experiments generate effects that are observable only among particular subgroups of our samples and, due to this, cannot be extrapolated to the entire population.

Our findings show that the frequency with which people stay informed about international issues is a determining factor in explaining how sensitive they are to framing effects on foreign affairs. In this sense, the more a person declares that they stay informed, the less sensitive he/she will be to framing effects on foreign policy issues. With this in mind, we might ask ourselves from which source individuals are receiving information about international affairs. The literature suggests that big media plays a gatekeeping role regarding foreign policy issues (Hill, 2003, p. 275) since it is the main channel through which most of the population receives information on themes (Puglisi and Snyder, 2011) with which we do not have personal, daily contact (Soroka, 2003, p. 28). However, in recent years new media and alternative forms of communication have gained space, and with the increasing use of social networks the monopoly of traditional media may be in dispute.

On the other hand, we also find that the more relevance a person gives to his country in the international arena, the more susceptible he/she is to information that shows its weakness compared to other countries. This finding is interesting if we think from Putnam's logic of two-level games (1988). Politicians might try to influence the electorate and gain an advantage in the domestic debate using a nationalist discourse. Building on this idea, we also corroborate Russel and Tokatlian's hypothesis, which suggests that the Argentinian opposition had used the Brazilian image as an "inverted mirror," presenting Brazil as an emerging power and Argentina as a country in decline in order to criticize the previous government (Russel and Tokatlian, 2011, p. 258).

Finally, as we have shown, most studies that have tried to analyze framing effects on foreign policy issues have relied on data collected in developed countries, mainly the United States. Given this reality, we believe that by applying our experiment in a developing country we are contributing an important new case study to a longstanding discussion. However, we also know that cultural differences and specific characteristics of each country may affect how sensitive people are to external affairs (Hermann and Ozkececi-Taner, 2011). For these reasons, in order to draw general conclusions about "Public Opinion," we should replicate this research in countries with different characteristics, isolating each country's specific effects.

APPENDIX I Absolute treatment

"De acuerdo a estadísticas del Banco Mundial, el Producto Bruto Interno de Brasil alcanzó en 2013 la suma de U$S 2.253 mil millones de dólares, habiendo crecido un 340% en los últimos 10 años. A su vez, las Fuerzas Armadas brasileras están compuestas por 327 mil tropas activas y su presupuesto alcanzó en 2013 la suma de U$S 30,3 mil millones."

According to World Bank statistics, the Brazilian Gross Domestic Product in 2013 reached the sum of US $2.253 billion dollars, growing 340% over the last 10 years. The Brazilian Armed Forces are composed of 327,000 active troops and its budget reached the amount of US$30.3 billion in 2013.

APPENDIX II Relative treatment

"De acuerdo a estadísticas del Banco Mundial, el Producto Bruto Interno de Brasil alcanzó en 2013 la suma de U$S 2.253 mil millones de dólares, habiendo crecido un 340% en los últimos 10 años y haciendo con que la economía brasilera sea 5,5 veces más grande que la argentina. A su vez, las Fuerzas Armadas brasileras están compuestas por 327.000 tropas activas y su presupuesto alcanzó en 2013 la suma de U$S 30,3 mil millones. Por su parte, el número de tropas activas en la Argentina es de 86.000 y el presupuesto militar es de U$S 5,2 mil millones, tan solo 15% del brasilero."

According to World Bank statistics, the gross domestic product of Brazil in 2013 reached the sum of US $2.253 billion dollars, growing 340% over the past 10 years and indicating that the Brazilian economy is 5.5 times larger than Argentina's. In turn, the Brazilian Armed Forces are composed of 327,000 active troops, and its budget reached the amount of US $30.3 billion in 2013. Meanwhile, the number of active troops in Argentina is 86,000 and the military budget is US $5.2 billion, only 15% of the Brazilian budget.

References

• Albuquerque, J. G. "As percepções das elites dos países do Cone Sul a respeito da política externa brasileira (1997-1999)". Pesquisa Nupri-USP, 2000.
• Albuquerque, J. G. "As percepções da elite e opinião pública sobre a política externa brasileira (1998-2000)". Pesquisa Nupri-USP, 2001.
• Aldrich, J. H., et al. "Foreign policy and the electoral connection". Annual Review of Political Science, vol. 9, p. 477-502, 2006.
• Ardanaz, M.; Murillo, V.; Pinto, P. "Sensitivity to issue framing on trade policy preferences: evidence from a survey experiment". International Organization, vol. 67, n° 2, p. 411-437, 2013.
• Babbie, E. The basics of social research Independence, KY: Cengage Learning, 2013.
• Bandeira, M. O eixo Argentina-Brasil: o processo de integração da América Latina. Brasília: Editora UnB, 1987.
• Barabas, J.; Jerit, J. "Are survey experiments externally valid?". American Political Science Review, vol. 104, n° 2, p. 226-242, 2010.
• Bernal-Meza, R. "Argentina y Brasil en la política internacional: regionalismo y Mercosur (estrategias, cooperación y factores de tensión)". Revista Brasileira de Política Internacional, vol. 51, n° 2, p. 154-178, 2008.
• Boettcher, W. A.; Cobb, M. D. "Echoes of Vietnam? Casualty framing and public perceptions of success and failure in Iraq". Journal of Conflict Resolution, vol. 50, n° 6, p. 831-854, 2006.
• Chong, D.; Druckman, J. "Framing theory". Annual Review of Political Science, vol. 10, p. 103-126, 2007.
• Consejo Argentino Para Las Relaciones Internacionales. La Opinión Pública Argentina sobre Política Exterior y Defensa 2002 (online), 2002. Obtenido de <http://www.cari.org.ar/pdf/encuesta2002.pdf>. Accedido en: 6/6/2014.
  » http://www.cari.org.ar/pdf/encuesta2002.pdf
• Consejo Argentino Para Las Relaciones Internacionales. La Opinión Pública Argentina sobre Política Exterior y Defensa 2006 (online), 2006. Obtenido de <http://www.cari.org.ar/pdf/encuesta2006.pdf>. Accedido en: 6/6/2014.
  » http://www.cari.org.ar/pdf/encuesta2006.pdf
• Consejo Argentino Para Las Relaciones Internacionales. La Opinión Pública Argentina sobre Política Exterior y Defensa 2010 (online), 2010. Obtenido de <http://www.cari.org.ar/pdf/encuesta2010.pdf>. Accedido en: 6/6/2014.
  » http://www.cari.org.ar/pdf/encuesta2010.pdf
• Consejo Argentino Para Las Relaciones Internacionales. La Opinión Pública Argentina sobre Política Exterior y Defensa 2015 (online), 2015. Obtenido de <http://cari.org.ar/pdf/encuesta2015.pdf>. Accedido en: 24/2/2016.
  » http://cari.org.ar/pdf/encuesta2015.pdf
• Dasgupta, N.; Hunsinger, M. "The opposite of a great truth is also true: when do student samples help versus hurt the scientific study of prejudice?". Psychological Inquiry, vol. 19, n° 2, p. 90-98, 2008.
• Donovan, R. J.; Jalleh, G. "Positive versus negatively framed product attributes: the influence of involvement". Psychology & Marketing, vol. 28, n° 4, p. 215-234, 1999.
• Druckman, J. N. "The implications of framing effects for citizen competence". Political Behavior, vol. 23, n° 3, p. 225-256, 2001.
• Druckman, J. N., et al. "The growth and development of experimental research in political science". American Political Science Review, vol. 100, n° 4, p. 627-635, 2006.
• Druckman, J. N.; Kam, C. D. "Students as experimental participants: a defense of the'narrow data base'". Available at SSRN 1498843, 2009.
• Druckman, J. N.; Kam, C. D. "Students as experimental participants". Cambridge Handbook of Experimental Political Science, vol. 1, p. 41-57, 2011.
• Echegaray, F. "O papel das pesquisas de opinião pública na consolidação da democracia: a experiência latino-americana". Opinião Pública, vol. 7, n° 1, p. 60-74, 2001.
• Faria, C. A. P. "Opinião pública e política externa: insulamento, politização e reforma na produção da política exterior do Brasil". Revista Brasileira de Política Internacional, vol. 51, n° 2, p. 80-97, 2008.
• Flachaire, E.; Hollard, G. "Individual sensitivity to framing effects". Journal of Economic Behavior & Organization, vol. 67, n° 1, p. 296-307, 2008.
• Foyle, D. C. "Leading the public to war? The influence of American public opinion on the Bush administration's decision to go to war in Iraq". International Journal of Public Opinion Research, vol. 16, n° 3, p. 269-294, 2004.
• Gaines, B.; Kuklinski, J.; Quirck, P. "The logic of the survey experiment reexamined". Political Analysis, vol. 15, n° 1, p. 1-20, 2007.
• Gartner, S. S. "The multiple effects of casualties on public support for war: an experimental approach". American Political Science Review, vol. 102, n° 1, p. 95-106, 2008.
• Gelpi, C.; Feaver, P. D.; Reifler, J. Paying the human costs of war: American public opinion and casualties in military conflicts. Princeton: Princeton University Press, 2009.
• Hermann, M. G.; Ozkececi-Taner, B. "The experiment and foreign policy decision making". Cambridge Handbook of Experimental Political Science, New York, p. 430-442, 2011.
• Hill, C. The changing politics of foreign policy Basingstoke: Palgrave Macmillan, 2003.
• Hirst, M. "Las relaciones Argentina-Brasil: de la asimetría al equilibrio". Integración Latinoamericana, vol. 12, p. 35-43, 1987.
• Hiscox, M. J. "Through a glass and darkly: attitudes toward international trade and the curious effects of issue framing". International Organization, vol. 60, n° 3, p. 755-780, 2006.
• Holsti, O. R. "Public opinion and foreign policy: challenges to the Almond-Lippmann consensus Mershon series: research programs and debates". International Studies Quarterly, vol. 36, n° 4, p. 439-466, 1992.
• Howlett, M. "A dialética da opinião pública: efeitos recíprocos da política pública e da opinião pública em sociedades democráticas contemporâneas". Opinião Pública, vol. 6, n° 2, p. 167-186, 2000.
• Hyde, S. D. "The future of field experiments in international relations". The annals of the American Academy of Political and Social Science, vol. 628, n° 1, p. 72-84, 2010.
• Hyman, H.; Sheatsley, P. B. The current status of American public opinion Silver Spring: National Council of Social Studies, 1950.
• Jacobs, L.; Page, B. "Who influences US foreign policy?". American Political Science Review, vol. 99, n° 1, p. 107-123, 2005.
• Kaufman, R. G. "'To balance or to bandwagon?' alignment decisions in 1930s Europe". Security Studies, vol. 1, n° 3, p. 417-447, 1992.
• Knoll, B.; Redlawsk, D.; Sanborn, H. "Framing labels and immigration policy attitudes in the Iowa caucuses". Political Behavior, p. 1-22, 2011.
• Kriner, D. L.; Shen, F. X. "Reassessing american casualty sensitivity the mediating influence of inequality". Journal of Conflict Resolution, p. 1-28, 2013.
• Lentner, H. H. "Public policy and foreign policy: divergences, intersections, exchange". Review of Policy Research, vol. 23, n° 1, p. 169-181, 2006.
• Lippmann, W. Public opinion Edición de dominio público. New Jersey: Transaction Publishers, 1932.
• Lopes, D. B. "A política externa brasileira e a 'circunstância democrática': do silêncio respeitoso à politização ruidosa". Revista Brasileira de Política Internacional, vol. 54, n° 1, p. 67-86, 2011.
• Lopes, D. B.; Faria, C. A. P. "Eleições presidenciais e política externa brasileira". Estudos Internacionais: Revista de Relações Internacionais da PUC Minas, vol. 2, n° 2, p. 139-147, 2014.
• Malamud, A. "A leader without followers? The growing divergence between the regional and global performance of Brazilian foreign policy". Latin American Politics and Society, vol. 53, n° 3, p. 1-24, 2011.
• Mcdermott, R. "Research transparency and data archiving for experiments". Political Science & Politics, vol. 47, n° 1, p. 67-71, 2014.
• Merke, F. "Neither balance nor bandwagon: South American international society meets Brazil's rising power". International Politics, vol. 52, n° 2, p. 178-192, 2015.
• Milani, C. R. S.; Pinheiro, L. "Política externa brasileira: os desafios de sua caracterização como política pública". Contexto Internacional, vol. 35, n° 1, p. 11-41, 2013.
• Mintz, A.; Redd, S. "Framing effects in international relations". Synthese, vol. 135, n° 2, p. 193-213, 2003.
• Morton, R. B.; Williams, K. C. Experimental political science and the study of causality: from nature to the lab. Cambridge: Cambridge University Press, 2010.
• Mouron, F.; Onuki, J. "¿Brasil es un líder en América del Sur? El papel brasileiro a través del concepto de liderazgo situacional". Estudos Internacionais: Revista de Relações Internacionais da PUC Minas, vol. 3, n° 1, p. 9-26, 2015.
• Mueller, J. American foreign policy and public opinion in a new era: eleven propositions. In: Norrander, B.; Wilcox, C. (eds.). Understanding public opinion Washington: CQ Press, p. 49-72, 2002.
• Nelson, T.; Oxley, Z.; Clawson, R. "Toward a psychology of framing effects". Political Behavior, vol. 19, n° 3, p. 221-246, 1997.
• Neuman, W. L.; Robson, K. Basics of social research Pearson, 2004.
• Payne, S. L. B. The art of asking questions: studies in public opinion. Princeton University Press, 1951.
• Puglisi, R.; Snyder, J. M. "Newspaper coverage of political scandals". The Journal of Politics, vol. 73, n° 3, p. 931-950, 2011.
• Rugg, D.; Cantril, H. "The wording of questions in public opinion polls". The Journal of Abnormal and Social Psychology, vol. 37, n° 4, 1942.
• Rugg, D.; Cantril, H. "The wording of questions". Gauging Public Opinion, p. 23-50, 1944.
• Russell, R.; Tokatlian, J. G. Percepciones Argentinas de Brasil: ambivalencias y expectativas, In: Sorj, B.; Fausto, S. (eds.). En Brasil y América del Sur: miradas cruzadas. Buenos Aires: Catálogos, p. 281-316, 2011.
• Schenoni, L. L. "Rise and hegemony: some observations on emerging powers from a South American perspective". Revista Brasileira de Política Internacional, vol. 55, n° 1, p. 31-48, 2012.
• Schroeder, P. "Historical reality vs. neo-realist theory". International Security, vol. 19, n° 1, p. 108-148, 1994.
• Schuman, H.; Presser, S. "Question wording as an independent variable in survey analysis". Sociological Methods & Research, vol. 6, n° 2, p. 151-170, 1977.
• Schuman, H.; Presser, S. Questions and answers in attitude surveys: experiments on question form, wording, and context. London: Sage, 1981.
• Selcher, W. A. "Brazilian-Argentine relations in the 1980s: from wary rivalry to friendly competition". Journal of Inter-American Studies and World Affairs, vol. 27, n° 2, p. 25-53, 1985.
• Sikkink, K.; Wolfson, L. "Las capacidades y la autonomía del Estado en Brasil y la Argentina: un enfoque neoinstitucionalista". Desarrollo Económico, vol. 32, n° 128, p. 543-574, 1993.
• Singer, J. D.; Bremer, S.; Stuckey, J. "Capability distribution, uncertainty, and major power war, 1820-1965". Peace, War and Numbers, vol. 19, 1972.
• Soroka, S. N. "Media, public opinion, and foreign policy". The International Journal of Press/Politics, vol. 8, n° 1, p. 27-48, 2003.
• Tomz, M., et al. "Software for interpreting and presenting statistical results". Journal of Statistical Software, vol. 8, n° 1, p. 1-30, 2003.
• Turgeon, M.; Rennó, L. "Informação política e atitudes sobre gastos governamentais e impostos no Brasil: evidências a partir de um experimento de opinião pública". Opinião Pública, vol. 16, n° 1, p. 143-159, 2010.
• Tversky, A; Kahneman, D. "The framing of decisions and the psychology of choice". Science, vol. 211, n° 4481, p. 453-458, 1981.
• Waltz, K. N. "Structural realism after the Cold War". International Security, vol. 25, n° 1, p. 5-41, 2000.

• 1
  This article was previously presented at the ISA-Flacso Joint General Conference (2014), the IX Summit of the Brazilian Political Science Association (2014), and the 2014 Gesis Summer School in Survey Methodology. The authors would like to thank Lorena Barberia, Aldana Romano, and Fiorella Wernicke for their comments, suggestions, and research assistance.
• 2
  Index composed of six variables: (a) Population, (b) Urban population, (c) Iron and Steel production, (d) Energy consumption, (e) Number of soldiers, (f) Military budget (Singer, Bremer and Stuckey, 1972).
• 3
  Along with Latinobarometer research (<http://www.latinobarometro.org/>) and the work focused on elites carried out by the Nupri-USP (Albuquerque, 2000, 2, 2001), this is the only initiative that consistently conducts traditional surveys in order to capture Argentine perceptions regarding international affairs.
• 4
  2,408 people and 192 opinion leaders interviewed between November 15th and December 15, 2001.
• 5
  1,616 people and 175 opinion leaders interviewed between October 8 and October 23, 2006.
• 6
  1.606 people and 170 opinion leaders interviewed between November 17th and November 28, 2010.
• 7
  1.600 people and 100 opinion leaders interviewed between July and August of 2015.
• 8
  A classic example of this type of research—one that also takes into account the question order effect—is the work carried out by Hyman and Sheatsley (1950). Through an experiment applied in 1948, the authors show that when first exposed to the possibility that the USSR allowed American journalists to enter the Soviet Union, the interviewees showed themselves to be more receptive to the belief that the United States should do the same with their communist counterparts. On the other hand, those who first were requested to express their opinion about the possibility of Soviet journalists working in the US presented lower acceptance rates when compared to the other group.
• 9
  For a summary of all of the articles published until 2003 addressing the issue of framing effects on US foreign policy, see Mintz and Redd (2003).
• 10
  A pre-test was applied in early December of 2013 to ensure that the questions were clear enough for all participants. 100 students answered the questionnaire, and the question that tried to capture political affinity in relation to the government had to be modified, since in its original format it generated a rejection superior to 10%.
• 11
  Of those 1,530 respondents, 13 were excluded from the final sample because they chose not to submit the questionnaire.
• 12
  Census applied in 2011 <http://www.uba.ar/institucional/censos/Estudiantes2011/estudiantes%202011.pdf>.
• 13
  If we were to only consider the indicators "academic reputation" and "reputation among employers” from the QS ranking, UBA would be the best-positioned university in Latin America.
• 14
  They are the first in their families to attend college.
• 15
  Questionnaires were applied by three research assistants supervised by one of the authors. We would like to emphasize that their roles were simply to make sure each person received the treatment to which he or she was assigned, thus avoiding any problem of contagion.
• 16
  Respondents placed themselves on a scale of 1 to 10, where 1 was extreme left and 10 extreme right.
• 17
  Respondents placed themselves on a scale of 1 to 10, where 1 meant that they totally disagreed with the government's policies and 10 meant that they totally agreed.
• 18
  Respondents placed Argentina on a scale of 1 to 10 in terms of its relevance in the international arena, with 1 being of little relevance and 10 being very relevant.
• 19
  Respondents had to answer how often they informed themselves about international affairs, having the options: daily, 2 to 3 times a week, once a week or not usually.
• 20
  This was applied as a closed question with ten options presented in alphabetical order. Participants had to choose three of them.
• 21
  Crimea was annexed by the Russian Federation on March 18, 2014, two weeks before the survey was applied.
• 22
  It was applied as a closed question, with ten options presented in alphabetical order. Participants had to choose three of them.

Publication Dates

• Publication in this collection
  Apr 2016

History

• Received
  Sept 2014
• Accepted
  Dec 2015