Evaluating the impact of the Cartão Mais Infância Ceará program on school performance^♦

Avaliação de impacto do programa Cartão Mais Infância Ceará sobre a performance escolar

Abstract

The primary objective of this paper is to evaluate the impact of the Cartão Mais Infância Ceará (CMIC) program on students’ school performance in the Portuguese and Mathematics exams of the Permanent Assessment System for Basic Education in Ceará (SPAECE). The analysis focuses on students from families benefiting from the program between the years 2018 and 2019. The method employed is the difference-in-differences model, coupled with the propensity score approach following the design proposed by Blundell and Dias (2009), which is suitable for repeated cross-sections data structures. The findings reveal that being part of a CMIC family is, on average, associated with a performance improvement of approximately 11.63% and 11.52% in the SPAECE Portuguese and Mathematics exams, respectively, compared to students from non-participating families.

Keywords:
Education; School performance; Transfer income; Impact evaluation; Conditional cash transfers

Resumo

Este artigo tem o objetivo de avaliar o impacto do programa Cartão Mais Infância Ceará (CMIC) no desempenho escolar, nos exames do Sistema Permanente de Avaliação da Educação Básica do Ceará (SPAECE) de português e matemática de alunos pertencentes a famílias beneficiárias desse programa entre os anos de 2018 e 2019. Para tanto, utiliza-se o método de Diferenças-em-Diferenças em conjunto com o escore de propensão nos moldes propostos por Blundell e Dias (2009), adequado para estruturas de dados de cross-sections repetidos. Os resultados indicam que pertencer a uma família beneficiária do CMIC está associado, em média, a um desempenho aproximadamente de 11,63% e 11,52% maior nos exames SPAECE de português e matemática, respectivamente, em relação aos alunos que não pertencem a famílias atingidas pelo programa.

Palavras-chave:
Educação; Desempenho escolar; Transferência de renda; Avaliação de impacto

1. Introduction

The Cartão Mais Infância Ceará (CMIC) program encompasses a range of initiatives aimed at promoting the development of children in early childhood. Officially established as state policy through a legislative act passed in March 2019, the primary objective of the program is to reduce child poverty.

Targeting families with children aged 0 to 5 years and 11 months who are in situations of extreme social vulnerability, CMIC includes a transfer income component among its various initiatives. In this regard, eligible families meeting the necessary criteria receive a monthly cash transfer of 100 reais through the Cartão Mais Infância program, with approximately 150,000 beneficiary families in 2022.

The educational model in the state of Ceará has become a benchmark in terms of quality and management. Despite the state still being among the less developed federative units in the country, it has achieved notable results in exams measuring the quality of basic education. In a recent study by the World Bank, Loureiro et al (2020) assert that Ceará has a basic education model capable of reducing learning poverty and that can be replicated in other states in the country based on certain pillars of management and efficiency. According to this study, education in Ceará recorded the highest increase in the national index of educational quality in elementary education between 2005 and 2017.

In this context, this article aims to investigate the factors contributing to the performance of education in the state of Ceará. To do so, this article evaluates the impact of CMIC program during the years 2018 and 2019 on the academic performance of students from families receiving financial resources from the program, even though it does not require an educational counterpart. The link between the financial resources of CMIC and the academic performance of children is that families can better provide for their children, thus leading to an improvement in their academic performance. For example, Rocha (2016) argues that educational public policies on school meals are crucial for basic education. Similarly, Gomes (2009) emphasizes the importance of attention to combating malnutrition by policymakers, as its effects can impact not only individuals’ health and well-being but also learning and the accumulation of human capital.

The primary objective of this paper is to evaluate the impact of the Cartão Mais Infância Ceará (CMIC) program on students’ school performance in the Portuguese and Mathematics exams of the Permanent Assessment System for Basic Education in Ceará (SPAECE). The analysis focuses on students from families benefiting from the program between the years 2018 and 2019. It is important to highlight that the student treated does not refer to a child aged between 0 to 5 years and 11 months, a direct beneficiary of the CMIC, as there is no availability of information on the academic performance of children in this age group, not even the SPAECE-alpha. Thus, the units treated refer to students from these families who are between the second year of elementary school and the third year of high school.

The Performance will be measured through mathematics and Portuguese language grades across all levels of education. The data panels utilized in this study were derived from the intersection of data from the school census obtained from the Permanent Assessment System for Basic Education in Ceará (SPAECE) and information from the CMIC, with the individual student serving as the unit of analysis.

Several statistical approaches can be used to assess program impacts. The most common ones are the Propensity Score Matching (PSM) method, Differences-in-Differences, Regression Discontinuity and the Instrumental Variables method. In evaluating the CMIC program, this article employs the Difference-in-Differences (DID) model along with the PSM approach proposed by Blundell and Dias (2009) and Villa (2016). In this case, a data panel is fixed for the years 2018 and 2019. The use of the combination of these two approaches is justified by the following characteristics: the Differences-in-Differences method, compared to PSM, assumes that unobserved heterogeneity is present in program participation, but such factors are time-invariant. The combination of these two approaches can address the selection bias problem by matching units in a common support.

The other two approaches that could also assess the CMIC program are similar in that they introduce an exogenous variable strongly correlated with program participation. However, in the case of Instrumental Variables, the instruments must be carefully selected. If the instruments are weak, the selection bias can potentially worsen. In the case of Regression Discontinuity, it is required that units below and above the cutoff point be quite similar. In the case of the CMIC program, among those eligible to participate, selection is made considering families with lower per capita incomes. Therefore, in the vicinity of the cutoff point, families could be very heterogeneous.

The findings reveal a significant and positive impact on the school performance of students in Portuguese and mathematics who belong to CMIC beneficiary families compared to their non-beneficiary counterparts. This assessment encompasses all stages of education in which these exams are administered. Regarding the heterogeneity of the effect of the CMIC, it can be said that there is a negative differential concerning race in the mathematics exam, meaning that the program’s effect on white students is lower than that on students of other races. It is worth noting that the results remained robust when we estimated the model with variables solely at the school level, solely at the student level, and various pairs of randomly selected samples. Additionally, it was found that the conducted matching exhibited adequate predictive capacity.

However, the positive impact result obtained in this article conflicts with those of Cireno, Silva, and Proença (2013), Camargo and Pazello (2014), and Habenschus (2020) for Brazil. On the other hand, it aligns with the findings of some international articles.

In addition to this introduction, Section 2 provides a literature review on the subject in question. Section 3 provides a detailed description of the CMIC program. Section 4 delineates the methodology, presenting the data sources and the econometric strategy employed to evaluate the program’s impact. Section 5 presents and examines the results, while Section 6 offers concluding remarks.

2. Literature Review

Numerous articles in the Brazil have sought to evaluate this public policy of conditional cash transfer in relation to school dropout rates (Vieira, 2020), expansion of vaccination coverage (Kern et al., 2018), and nutritional well-being of beneficiaries (Camelo et al., 2009).

Cireno, Silva, and Proença (2013) utilized a database from the Brazilian Ministry of Education (MEC) and the Ministry of Social Development and Fight Against Hunger (MDS) to analyze the academic performance and educational trajectory of students in the 5th and 9th grades who were beneficiaries of the program. The program can impact student performance through the conditionality of school attendance and the improvement of family income. In their analysis, the authors employed explanatory variables in a difference-in-differences model, including average performance in the Brazilian national assessment of educational achievement (ANRES), mostly known as Prova Brasil, dropout and school failure rates, and age-grade distortion.

The results indicated lower academic performance, as measured by the Brazilian national assessment of educational achievement (Prova Brasil), among program participants compared to non-participants. However, this performance gap diminished from the 5th to the 9th grade, suggesting that the program contributes to reducing educational inequalities. Nonetheless, the financial impact on beneficiary students persists, as they still experience an income deficit compared to non-participants, despite the benefits provided by the program.

Camargo and Pazello (2014) employed a logit model to examine the impact of the proportion of students benefiting from the Bolsa Família program on educational indicators in schools across Brazil in 2004. The estimated marginal effects indicated that an increase in the proportion of beneficiary students reduced dropout rates. However, like the findings of Cireno, Silva, and Proença (2013), the estimates revealed a negative relationship between participation in the program and school performance.

Habenschus (2020) also investigated the impact of the Bolsa Família program on the academic performance of students who had experienced educational delays in the state of Ceará. Additionally, the study evaluated the program’s influence on migration in the semiarid region of the state. Using a regression discontinuity design (RDD), he noticed that the results demonstrated a significant impact of the program in reducing the migration of beneficiary children. However, the impact on improved grades in proficiency exams was not statistically significant.

As evident from the studies mentioned earlier, the impact of direct cash transfer programs solely focused on income has been primarily observed in the reduction of dropout rates. This indicates that interventions aimed at enhancing school performance may necessitate an extended timeframe for their effects to materialize or may necessitate the development and enhancement of program components that are directly aligned with academic performance.

The Permanent Assessment System for Basic Education in Ceará (SPAECE), developed by the Ceará State Department of Education (SEDUC), serves as an instrument for evaluating school performance. Since 1992, Portuguese language and mathematics exams have been administered to 5th and 9th-grade classes in Elementary School (ES), as well as to 3rd-year students in High School (HS). Additionally, since 2007, a version of the Portuguese language exam, known as SPAECE-alfa,¹ has been applied to 2nd-grade elementary school students.

The SPAECE exams are frequently employed as performances measures in research exploring the determinants of school performance. For instance, in a study on peer effects, Cruz (2022) investigates the influence of the presence and academic performance of students from private schools on the school performance of other students in high school, specifically within the same class at Adriano Nobre School in Itapajé, Ceará. The findings indicate a positive peer effect, as measured by the average proficiency of students from private schools, on the mathematics proficiency of other students.

In another study, Souza, Ciríaco, and Soares (2021) examine the impact of the Programa Jovem do Futuro (Youth of the Future Program) on performance in state schools in Ceará. They employ the propensity score for multiple treatments weighted by generalized boosted models (GBM) technique. The results indicate that a combination of mixed methodologies within the program, encompassing both essential and optional components, has more significant positive effects than more basic combinations. Moreover, the effect is more pronounced in the Portuguese language grades compared to mathematics.

At the international level, there are also several studies that have analyzed the impacts of income transfer programs on the academic performance of children. As an example, Maynard and Murnane (1979) examined the negative income tax program’s effects on the performance of children in fourth through tenth grade in the American economy. Using a differences-in-differences model, children were divided into two groups: fourth through sixth grades and seventh through tenth grades. The results indicated that the program had a positive effect on the academic performance of children between the fourth and sixth grades. However, for the second group, there was no difference in performance between the control and treated groups.

In 1997, Mexico implemented PROGRESA (Programa de Educación, Salud y Alimentación), now called Oportunidades. This program is based on income tax subsidies for poor families and is considered one of the largest randomized interventions ever established by a country. Its goal was to pursue a number of outcomes, including malnutrition, high infant mortality, high fertility, and school attendance. See Khandker, Koolwal, and Samad (2010). Using data from this program, Todd and Wolpin (2006) conducted an ex-ante evaluation of PROGRESA. They found that girls between 12 and 15 years of age increased their schooling by 8.3 percentage points, compared to the actual experimental increase of 11.3 percentage points. For boys, the predicted and experimental estimates were 2.8 and 2.1 percentage points, respectively.

Since the 1990s, Latin American countries have also implemented direct income transfer programs. In this context, Garcia and Hill (2010) investigated the effects of these types of programs on student academic performance in Peru. They used the propensity score matching method to estimate the impact of the program on academic performance. The estimates indicated a positive effect on academic performance for children between 7 and 12 years old in rural areas. However, for adolescents, the program showed no impact. Estimates for rural adolescents indicated that program participants had lower performance than the control group.

For the Ecuadorian income transfer program, Paxson and Schady (2010) investigated the impact of this program on early childhood. The results indicated that income transfer modestly facilitated the physical and socioemotional development of these children.

On the other hand, Baird et al. (2014), using data from 78 reports covering 37 different studies, found that cash transfer programs had positive effects on enrollment numbers and school attendance. However, the effect on academic performance was negligible. In a similar vein, Snilstveil et al. (2015) conducted a synthesis of studies between 1990 and June 2015, considering low- and middle-income countries. Their results also showed positive effects on enrollment numbers and school attendance but insignificant effects on academic performance.

3. The CMIC Program

The Government of the State of Ceará, with the aim of providing a comprehensive framework for child development, established the “Cartão Mais Infância Ceará” (CMIC) program in 2015. This program, created through state legislation, was formally institutionalized as a public policy of the state in 2021. The CMIC program, akin to the Bolsa Família Program (PBF), includes the direct transfer of income to vulnerable families with children in early childhood (0-6 years), with an eligibility criterion of a per capita income of R$ 89.00.

One of the primary objectives of the CMIC program is the direct transfer of income through the “Cartão Mais Infância.” The financial benefit currently stands at R$ 100, targeting families in situations of vulnerability with children in early childhood. As per Decree No. 33,905 of January 27, 2021, families are eligible for this benefit if they have a per capita income of R$ 89.00 and have children aged 0 to 5 years and 11 months. Families that meet these requirements are selected to participate in this program according to the following criteria: I - urban households without piped water in at least one room; II - inappropriate construction materials for the walls of the household (wattle and daub, thatch, reclaimed wood, or other materials); III - absence of a bathroom or toilet in the household or property; IV - makeshift households, consisting of spaces precariously adapted by families for habitation, which may be in private areas such as abandoned buildings or houses, constructions, rural campgrounds, or public areas such as tents and shelters; V - collective households, consisting of spaces where families or individuals reside and are subject to administrative rules, such as shelters, hostels, and other similar accommodations; VI - lower per capita income; VII - families with a higher number of children up to 12 years old in their family composition.

The CMIC program is not limited to direct income transfers. It has partnered with the state government, secretariats, and municipalities to implement a range of public policies. The program consists of four key pillars: the first pillar of the program is “Time to Be Born,” which aims to reduce infant mortality through comprehensive care for both the child and the expectant mother; the second pillar, “Time to Grow,” complements the first and seeks to assist both the mother and the child in growing up healthily. To do so, healthcare professionals and educators are allocated to provide the child with access to nutritional and socioemotional resources for full development; the third foundation of the program is “Time to Play,” which focuses on leisure and entertainment activities for children. The state government allocates resources to bring games and playful activities to children in parks, schools, and theaters. This phase also includes the provision of cultural services and libraries; finally, “Time to Learn” is the fourth pillar of this program. This pillar aims to offer quality and equitable early childhood education to children in Ceará. This project includes modern and welcoming schools, teacher training and development, and the free provision of school materials for the children.

The CMIC program is designed to be a comprehensive approach to the integral development of children in their early years. Beyond being a simple income transfer program, it addresses basic needs and provides tools for children to thrive. It is a project that demonstrates the commitment of the state and municipal governments to support children from birth to the early stages of literacy through the allocation of resources and efforts.

4. Methodology

4.1. Data Source and Description

In order to provide a comprehensive depiction of the variables employed in the impact assessment of CMIC on academic achievement during the 2018-2019 timeframe, Chart 1 delineates the characterization of outcome variables - encompassing measurements of academic performance across different educational stages - alongside the treatment category, denoting students enrolled in the program within schools located in the state of Ceará. Additionally, the analysis incorporates covariates as part of its methodology.

To construct the necessary database for the evaluation, data from the school census, SPAECE, (provided by SEDUC), and information from the Cartão Mais Infância Ceará program (Mais Infância Card), provided by the Secretariat of Social Protection, Justice, Citizenship, Women, and Human Rights of the Government of the State of Ceará (SPS) were integrated. The students’ National Institute of Educational Studies and Investigations (INEP) code was used to align the census and SPAECE data. For cross-referencing this information with that of the Mais Infância Card, the Brazilian Social Security Number (SSN-known as NIS in Portuguese) of the student, their name, and their mother’s name, to address homonym issues were utilized, along with the mother’s Individual Taxpayer Registration Number (ITIN) which is called CPF in Brazilian Portuguese.

While over 2,500 students were identified in the census using this dataset, the number was reduced to approximately 115 students when searching for those in the treatment group who belonged to the educational stages undergoing SPAECE assessment. The resulting panels consisted of approximately 747,202 and 582,653 individuals. In this study, the data panels used were obtained from the Portuguese language and mathematics exams between the years 2018-2019.

The vector X of covariates was selected based on a literature review of relevant explanatory variables that influence school performance. Age, sex, race, and Bolsa Familia variables were referenced from works by Rodrigues et al. (2020), Melo and Suzuki (2021), Benevides and Soares (2020), and Park et al. (2020). Variables related to physical infrastructure, geography, and administrative dependence were referenced by Hanushek (2010), Rodrigues et al. (2020) and Benevides and Soares (2020). It would be important to include family composition variables, parents’ educational level, as well as their occupational situations in the model, however, these variables are not available for the sample used

Tables 1 and 2 present mean statistics and standard errors for the Portuguese language and mathematics panels during the study period. The first column of Table 1 shows that, on average, the natural logarithm of the Portuguese language exam score is approximately 5.45. Regarding student profiles, approximately 50% are male, only 14% are white, 88% belong to families benefiting from the Bolsa Família program, and they are 13 years old in public schools in Ceará. The low average age can be attributed to the inclusion of second-grade students participating in SPAECE-Alfa in the Portuguese language panel. Furthermore, approximately 93% of schools have internet access, 58% have a library, and 61.5% have a computer lab, while less than 1% have access to the public sewage network.

Chart 1:
Data Sources and Description

Variable	Description/References	Source
Dependent
School Performance (Y)	Student grades in Portuguese (PL) and mathematics (MATH) exams within the scope of SPAECE for the 2nd, 5th, and 9th grades of Elementary School (ES) and the 3rd grade of High School (HS). These exams are administered annually by the Ceará State Department of Education (SEDUC).	SEDUC
Treatment
Mais Infância (T )	Binary variable that takes the value 1 if the student belongs to a beneficiary family of the Mais Infância Card, and 0 otherwise.	Developed by the authors using SPS data
Intervention Dummy $(D_{2019})$	Dichotomous variable that identifies the start of the intervention, i.e., the initiation of the Mais Infância Card. As CMIC became officially established after a state law passed in March 2019, this variable takes the value 1 if the period in the sample refers to the year 2019, and 0 otherwise.	Developed by the authors
$D_{2019}\times T$	Product of the variable Mais Infância and the dummy variable that identifies the start of the intervention. Therefore, it is also a binary variable that takes the value 1 if the student belongs to a beneficiary family of CMIC and the intervention has already started, and 0 otherwise. The coefficient of this variable will identify the impact of CMIC on school performance in Portuguese and mathematics exams.	Developed by the authors
Covariates (X)
Age	Age of the student.	School Census
Gender	Binary variable that takes the value 1 if the student is male, and 0 otherwise. Park et al (2020) and Melo and Suzuki (2021).	School Census
Race	Binary variable that takes the value 1 if the student self-identifies as white, and 0 otherwise.	School Census
Bolsa Família	Proxy for family income. It is a dichotomous variable that takes the value 1 if the student’s family is a beneficiary of the program, and 0 otherwise.	SEDUC
Computer lab	Binary variable that takes the value 1 if the school has a computer lab, and 0 otherwise.	School Census
Library	Binary variable that takes the value 1 if the school has a library, and 0 otherwise.	School Census
Internet access	Binary variable that takes the value 1 if the school has internet access, and 0 otherwise.	School Census
Sewage network	The binary variable takes the value 1 if the school has access to the public sewage network, and 0 otherwise.	School Census
Location	The dichotomous variable takes the value 1 if the school is located in an urban area, and 0 if it is in a rural area. It is a variable related to a higher incidence of child labor, poverty of educational resources, school infrastructure, and teacher quality.	School Census
State-owned	Dummy variable that takes the value 1 if the school belongs to the state education system, and 0 otherwise. In this case, the other considered systems are municipal and federal, as private schools were excluded from the sample.	School Census

• Source: Developed by the authors

Lastly, on average, only 24% of schools belong to the state public education network of Ceará, and 78% are located in urban areas.

The third and fourth columns of Table 1 provide the same information but for the year 2018 and subsamples of the panel referring to the treatment (T) and control (C), groups. The treatment group (T) includes students from families benefiting from the CMIC, while the control group (C) consists of students who do not participate in the program. The same analysis is presented in the fourth and fifth columns but for the year 2019. Noticeable differences can be observed in most of the variables included in the panel. For instance, in 2019, there is an approximately 32.8 percentage point difference in the sex variable between the control and treatment groups. These differences may be attributed to the fact that the sample primarily consists of the control group, as only a few students identified in the CMIC took the SPAECE exam in 2018 and 2019.

Table 1
Descriptive Statistics of the Portuguese Language Exam

Variables	Overall	2018		2019			Mean difference			$\left(a\right)-\left(b\right)$
		T	C	T	C		$T_{18}-C_{18}$	$T_{19-18}^{\left(a\right)}$	$C_{19-18}^{\left(b\right)}$
$\ln \left(PL\right)$	5.452	5.280	5.437	5.525	5.467		-0.157**	0.246*	0.0301*	0.216
	0.275	0.326	0.278	0.224	0.272
Gender	0.502	0.185	0.502	0.174	0.502		-0.316*	-0.011	0.001	-0.012
	0.500	0.396	0.500	0.381	0.500
State-owned	0.248	0.333	0.249	0.783	0.247		0.084	0.449*	-0.002**	0.452
	0.432	0.480	0.432	0.414	0.431
Race	0.138	0.111	0.139	0.113	0.137		-0.028	0.002	-0.002**	0.004
	0.345	0.320	0.346	0.318	0.344
Location	0.782	0.667	0.777	0.896	0.786		-0.111	0.229**	0.008*	0.221
	0.413	0.480	0.416	0.307	0.410
Age	12.865	12.815	12.883	19.565	12.844		-0.069	6.750*	-0.039*	6.790
	4.137	5.167	4.156	6.209	4.114
Bolsa Família	0.880	0.815	0.881	0.983	0.880		-0.066	0.168**	-0.002**	0.169
	0.325	0.396	0.324	0.131	0.326
Internet access	0.930	0.889	0.913	0.957	0.948		-0.024	0.068	0.035*	0.033
	0.254	0.320	0.282	0.205	0.222
Library	0.579	0.593	0.560	0.826	0.599		0.033	0.233**	0.039*	0.194
	0.494	0.501	0.496	0.381	0.490
Computer lab	0.615	0.667	0.655	0.861	0.574		0.012	0.194**	-0.081*	0.275
	0.487	0.480	0.475	0.348	0.494
Sewage network	0.009	0.000	0.008	0.009	0.011		-0.008*	0.009	0.003*	0.005
	0.096	0.000	0.087	0.093	0.104
Nº Obs	747,707	27	375,762	115	371,803

• Source: Developed by the authors. Note: 1. T denotes that the statistic refers to the treatment group, while C denotes the control group. 2. Standard errors are reported below the means. 3. *, **, *** denote statistical significance at the 1%, 5%, and 10% levels, respectively.

In the sixth column, the difference between the average results of the treatment group in 2019 and 2018 is calculated. The average score in the Portuguese language exam increased by approximately 0.25% during this period. When conducting the same analysis for the control group in the seventh column, it can be seen that the average growth in exam scores was only 0.03% for this group. The last column represents the difference between the sixth and seventh columns, providing an initial approximation of the expected impact of the program on school performance. As the difference for the natural logarithm of the Portuguese exam score $T_{19-18}-C_{19-18}$ is positive, it is expected that the CMIC has had a positive impact on performance in this exam. The next step involves the difference-in-differences (DID) model to determine the significance of this result.

A similar pattern is observed in the data panel for the mathematics exam, as shown in Table 2. The average natural logarithm of the mathematics exam score is approximately 5.51. The sample consists of students who are approximately 50% male, with only 14.5% identifying as white. The average age is around 14 years old, excluding students from the 2nd year of elementary school, as the mathematics exam is not administered to them. Additionally, 87.8% of the students belong to families receiving the conditional cash transfer from the Bolsa Família program.

Table 2
Descriptive Statistics of the Mathematics Exam

Variables	Overall	2018		2019			Mean difference			$\left(a\right)-\left(b\right)$
		T	C	T	C		$T_{18}-C_{18}$	$T_{19-18}^{\left(a\right)}$	$C_{19-18}^{\left(b\right)}$
$\ln \left(MATH\right)$	5.519	5.340	5.513	5.530	5.525		-0.173*	0.190*	0.013*	0.178
	0.230	0.235	0.232	0.199	0.228
Gender	0.499	0.211	0.498	0.170	0.499		-0.288*	-0.041	0.001*	-0.041
	0.500	0.419	0.500	0.377	0.500
State-owned	0.328	0.526	0.333	0.840	0.322		0.193***	0.313**	-0.011*	0.325
	0.469	0.513	0.471	0.369	0.467
Race	0.145	0.105	0.144	0.123	0.146		-0.039	0.017	0.001	0.016
	0.352	0.315	0.352	0.330	0.353
Location	0.805	0.737	0.802	0.906	0.808		-0.065	0.169	0.006*	0.163
	0.397	0.452	0.399	0.294	0.394
Age	14.504	15.368	14.556	20.557	14.448		0.813	5.188*	-0.108*	5.296
	3.432	4.633	3.498	5.399	3.358
Bolsa família	0.878	0.842	0.878	0.981	0.877		-0.036	0.139	-0.001	0.140
	0.328	0.375	0.327	0.137	0.328
Internet access	0.939	1.000	0.924	0.962	0.955		0.076*	-0.038**	0.031*	-0.069
	0.240	0.000	0.265	0.192	0.208
Library	0.618	0.684	0.589	0.868	0.648		0.095	0.184	0.059*	0.125
	0.486	0.478	0.492	0.340	0.478
Computer lab	0.662	0.790	0.698	0.906	0.624		0.091	0.116	-0.074*	0.190
	0.473	0.419	0.459	0.294	0.484
Sewage network	0.010	0.000	0.009	0.009	0.010		-0.009*	0.009	0.001*	0.008
	0.097	0.000	0.095	0.097	0.100
Nº Obs	582,653	19	299,203	106	283,325

• Source: Developed by the authors. Note: 1. T denotes that the statistic refers to the treatment group, while C denotes the control group. 2. Standard errors are reported below the means. 3. *, **, *** denote statistical significance at the 1%, 5%, and 10% levels, respectively.

When examining the subsamples of the treatment and control groups for the years 2018 and 2019, Table 2 reveals significant differences between these groups. The mean difference in the natural logarithm of mathematics performance between 2019 and 2018 for the treatment group indicates a positive growth of approximately 0.19%. Similarly, the control group also experienced a positive growth, but of only 0.01%. Thus, the difference $T_{19-18}-C_{19-18}$ is positive, suggesting a positive impact of the CMIC program on mathematics performance.

Considering that many of the covariates used in this study have been extensively employed in previous research, one can speculate about their expected signs in the estimation. It is anticipated that the dummies for sex and race will have a positive and significant sign, indicating that male and white students have a performance advantage compared to the reference categories. Positive signs are also expected for dummies related to school infrastructure, as access to computer labs, libraries, internet access, and public sewage networks contribute to a conducive learning environment. In terms of location, a positive sign is expected, as schools in rural areas are typically associated with poorer infrastructure and less qualified teachers. The expected impact of schools belonging to the state network, compared to municipal and federal schools, on school performance is still subject to debate.

4.2. Difference-in-Differences Model (DID)

The primary objective of this paper is to evaluate the influence of the CMIC program on the academic achievement of students belonging to participating families. More specifically, we will compare the performance of these students in mathematics and Portuguese language exams with that of their peers who do not receive benefits from the program.

To achieve this, two groups will be formed: a control group consisting of students who do not participate in the program, and a treatment group comprising students from eligible/beneficiary families. The SPAECE exam scores of these groups in the year 2018, ² before the program’s initiation, will be compared with their scores in the year 2019. It is important to note that the CMIC program began in March 2019.

Since it is not possible to longitudinally track the performance of individual students annually through SPAECE,³ this study adopts a repeated cross-section data structure. This situation arises when the research sample belongs to the same population before and after the intervention, but changes in the composition of the treatment and control groups before and after the intervention can confound the policy effect. To address this issue, the difference-in-differences (DID) model is employed, along with the propensity score approach proposed by Blundell and Dias (2009) and Villa (2016).

In this approach, a vector of covariates X at the student and school levels, including gender, type of school (state, municipal, federal), race, location, age, family allowance, internet access, library availability, computer lab, and sewage access, can be used to calculate the propensity score and kernel weights following the method outlined by Heckman, Ichimura, and Todd (1997, 1998). The propensity score ($p_i$) for both groups⁴ is given by:

$p_i=E\left(Z_i=1|X_i\right)$ (1)

where $Z_i=1$ denotes the unit that undergoes the treatment. Kernel matching is defined based on the propensity score, considering the covariates and the kernel weights are calculated as follows:

$w_i=\frac{K\left(\frac{p_i-p_j}{h_n}\right)}{{{\displaystyle \sum }}_j^{}K\left(\frac{p_i-p_j}{h_n}\right)}$ (2)

where $K(.)$ is a kernel function and $h_n$ represents the selected bandwidth.⁵ Following the approach of Blundell and Dias (2009), the weights obtained through equation (2) are incorporated into the original DID estimator to obtain the DID⁶ treatment effect as shown in equation (3)

$DID=\left\{E\left(Y_{it=1}|D_{it=1}=\mathrm{1,}{Z}_i=1\right)-{\omega }_{it=1}^c\times E\left(Y_{it=1}|D_{it=1}=\mathrm{0,}{Z}_i=0\right)\right\}-{\omega }_{it=0}^t\times \left\{E\left(Y_{it=0}|D_{it=0}=\mathrm{0,}{Z}_i=1\right)-{\omega }_{it=0}^c\times E\left(Y_{it=0}|D_{it=0}=\mathrm{0,}{Z}_i=0\right)\right\}$ (3)

where $t=0$ refers to the baseline year (2018), $t=1$ represents the follow-up year (2019) e $D_{it}$ and is the indicator variable for the intervention. Additionally, ${\omega }_{it=0}^c$ and ${\omega }_{it=1}^c$ denotes the kernel weights for the control group in the years 2018 and 2019, respectively, while ${\omega }_{it=0}^t$ refers to the kernel weight for the treatment group in the year 2018.⁷ The estimated standard errors were clustered at the school level, and Tables 4 and 5 in the appendix present the t-tests for assessing the balance in the means differences with the weighted covariates between the control and treated groups in the initial period.

Therefore, one approach to estimate equation (3) is to apply the econometric specification of the difference-in-differences (DID) approach, as expressed in equation (4):

$\ln \left(Y_{it}\right)={\beta }_0+{\beta }_1\times T_i+{\beta }_2\times D_{2019}+{\beta }_3\times T_i\times D_{2019}+\sum _{i=1}^k{\gamma }_i{X}_{it}+{ϵ}_{it}$ (4)

where the dependent variable is the logarithm of the grade for the i-th (umpteenth) student in period t. All the variables in this model are defined in Figure 1, and their parameter estimation is conducted using weighted least squares, with the weight matrix constructed based on equation (2). Without considering the covariates X, the expected value in equation (4) yields the following results:

${\widehat{\beta }}_0$: the mean grade of the control group before the intervention period;

${\widehat{\beta }}_0+{\widehat{\beta }}_2$: the mean grade of the control group after the intervention period;

Difference: $({\widehat{\beta }}_0+{\widehat{\beta }}_2)-{\widehat{\beta }}_0={\widehat{\beta }}_2$. The difference in mean grades after and before the intervention period of the control group;

${\widehat{\beta }}_0+{\widehat{\beta }}_1$: the mean grade of the treated group in the period before the intervention period;

${\widehat{\beta }}_0+{\widehat{\beta }}_1+{\widehat{\beta }}_2+{\widehat{\beta }}_3$: the mean grade of the treated group after the intervention period;

Difference: $({\widehat{\beta }}_0+{\widehat{\beta }}_1+{\widehat{\beta }}_2+{\widehat{\beta }}_3)-{\widehat{(\beta }}_0+{\widehat{\beta }}_1)={\widehat{\beta }}_2+{\widehat{\beta }}_3$. The difference in mean grades after and before the intervention period of the treated group;

The estimate of the Differences in Differences (DID) estimator will be: $({\widehat{\beta }}_2+{\widehat{\beta }}_3)-{\widehat{\beta }}_2={\widehat{\beta }}_3$.

5. Results

The following section presents the results of the estimated model for the two data panels. The first panel consists of students who took the SPAECE exam for the Portuguese language in all stages of education in the years 2018 and 2019, while the second panel includes a similar group of students who took the mathematics exam. However, it should be noted that the mathematics exam excludes students from the 2nd year of Elementary School, as the SPAECE-alfa program focuses solely on the Portuguese language exam.

In both panels, we were able to identify approximately 115 and 28 students, respectively, who were somehow connected to the CMIC program between the years 2018 and 2019. Their identification was based on information such as their first name, mother’s name, Brazilian Social Security Number (NIS in Portuguese) and Individual Taxpayer Registration Number (CPF in Portuguese). In addition to the previously mentioned variables, the panels were estimated by including teaching stage dummies to control for unobservable characteristics specific to each stage of education that remain constant over time. The standard errors of the estimate were clustered at the school level. The balancing t-test for the difference in means with the weighted covariates between the control and treated groups in the initial period is reported in Tables 4 and 5 in the appendix. Except for the gender dummy, there were no significant differences in the means of the other variables between the treatment and control groups in the initial period.

The main focus of interest, the DID estimate, measures the impact of the CMIC program on the performance of students from families participating in the program after its implementation. The estimated coefficients for this variable are presented in Table 3.A and were found to be positive and statistically significant at a significance level of 8.4% (p-value) for the Portuguese language exam (PL) and 5.5% (p-value) for the mathematics exam (MATH). These results indicate that being part of a family benefiting from the CMIC program is associated, on average, with a performance of approximately 11.63% higher in the SPAECE Portuguese and Mathematics exams compared to students who do not belong to families participating in the program.

Table 3.A
Results of the Estimated Models

Portuguese Exam (PL)
	ln(PL)	Standard deviation.	Value-p
2018 (baseline)
Control	5.8340
Treatment	5.7220
Diff (T-C)	-0.1120	0.0560	0.0450**
2019 (follow-up)
Control	5.8610
Treatment	5.8590
Diff (T-C)	-0.0020	0.0300	0.9560
DID	0.1100	0.0640	0.0840***
Matemathics Exam (MATH)
	ln (MATH)	Standard deviation	Value-p
2018 (baseline)
Control	5.8340
Treatment	5.7250
Diff	-0.1080	0.0530	0.0410**
2019 (follow-up)
Control	5.8450
Treatment	5.8470
Diff (T-C)	0.0020	0.0290	0.9480
DID	0.1100	0.0580	0.0550***

• Source: Developed by the authors. Note: *, **, *** denote statistical significance at the 1%, 5%, and 10% levels, respectively.

It is important to highlight that this result goes against previous findings related to the impact of cash transfer programs on school performance. The existing consensus in this type of study is that cash transfer programs affect enrollment rates and school attendance (Rawlings and Rubio, 2005; García and Saavedra, 2017). On the other hand, it is possible to say that there is no consensus regarding the impact of these programs on school performance, since, while some research finds positive impacts (Garcia and Hill, 2010), there are studies that do not find significant effects (Baird et al, 2014; Snilstveit, 2015). In this sense, this research joins a small group of research in which a positive effect on academic performance in Portuguese and mathematics is observed ⁸. Furthermore, it is not known to the authors that there are other studies with similar evidence for the Brazilian case.

As approximately 90% of the families in the treatment and control groups present in the sample correspond to families benefiting from Bolsa Família, it is possible to speculate that the CMIC income transfer works as a complement to Bolsa Família, allowing students belonging to these families to have access to a better diet and have greater freedom to dedicate themselves to studies, however, additional studies are necessary.

Regarding the covariate group, as shown in Table 3.B, the race variable has a positive and significant impact on the Portuguese language exam panel, while the age variable has a negative and significant impact. This implies that, on average, white students and younger students performed better in both the Portuguese language and mathematics exams during the analyzed period. The gender dummy variable also had a statistically significant impact on the Portuguese language exam panel. Furthermore, being a beneficiary of the conditional cash transfer program called Bolsa Família is associated with improved performance in the Portuguese language exams.

However, the binary variables related to administrative dependence, location, internet access, library, and computer lab did not have a statistically significant impact on both exams. In terms of infrastructure, only the dummy variable indicating access to the public sewage network had a negative and statistically significant impact.

Table 3.B
Results of Estimated Models (Covariates)

Variables	Exams
	Portuguese	Value-p	Matemathics	Value-p
Gender	-0.0690*	0.0000	0.0210	0.2780
	(0.0190)		(0.0190)
Race	0.0390**	0.0320	0.0190	0.4950
	(0.0180)		(0.0280)
Age	-0.0150*	0.0000	-0.0150*	0.0000
	(0.0030)		(0.0030)
Bolsa família	0.0560***	0.0930	0.0340	0.2660
	(0.0030)		(0.0310)
State-owned	-0.0400	0.7840	-0.0460	0.4110
	(0.1440)		(0.0560)
Location	-0.0170	0.4870	0.0290	0.1220
	(0.0250)		(0.0190)
Internet access	0.0310	0.4840	0.0150	0.5160
	(0.0450)		(0.0230)
Library	-0.0060	0.8130	-0.0160	0.4160
	(0.0250)		(0.0200)
Computer lab	0.0200	0.4980	0.0010	0.9440
	(0.0290)		(0.0210)
Sewage network	-0.0530*	0.0010	-0.0820**	0.0590
	(0.0160)		(0.0430)
$D_{2ºgrade}$	-0.5220*	0.0000
	(0.1480)
$D_{5ºgrade}$	-0.3120*	0.0180	-0.3160*	0.0000
	(0.1320)		(0.0640)
$D_{9ºgrade}$	-0.1010	0.4060	-0.1520*	0.0070
	(0.1210)		(0.0570)

• Source: Developed by the authors. Note: *, **, *** denote statistical significance at the 1%, 5%, and 10% levels, respectively.

Finally, the variables related to the teaching stage consistently showed a negative and significant impact, except for the 9th-grade dummy variable in the Portuguese exam. This suggests a negative differential in terms of school performance compared to the reference group - 3rd year of high school.

In order to check whether there is heterogeneity in the effect between the teaching stages, equation (4) was estimated with the inclusion of treatment dummies interacted with the treatment. In the Portuguese exam, the results found show that there is a positive and statistically significant difference in the effect of the program for the fifth- and ninth-year classes in relation to the third year of high school. Furthermore, with the inclusion of these interacted dummies, the effect of the program is no longer significant, which leads us to believe that these teaching stages were responsible for bringing about the positive and significant effect of the program. In the mathematics exam, no statistically significant heterogeneous effect is observed and the overall impact of the program remains positive and statistically significant after the inclusion of the interacted dummies⁹.⁹

5.1. Matching Quality and Robustness of Results

Two assumptions are required for good quality of the DID estimation. Regarding the first assumption, it is necessary to verify if there is parallelism in the results before the treatment period. Unfortunately, there is no information before the year 2018 in the database for the treated group at the student level. This is because families must have children aged between 0 and 5 years and 11 months to participate in CMIC. Therefore, this means that in years before 2018, students will no longer be part of the sample either because the children are not old enough to take the exam or because their children will be in educational stages where the SPAECE exam is not conducted. After all, besides the treatment group sample being small, the exam is only conducted in the 2nd, 5th, and 9th years of elementary school and the 3rd year of high school.

To verify if there is evidence supporting the hypothesis of parallel trends, the results are analyzed at the school level. For this, the treated school group consists of those that had at least one treated student (a child from a family belonging to CMIC) in the year 2019. In this sense, the average grades of treated students from these schools are calculated before and after the year 2019. The control school group consists of those whose students were not treated. In this case, the average grades of students from these schools are also calculated before and after the year 2019. The graphs in Figure 1 highlight the parallelism of the average grades of schools in Portuguese and mathematics exams between the years 2015 and 2019.

Figure 1
Parallel Trends at the School Levels

In relation to the second hypothesis, a test of the difference in means of covariates is conducted between the treated and control groups before the intervention period after matching. When we compare the results of the seventh column of Tables 1 and 2, before matching, with the results of the mean difference tests in Tables 3 and 4 in the appendix after matching, it is observed that now only the gender variable presents a difference in statistically significant mean.

In this sense, the verification of these two hypotheses allows us to conclude that the performed matching exhibits adequate predictive capacity and that the results of DID estimation are more reliable.

To check the robustness of the estimated results from equation (4), the following procedures are adopted. All results from these procedures are available in Tables 6 and 7 in the appendix¹⁰. Firstly, to mitigate the problem of differences in sample size between the treatment and control groups, random samples were drawn from both groups without replacement, fixing the number of observations from 40 to 100 pairs of the treatment and control groups before and after the intervention. The program’s effect becomes positive and statistically significant in all sample pairs for Portuguese and mathematics exams. The results range between 0.172 and 0.224 and 0.141 and 0.340 for Portuguese and Mathematics exams, respectively.

Next, regressions are also estimated for subsets of explanatory variables from equation (4). A regression is conducted only with student-level variables, and another one with school-level variables using the complete data sample. The results associated with the program’s effect remain positive and statistically significant in all cases.

Finally, estimations were made with interactions between the variables indicating treatment and the start of the intervention with gender and race dummies to check for heterogeneity in the effect of CMIC. After including these interactions, the average effect of the program remains statistically positive and significant in Portuguese and mathematics exams. Regarding the heterogeneity of the effect, it can be said that there is a negative differential concerning race in the mathematics exam, meaning that the program’s effect on white students is lower than that on students of other races.

6. Conclusion

The primary aim of this paper was to examine the causal impact of the CMIC program on school performance. Data from the school census, SPAECE, and information from the program were used to identify students from beneficiary families in 2019.

The results from the difference-in-differences approach combined with propensity score matching, indicate a positive and statistically significant impact of the CMIC program on students’ school performance in both the Portuguese language and mathematics SPAECE exams across all teaching stages. Additionally, at the individual level, the results suggest that white and younger students performed better on both exams during the analyzed period. Gender also had a significant impact, with male students performing worse in the Portuguese language exam on average. Furthermore, being a beneficiary of the Bolsa Familia program was associated with better performance in the Portuguese exam.

The binary variable representing administrative dependence did not show a significant impact in either exam. Similarly, the geographical location of schools, whether urban or rural, did not have a significant impact. Regarding school infrastructure variables, except for access to sewage network, none of them showed a significant impact on school exams. While this result is unexpected, Benevides (2020) obtained even a negative and significant effect on school performance in one of their estimates with an infrastructure index. Given the lack of significance in this set of variables, it is crucial for governmental entities to carefully address this issue and implement public policies to improve and maintain school infrastructure. This need becomes even more evident when considering the statistics related to these variables. With the exception of internet access, which already reaches over 90% of schools, approximately 40% of schools still lack a library or computer lab, and less than 1% have access to a public sewage network.

The positive impact of the CMIC program on school performance represents a novelty in terms of assessing and evaluating the impacts of transfer income programs, as it contradicts the findings of Camargo and Pazello (2014), Cireno, Silva, and Proença (2013), and Habenschus (2020), who did not find a significant impact of Bolsa Família on school performance or even found a negative relationship. On the other hand, this result is consistent with the findings of international articles. Another result obtained is a negative differential in the mathematics exam concerning race, indicating that the program’s impact on white students is lower than on students of other races. It is worth noting that the positive impact result remained consistent after conducting robustness experiments.

Finally, to enhance the accuracy of evaluations related to the CMIC program, it is recommended to develop performance measures specifically targeting the program’s target audience, such as children attending daycare or preschool. Moreover, improvements in the program’s databases and information systems that facilitate linking with school data are needed to better identify indirect beneficiaries and study the program’s indirect effects more effectively.

It is possible to point out possible research paths in relation to CMIC that allow overcoming some limitations of this study. Firstly, it would be important to expand the sample belonging to the treatment group so that it would be possible to carry out an assessment in which the teaching stages were treated individually. Furthermore, it would be interesting for the state government to develop an exam that would allow measuring the learning of children between 0 and 5 years old so that it would be possible to verify the effect of CMIC and other arms of the program on this group that is its direct beneficiary. Finally, research is needed to accurately verify whether there is in fact a complementarity between the CMIC and Bolsa Família and to precisely analyze the policy transmission mechanisms.

Agradecimentos

A paper resulting from research funded by FUNCAP for the State Secretariat for Planing and Management of Ceará(SEPLAG)

References

• Baird, S., Ferreira, F. H.G., Özler, B, and Woolcock, M. 2014. “Conditional, unconditional and everything in between: a systematic review of the effects of cash transfer programmes on schooling outcomes.” Journal of Development Effectiveness, 1-43.
• Benevides, A. A., and Soares, R. B. 2020. “Diferencial de desempenho de alunos das escolas militares: o caso das escolas públicas do Ceará.” Nova Economia, 30(1), 317-343.
• Blundell, R., and Dias, M. C. 2009. “Alternative Approaches to Evaluation in Empirical Microeconomics.” The Journal of Human Resources, 44(3), 565-640.
• Camargo, P. C., and Pazello, E. T. 2014. “Uma análise do efeito do programa bolsa família sobre o desempenho médio das escolas brasileiras.” Economia Aplicada [online], 18(4), 623-640.
• Camelo, R. de S., Tavares, P. A., and Saiani, C. C. S. 2009. “Alimentação, nutrição e saúde em programas de transferência de renda: evidências para o Programa Bolsa Família.” Revista EconomiA, 10(4), 685-713.
• Ceará. “Decreto nº 33905, de 27 de janeiro de 2021. Dispõe sobre o cartão mais infância Ceará - CMIC, atualizando a respectiva legislação em conformidade com a lei nº 17.380, de 05 de janeiro de 2021, que trata do programa mais infância Ceará, e dá outras providências.” Fortaleza.
• Cireno, F., Silva, J., and Proença, R. P. 2013. “Condicionalidades, desempenho e percurso escolar de beneficiários do Programa Bolsa Família.” In: Programa Bolsa Família: uma década de inclusão e cidadania. Brasília: Ipea, 297-304.
• Cruz, F. E. M. 2022. “Peer effect sobre o desempenho escolar: um estudo de caso na escola de educação profissional Adriano Nobre no Ceará.” Dissertation (Master’s in Public Sector Economics) - Faculdade de Economia, Administração, Atuária e Contabilidade - FEAAC, Programa de Economia Profissional - PEP, Universidade Federal do Ceará - UFC, Fortaleza (CE).
• Garcia, S., and Saavedra, J. E. 2017. “Educational Impacts and Cost-Effectiveness of Conditional Cash Transfer Programs in Developing Countries: A Meta-Analysis.” Review of Educational Research, 87(5), 921-965.
• Garcia, S., and Hill, J. 2010. “Impact of Conditional Cash Transfers on Children’s School Achievement: Evidence from Colombia.” Journal of Development Effectiveness, 2(1), 117-137.
• Gomes, L. S. 2020. “Impactos do programa nacional de alimentação escolar (PNAE) sobre o desempenho educacional no nordeste brasileiro.” Dissertação de mestrado em economia pelo programa de pós-graduação em economia da universidade do estado do Rio Grande do Norte. Mossoró.
• Khandker, S. R., Koolval, G. B., and Samad, H. A. 2010. “Handbook on Impact Evaluation: Quantitative Methods and Practices.” The World Bank.
• Hanushek, E. A. 2010. “Education Production Functions: Developed Country Evidence.” International Encyclopedia of Education, 2, 407-411.
• Habenschus, M. I. A. T. 2020. “Programa Bolsa Família: migração, desempenho escolar e efeito de longo prazo na pobreza.” Thesis (Ph.D) - Faculdade de Economia, Administração e Contabilidade de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto.
• Heckman, J. J., Ichimura, H., and Todd, P. E. 1997. “Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme.” Review of Economic Studies, 64, 605-654.
• Heckman, J. J., Ichimura, H., and Todd, P. E. 1998. “Matching as an econometric evaluation estimator.” Review of Economic Studies, 65, 261-294.
• Kern, A. P., Vieira, M., and Freguglia, R. 2018. “Impactos do programa bolsa família na imunizacão das crianças.” In 46° ENCONTRO NACIONAL DE ECONOMIA. Anais do 46° Encontro Nacional de Economia. Rio de Janeiro, Brasil, 1-20.
• Maynard, R. A., and Murnane, R. J. 1979. “The Effects of a Negative Income Tax on School Performance: Results of an Experiment.” Journal of Human Resources, 14(4), 463-476. doi:10.2307/145317.
  » https://doi.org/10.2307/145317
• Melo, A. P., and Suzuki, M. 2021. “Temperature, effort, and achievement: Evidence from a large-scale standardized exam in Brazil.” Working Paper.
• Park, R. J., Goodman, J., Hurwitz, M., and Smith, J. 2020. “Heat and Learning.” American Economic Journal: Economic Policy, 12(2), 306-339.
• Paxson, C., and Schady, N. 2010. “Does Money Matter? The Effects of Cash Transfers on Child Development in Rural Ecuador.” Economic Development and Cultural Change, 59(1), 187-229.
• Rawlings, L. B., and Rubio, G. M. 2005. “Evaluating the Impact of Conditional Cash Transfer Programs: Lessons from Latin America.” Oxford: Oxford University Press and The World Bank.
• Rocha, N. P. 2016. “Avaliação do Programa Nacional de Alimentação Escolar e dos Fatores Associados à Situação de (In)segurança Alimentar e Nutricional em Crianças.” Dissertação Mestrado em Ciências da Nutrição. Universidade Federal de Viçosa. Minas Gerais.
• Rodrigues, L. O., Costa, E. M., Silva, V. H. M. C., Mariano, F. Z., and Jesus Filho, J. 2020. “A note on performance differences between urban and rural schools in Brazil.” Revista Brasileira de Economia, 74(4), 494-507.
• Santana, O. M. M. L. de, et al. 2022. “Programa Mais Infância Ceará: De Programa a Política.” Fortaleza: SEDUC. E-book. ISBN 978-65-89549-42-0.
• Silverman, B. W. 1986. “Density Estimation for Statistics and Data Analysis.” Chapman & Hall, London.
• Smit, J. A., and Todd, P. 2005. “Does Matching Overcome Lalonde’s Critique of Nonexperimental Estimators?” Journal of Econometrics, 125, 305-353.
• Snilstveit, B., Phillips, D., and Martina, V. 2015. “Interventions For Improving Learning Outcomes and Access To Education In Low- And Middle-Income Countries: A Systematic Review.” International Initiative for Impact Evaluation, London International Development Centre.
• Souza, C. B., Ciríaco, J. A., and Soares, R. B. 2021. “Análise da Eficiência das Metodologias Desenvolvidas pelo Programa Jovem do Futuro Sobre o Desempenho Escolar dos Alunos das Escolas Públicas Estaduais do Ceará.” In IPECE, Economia do Ceará em Debate.
• Todd, P., and Wolpin, K. 2006. “Ex Ante Evaluation of Social Programs.” PIER Working Paper 06-122. Penn Institute for Economics Research, University of Pennsylvania, Philadelphia.
• Vieira, P. A. 2020. “O programa bolsa família como inclusão social e seu impacto na evasão escolar: análise da condicionalidade frequência escolar.” Estudos IAT, 5(3), 217-232.
• Villa, J. M. 2016. “diff: Simplifying the Estimation of Difference-in-differences Treatment Effects.” The Stata Journal, 16(1), 52-71.

• ♦
  A paper resulting from research funded by FUNCAP

• Classificação JEL

  I20, D04, E64.

• 1
  For further information, please refer to SPAECE - Ceará State Department of Education (seduc.ce.gov.br).
• 2
  Although the program was created in 2015, the first law that consolidated this initiative as a public policy was enacted in 2019. In this sense, the year 2019 will be taken as the reference year in the analysis of the DID model.
• 3
  For example, the student who took the SPAECE exam in the 5th grade in 2018 will only take it again in 2022, in the 9th grade.
• 4
  Propensity scores are estimated assuming a Probit distribution.
• 5
  The Epanechnikov kernel function was used, which efficiently minimizes the mean integrated square error. It was employed as a starting point for hn the value 0,0003, defined according to hSϕ=0,9n-15, as suggested by Silverman (1986). After visual inspection of the smoothed data, it was defined hn*=0,04 for both samples. It is important to note that there were no significant changes in the estimated model for various selected bandwidth values. According to Smith and Todd (2005), Kernel estimators are slightly insensitive to bandwidth width.
• 6
  To enhance the internal validity of the DID estimate, (3) was restricted to a common support of propensity score for the treated and control groups.
• 7
  These three sets of kernel weights are independently calculated according to the estimated propensity score and do not require a panel data structure.
• 8
  See García and Saavedra (2022) for a complete review of the impacts of cash transfer programs on educational outcomes.
• 9
  Results available upon request.
• 10
  The results of the mean difference tests in the period before the intervention can be seen in Tables 8 and 9 in the appendix. As expected, no significant mean differences are found in the randomly extracted subsamples.

APPENDIX

Table 4
Test of Mean Differences in Covariates between Treated and Control Groups before the Intervention Period after Matching. (Portuguese)

Weighted variables	Control	Treatment	Mean difference	p-Value
Gender	0.4660	0.1850	-0.2810	0.0002***
State-owned	0.2630	0.3330	0.0700	0.4386
Race	0.1350	0.1110	-0.0240	0.6954
Location	0.7620	0.6670	-0.0950	0.2952
Age	12.7250	12.8000	0.0750	0.9384
Bolsa Família	0.8730	0.8150	-0.0580	0.4368
Internet access	0.9100	0.8890	-0.0210	0.7225
Library	0.5600	0.5920	0.0330	0.7301
Computer lab	0.6540	0.6660	0.0120	0.8919
Sewage network	0.0000	0.0000	0.0000	-
$D_{2ºgrade}$	0.2510	0.3340	0.0830	0.3600
$D_{5ºgrade}$	0.3060	0.3330	0.0270	0.7684
$D_{9ºgrade}$	0.2040	0.1850	-0.0180	0.8059

• Source: Developed by the authors. Note: 1. Means and t-test are estimated using linear regression. 2.*** p<0.01; ** p<0.05; * p<0.10

Table 5
Test of Mean Differences in Covariates between Treated and Control Groups before the Intervention Period after Matching. (Mathematics)

Weighted variables	Control	Treatment	Mean difference	p-Value
Gender	0.4300	0.1670	-0.2630	0.0029***
State-owned	0.3800	0.4980	0.1180	0.3168
Race	0.1380	0.1120	-0.0260	0.7259
Location	0.8110	0.7250	-0.0860	0.4121
Age	14.5020	15.1830	0.6810	0.5230
Bolsa Família	0.8690	0.8330	-0.0360	0.6798
Internet access	1.0000	1.0000	0.0000	-
Library	0.6150	0.6650	0.0500	0.6523
Computer lab	0.7290	0.7770	0.0480	0.6258
Sewage network	0.0000	0.0000	0.0000	-
$D_{5ºgrade}$	0.4070	0.4980	0.0910	0.4376
$D_{9ºgrade}$	0.2450	0.2790	0.0340	0.7469

• Source: Developed by the authors. Note: 1. Means and t-test are estimated using linear regression. 2. *** p<0.01; ** p<0.05; * p<0.10.

Table 6
Results for Robustness Exercises (Portuguese)

Variables		Random Subsamples								Covariate Subsets			Full Sample
		n = 40	n = 50	n = 60	n = 70	n = 80	n = 90	n = 100		Student	School
DID		0.198***	0.191***	0.172***	0.224**	0.214**	0.200**	0.177***		0.107**	0.157**		0.109***
		0.100	0.104	0.098	0.110	0.089	0.077	0.093		0.054	0.062		0.063
Gender		‐0.237*	‐0.113***	‐0.153**	‐0.044	-0.040	-0.040	-0.064		‐0.068*			‐0.074*
		0.083	0.063	0.064	0.053	0.041	0.038	0.042		0.022			0.010
Race		0.093	0.075	0.106**	0.056	0.033	0.069**	0.050		0.043**			0.048**
		0.087	0.055	0.049	0.054	0.054	0.030	0.037		0.018			0.019
Age		‐0.053*	‐0.041**	‐0.055**	‐0.025*	-0.017	-0.014	-0.041*		‐0.015*			‐0.015*
		0.013	0.019	0.024	0.008	0.015	0.003	0.010		0.003			0.003
Bolsa Família		0.035	0.16***	‐0.087	0.235**	0.173**	0.217*	0.188**		0.074**			0.055***
		0.086	0.094	0.124	0.113	0.083	0.066	0.092		0.034			0.032
Location		0.031	‐0.024	‐0.049	‐0.054	0.111**	-0.035	-0.063			‐0.018		‐0.018
		0.079	0.050	0.059	0.057	0.057	0.053	0.053			0.028		0.025
State-owned		‐0.223	0.106	0.187***	0.187	-0.182	-0.171	0.018			0.125*		‐0.042
		0.156	0.110	0.108	0.113	0.236	0.231	0.079			0.043		0.140
Internet access		0.055	0.097	0.015	0.100	0.001	-0.018	0.015			0.058		0.031
		0.112	0.124	0.103	0.124	0.119	0.062	0.065			0.053		0.045
Library		‐0.019	0.006	‐0.017	‐0.051	0.065	0.035	0.040			0.004		‐0.005
		0.086	0.075	0.078	0.057	0.059	0.041	0.048			0.036		0.025
Computer lab		‐0.032	‐0.021	0.049	‐0.026	-0.006	-0.020	-0.021			0.046		0.019
		0.099	0.074	0.063	0.064	0.073	0.073	0.052			0.040		0.029
Sewage network		0.000	0.000	0.000	0.000	0.000	-0.06*	0.000			‐0.031**		‐0.053*
		0.000	0.000	0.000	0.000	0.000	0.023	0.000			0.014		0.016
Race*T* $D_{2019}$													‐0.025
													0.045
Gender*T* $D_{2019}$													0.017
													0.064
$D_{2ºgrade}$		‐1.240*	‐0.601**	‐0.685**	‐0.335***	-0.675***	-0.662*	-0.646*		‐0.489*			‐0.524*
		0.268	0.301	0.331	0.173	0.389	0.256	0.161		0.049			0.146
$D_{5ºgrade}$		‐0.926*	‐0.382	‐0.431	‐0.139	-0.421	-0.392***	-0.468*		‐0.281*			‐0.315*
		0.216	0.233	0.286	0.154	0.315	0.220	0.159		0.034			0.128
$D_{9ºgrade}$		‐0.287**	‐0.039	0.034	0.142	-0.107	-0.165	-0.055		‐0.074*			‐0.103
		0.146	0.124	0.148	0.105	0.168	0.140	0.090		0.022			0.118

• Source: Developed by the authors. Note: *, **, *** denote statistical significance at the 1%, 5%, and 10% levels, respectively.

Table 7
Results for Robustness Exercises (Mathematics)

Variables		Random Subsamples								Covariate Subsets			Full Sample
		n = 40	n = 50	n = 60	n = 70	n = 80	n = 90	n = 100		Student	School
DID		0.171***	0.340*	0.147***	0.206**	0.201***	0.141***	0.166**		0.127**	0.161*		0.105***
		0.098	0.126	0.084	0.104	0.103	0.082	0.083		0.054	0.049		0.055
Gender		-0.068	0.104	0.093***	0.065	0.009	0.085***	0.076***		0.020			‐0.002
		0.088	0.086	0.055	0.053	0.068	0.044	0.041		0.020			0.007
Race		0.156	0.156	0.028	0.066	0.070	-0.017	0.024		0.019			0.062**
		0.133	0.119	0.045	0.096	0.079	0.063	0.051		0.029			0.025
Age		-0.024	-0.034	-0.014*	-0.044**	-0.013	-0.060*	-0.016*		‐0.015*			‐0.015*
		0.018	0.027	0.003	0.019	0.022	0.019	0.003		0.003			0.003
Bolsa Família		0.028	0.167	0.068	0.139	0.217**	0.153**	0.113		0.024			0.031
		0.084	0.109	0.106	0.095	0.103	0.070	0.068		0.028			0.028
Location		0.164**	0.110***	0.020	0.180*	0.111***	0.070***	0.051			0.036**		0.027
		0.080	0.062	0.052	0.046	0.061	0.040	0.048			0.018		0.018
State-owned		0.006	0.101	-0.044	-0.329*	-0.078	-0.283*	-0.050			0.081*		‐0.052
		0.122	0.121	0.121	0.100	0.152	0.075	0.098			0.027		0.058
Internet access		0.042	0.000	-0.094	0.000	-0.054	-0.046	-0.042			0.039		0.006
		0.137	0.000	0.079	0.000	0.098	0.098	0.060			0.03		0.029
Library		-0.011	-0.005	-0.047	-0.051	-0.047	-0.013	-0.075			‐0.017		‐0.014
		0.056	0.060	0.041	0.053	0.062	0.037	0.048			0.019		0.019
Computer lab		-0.034	-0.009	0.056	0.010	0.074	-0.002	-0.013			0.002		‐0.002
		0.091	0.096	0.081	0.056	0.076	0.056	0.056			0.020		0.021
Sewage network		0.000	0.000	0.000	-0.154*	0.000	-0.064	-0.116**			‐0.062***		‐0.081***
		0.000	0.000	0.000	0.041	0.000	0.107	0.055			0.036		0.043
Race*T* $D_{2019}$													‐0.111***
													0.062
Gender*T*$D_{2019}$													0.082
													0.061
$D_{5ºgrade}$		-0.332	-0.481***	-0.299**	-0.805*	-0.276	-0.921*	-0.352*		‐0.489*			‐0.324*
		0.236	0.265	0.148	0.174	0.275	0.197	0.105		0.049			0.066
$D_{9ºgrade}$		-0.147	-0.124	-0.110	-0.462*	-0.110	-0.399*	-0.150***		‐0.281*			‐0.159*
		0.123	0.145	0.115	0.074	0.155	0.095	0.077		0.034			0.060

• Source: Developed by the authors. Note: *, **, *** denote statistical significance at the 1%, 5%, and 10% levels, respectively.

Table 8
Mean Difference Test of Covariates in 2018 at Robustness checks (Portuguese)

Variables		n = 40	n = 50	n = 60	n = 70	n = 80	n = 90	n = 100		Student	School		Full Sample
Gender		0.099	0.025	-0.103	0.009	-0.046	0.037	-0.020		-0.316***			-0.281***
Race		0.112	0.017	-0.005	0.090	-0.019	0.040	-0.115		-0.028			-0.024
Age		1.175	1.184	0.217	-1.220	0.770	2.005	0.205		-0.068			0.057
Bolsa Família		0.033	-0.098	-0.115	0.011	0.053	0.171	0.114		-0.066			-0.058
Location		-0.186	0.179	0.031	0.038	0.024	-0.120	-0.081			-0.122		-0.094
State-owned		0.112	0.162	0.043	0.108	0.087	0.124	-0.065			0.096		0.069
Internet access		-0.090	0.198	0.052	0.031	0.018	0.072	-0.021			-0.027		-0.022
Library		0.216	0.141	0.001	-0.123	-0.003	-0.015	0.086			0.029		0.032
Computer lab		-0.144	0.235	-0.024	-0.035	0.074	0.058	0.016			0.019		0.012
Sewage network		0.000	0.000	0.000	0.000	0.000	0.000	0.000			0.000		0.000
$D_{2ºgrade}$		-0.016	-0.131	-0.093	-0.028	-0.065	-0.176	-0.007		0.100			0.084
$D_{5ºgrade}$		-0.160	-0.028	0.043	-0.024	-0.021	0.010	-0.034		0.049			0.026
$D_{9ºgrade}$		0.096	0.065	0.052	0.108	0.048	0.062	0.086		-0.070			-0.018

• Source: Developed by the authors. Note: 1. Means and t-test are estimated using linear regression. 2. *** p<0.01; ** p<0.05; * p<0.10.

Table 9
Mean Difference Test of Covariates in 2018 at Robustness checks (Mathematics)

Variables		n = 40	n = 50	n = 60	n = 70	n = 80	n = 90	n = 100		Student	School		Full Sample
Gender		-0.253	0.196	-0.221	0.025	-0.033	-0.114	-0.005		‐0.329***			‐0.262***
Race		0.113	-0.076	0.128	0.015	-0.007	0.039	0.037		‐0.030			‐0.026
Age		-1.931	1.097	1.640	0.346	1.094	1.027	-0.177		0.777			0.631
Bolsa Família		0.333	-0.082	0.196	-0.143	-0.025	0.029	0.107		‐0.045			‐0.037
Location		-0.103	-0.018	0.084	0.109	0.120	-0.040	0.059			‐0.123		‐0.082
State-owned		-0.250	0.154	0.230	-0.026	0.228	-0.054	0.029			0.187		0.115
Internet access		0.000	0.000	0.000	0.000	0.000	0.000	0.000			0.000		0.000
Library		-0.144	0.221	0.279	-0.054	0.158	0.051	0.128			0.051		0.048
Computer lab		0.077	-0.072	0.058	0.030	0.027	-0.017	0.109			‐0.001		0.047
Sewage network		0.000	0.000	0.000	0.000	0.000	0.000	0.000			0.000		0.000
$D_{5ºgrade}$		0.280	-0.107	-0.292	0.000	-0.171	-0.056	-0.044		0.127			0.089
$D_{9ºgrade}$		-0.197	-0.018	0.177	0.026	0.110	-0.058	0.159		‐0.057			0.036

• Source: Developed by the authors. Note: 1. Means and t-test are estimated using linear regression. 2. *** p<0.01; ** p<0.05; * p<0.10.

EDITOR-CHEFE

Dante Mendes Aldrighi https://orcid.org/0000-0003-2285-5694 Professor - Department of Economics University of São Paulo (USP)

EDITOR ASSOCIADO

Fábio Waltenberg https://orcid.org/0000-0003-3404-7424

Publication Dates

• Publication in this collection
  07 Oct 2024
• Date of issue
  Apr-Jun 2024

History

• Received
  11 June 2023
• Reviewed
  06 Dec 2023
• Accepted
  16 Feb 2024