Elos Program's Efficacy Evaluation in School Management of Child Behavior: A Non-Randomized Controlled Trial

Avaliação da Eficácia do Programa Elos no Manejo Escolar do Comportamento Infantil: Um Ensaio Controlado Não Randomizado

Abstract

The Elos Program resulted from a Brazilian cultural adaptation of the Good Behavior Game, a preventive strategy for classroom management to reduce aggressive and disruptive behaviors. The goal is to discuss the Elos Program’s efficacy during its implementation in 16 elementary Brazilian public schools in two cities in 2016. The design was a non-randomized controlled trial with 80 classes in each group, experimental and control, involving 1,731 students. The study used the Generalized Estimating Equation model to verify the program's effect. Elos seemed to be effective in reducing aggressiveness and disruptive behavior in boys. These results are aligned with others GBG international studies and suggest that, after a randomized trial, the program would be ready to be disseminated in Brazil.

Keywords:
Good Behavior Game; efficacy; cultural adaptation; prevention program; child mental health

Resumo

O Programa Elos é resultante da adaptação cultural brasileira do Good Behavior Game (GBG), uma estratégia preventiva para a gestão de sala de aula, visando diminuir comportamentos agressivos e disruptivos. O objetivo foi discutir a eficácia do Elos em sua implementação, em 16 escolas públicas de ensino fundamental, em duas cidades brasileiras, em 2016. Delineado como ensaio clínico controlado não randomizado, com 1.731 estudantes, em 80 turmas divididas entre experimental e controle, utilizou-se Generalized Estimating Equation para verificar o efeito do programa. Houve indicações da eficácia do Elos na redução da agressividade em meninos. Esses resultados estão alinhados com outros estudos internacionais do GBG e sugerem que, após um estudo randomizado, o programa estaria pronto para sua disseminação no Brasil.

Palavras-chave:
Good Behavior Game; eficácia; adaptação cultural; programa preventivo; saúde mental infantil

Studies have shown a significant association between childhood vulnerabilities inscribed in the developmental processes, based on social interaction patterns, and future antisocial behaviors (Catalano & Hawkins, 1996; Quinn et al., 1995). Some of the behavior problems in adulthood, such as substance abuse and psychological distress, have their origins in learned behaviors in childhood. Accordingly, aggressive, disruptive, inattentive, and social isolation behaviors that present themselves at an early age may predict future antisocial behaviors (Kellam et al., 1998; Poduska et al., 2008). Therefore, prevention researchers suggested that it is essential to intervene early in development processes, creating environments sensitive to children's psychosocial needs, providing them with protective elements at different levels: individual, family, institutional, and community.

Thus, ecosystems that offer opportunities for learning emotional responsiveness, strengthening of bonds, and social interactions are essential for children's positive development (Britto et al., 2017; Catalano & Hawkins, 1996; Daelmans et al., 2017). Nurturing interactions are crucial to mitigate the psychosocial risks, protecting children in their developmental processes, activated through the nurturing care of trusted adults (Daelmans et al., 2017; Hawkins et al., 2015) and peer mediation (Quinn et al., 1995). These arguments justify the implementation of mental health prevention programs during childhood, which should mitigate risk and increase protective factors from different psychosocial dimensions to reduce future antisocial behaviors, such as problems related to drug use and involvement in violence (Catalano & Hawkins, 1996; Sloboda & Petras, 2014).

The school stands out among possible preventive intervention contexts, given the crucial socializing role this institution plays in young people's lives (Sloboda & Petras, 2014). The diversity of psychosocial training resulting from various contexts of adversity in the first stages of children's socialization makes many of them start school in a situation of vulnerability. Therefore, these children behave and feel negatively affected by the school environment's cognitive, emotional, and relational demands and do not engage in school activities and harmonious living with colleagues and reference adults. In turn, the school and its teaching staff do not always have technical resources to mediate children's socio-emotional development and retrace vulnerable trajectories. Addressing children's social and emotional needs would require teachers, who are already overloaded with their daily activities, to have a complementary look at these pedagogical dimensions (Lorenzo & Schneider, 2021). Accordingly, programs that assist teachers in mediating educational processes and also support individual and contextual changes may be essential strategies for breaking cycles of intergenerational exclusion and psychosocial vulnerabilities.

These challenges are even more significant when developing interventions for Brazilian public schools. They have been undergoing historical scrapping, their teaching staff lives an experience of work overload, and due to excessive demand, professionals lose the ability to perceive the outcome of activities they conduct (Monceau, 2008). For this reason, offering strategies for classroom management, aiming at including all students in the learning process, could be an important pedagogical tool and a great preventive strategy.

Thus, the purpose of this study is to analyze the Elos Program - Building Collectives, which aims at 6 to 10 years old children and is implemented from the 1st to the 5th grade of Brazilian public elementary schools by teachers who were trained for the task. This program resulted from a cultural adaptation of the Good Behavior Game (GBG) for Brazilian schools, a North American classroom management strategy aiming to prevent children's aggressive and disruptive behaviors. GBG developers created the program as a group contingency intervention with a solid empirical principle (Barrish et al., 1969) of promoting self-regulation and peer social control (Ford et al., 2013).

The GBG aims to promote social control of behaviors through peer mediation, as recognizing the team's achievements depends on each individual's behavior and the group as a whole (Ford et al., 2013; Joslyn et al., 2019). The teacher distributes the students in teams based on an analysis of their social interactions, behaviors, and learning conditions, aiming to promote social mediations. The students discuss rules of coexistence to be followed when playing the game. The game consists of routine curricular activities that the groups should complete following the pre-established rules. The recognition of the team's achievements depends on each individual's behavior and the group. (Ford et al., 2013).

Some reviews reported results that demonstrate the effectiveness of the original program GBG in decreasing aggressive, disruptive behaviors and other psychosocial and educational vulnerabilities in children (Bowman-Perrott et al., 2016; Flower et al., 2014; Leflot et al., 2013), with evident effects remaining throughout participant's developmental trajectories. Several longitudinal studies have demonstrated impacts on future antisocial behaviors, such as drug abuse, sexual risky, and delinquency, through its function as a personal protective factor (Kellam et al., 2008; Kellam et al., 2011; Kellam et al., 2014; Poduska et al., 2008). For this reason, many experts considered GBG to be a kind of behavioral vaccine (Embry, 2002), and different cultural adaptations confirmed this effectiveness for other countries and cultures (Bayer et al., 2009; Nolan et al., 2014). "The Good Behavior Game (GBG) has been tested internationally with culturally, linguistically, and socioeconomically diverse student populations and has demonstrated a consistent pattern of experimental success" (Nolan et al., 2014, p. 192). More recent meta-analyses and reviews have argued that GBG's effect size may vary according to the expected outcome, the initial risk, and the participants' gender (Smith et al., 2019). The authors suggest the necessity for new research that evaluates GBG in different contexts and new populations. They recommend using innovative data collection methods to apprehend indirect effects and pursue a more precise definition of the mechanisms that can be accounted for the long-term effects and behavior variations (Joslyn et al., 2019).

The successive confirmations of the GBG's effectiveness were one of the main reasons UNODC indicated it as a promising program for adaptation by the Brazilian government as a preventive public policy on mental health and drug abuse problems. The program's adaptation process to Brazilian culture started in 2013, organized by the Mental Health, Alcohol, and other Drugs Coordination of the Brazilian Ministry of Health (BMH). This project involved the joint adaptation of two other evidence-based preventive programs: the European Unplugged and the Strengthening Families Program, forming a broad preventive public policy that would start with actions in childhood (GBG), going through adolescence (Unplugged), and providing support for households (SFP) (Ministério da Saúde, 2018).

The Universidade Federal de Santa Catarina and the Universidade Federal de São Paulo evaluated its implementation and results. In the first year, the authors carried out a pre-pilot study to evaluate the original Good Behavior Game implementation process translated into Portuguese, aiming to test the program's acceptability, feasibility, and fidelity. It involved six public schools, three in Santa Catarina and three in São Paulo, including 28 teachers, nine technical teams, six coaches, and 684 students (Schneider et al., 2014).

In 2014, the authors conducted a longitudinal pilot study to evaluate the program's first cultural adaptation, named Programa Elos: Construindo Coletivos. This study had a quasi-experimental design conducted with a pre-posttest of the only intervention groups to identify temporal changes in classroom behavior. It involved four cities in São Paulo and Santa Catarina states, ten schools, 34 classes, and 614 students (Schneider et al., 2016). In this same study, the authors conducted a psychometric validation of the TOCA-R scale (Teacher Observation of Classroom Adaptation - Revised) for the context of Brazilian schools, directing towards a future study of the program's efficacy (Schneider et al., 2020).

The third study, developed in 2016, presented in this manuscript, aimed to evaluate the Elos Program's efficacy on children's psychosocial behaviors during its implementation in Brazilian elementary classrooms.

Method

Study Design

A non-randomized controlled trial, parallel and with two groups, was carried out in 2016 at two Brazilian cities. It was not possible to conduct the randomization of the classes since the Municipal Education Departments of São Paulo (SP) and São Bernardo do Campo (SBC), both in São Paulo State, agreed to participate in the study on the condition that they could appoint the schools and classes that should receive the program and enter the intervention group. The evaluation team selected the control classes in the same schools and grades to guarantee parity of conditions in the study, with pre and post-tests also performed in the same period.

Sample Size

The researchers made the sample size calculation of the present study based on estimates from the pilot quasi-experimental study of the Elos Program in 2014 (Schneider et al., 2014). Expected changes were: a) a mean decrease of 0.13 points (standard error of 0.04) in the aggressiveness scale after the intervention (p = 0.001); b) a mean reduction of 0.27 (0.04) points in the disruptive behavior scale (p < 0.001); c) a mean increase of 0.14 (0.04) points in the engagement scale after the intervention (p < 0.001); and d) a mean increase of 0.14 (0.04) points in the self-control scale after the intervention (p = 0.005).

In order to calculate the sample size, a type I error of 0.05 (α = 5%), and a type II error of 0.20 (β = 20%), we established 80% power. For each result described above, the authors calculated the sample size so that the largest size obtained would be adopted. The authors used STATA/SE 13.1 for Windows to conduct statistical analysis. The researchers got a sample size of 380 students, that is, 190 students in each group. They increased the sample size by 50% to account for losses at follow-up, ending with a final sample size of at least 570 students (285 in each group).

Participants

Sixteen public schools of elementary education (1^st to 5^th grade) participated in the study, 11 schools in the city of São Paulo, and five schools in the city of São Bernardo do Campo generating a final sample size of 2,319 students at baseline, whose age varied from 07 to 14 years old. The study included 20 intervention classes and 20 control classes in each city, 40 classes per group, and 80 classes in total, this also being the number of participating teachers. There was a loss of 515 students (22%) and 13 teachers (16%) on the post-test. Five teachers did not complete the program due to retirement, leave, or school transfer. These were also reasons why three control group teachers did not complete the questionnaires in the second moment. However, in this group, five teachers did not complete the post-test due to lack of time at the end of the school semester, according to their justification, with more significant data losses in this group.

Considering that this study aimed to understand the Elos Program's efficacy on students exposed to the entire intervention, only among adherent teachers, the authors opted for a complete case approach, excluding the classes in which the teacher interrupted the implementation and did not finish the follow-up measure. Between pre-test and post-test, it was possible to pair 1,731 students, 941 from the experimental group and 790 from the control group, in 67 classes, as shown in Figure 1.

Figure 1
Flowchart of the Elos Program Efficacy Study in Brazilian Schools in 2016

Instruments and Outcomes

The study used the instrument "Mapeamento das Interações dos Estudantes" (MINE), the Brazilian cultural adaptation of Teacher Observation of Classroom Adaptation - Revised (TOCA-R), which is the traditional scale used by the GBG studies to assess it is efficacy and effectiveness (Kellam et al., 2011; 2014). The authors conducted a psychometric study of the cultural adaptation of TOCA-R's version in 2014, during the pilot quasi-experimental evaluation of the Elos Program. The exploratory factor analysis results showed an acceptable fit for five factors within 25 items, with a total variance explained of 60% and a mean residual error of 0.02. Confirmatory factor analysis showed a satisfactory fit for the model (χ² = 961, df = 265, RMSEA = 0. 078 [0.07, 0.08 95% CI], CFI = 0.9), with configurational, metric, and scalar invariance of the latent structure also being identified. The variation amplitude of the precision coefficients between the five dimensions of the instrument (α = 0.78-0.92, ω = 0.76-0.92) demonstrated validity and reliability for evaluating the Elos Program in Brazilian schools (Schneider et al., 2020).

The MINE, with targets ages from 06 to 12 years old, consists of 25 statements about the child's behavior, which the teacher assesses on a six-point scale ("Never," "Rarely," "Sometimes," "Often," "Very often," "Almost Always"). The items assess five dimensions of the students' behaviors, as indicated in the factor analysis: 1) aggressive behavior; 2) disruptive behavior; 3) engagement in activities; 4) socialization; 5) self-control. The researchers considered these as the dependent variables of this study to assess the Elos Program's efficacy (independent variable). At the end of the questionnaire, the teacher should also classify each child into one of four categories based on an overall assessment of the child's behavior: Aggressive, Distracted, Shy, and Cooperative (Schneider et al., 2020).

Data Collection and Study Settings

The team created a questionnaire using Google Forms online platform with all 25 items from the MINE scale (Schneider et al., 2020) and sent a link to the participating teachers, who had to add each student's name to fill the questionnaire for each of them. Participants completed the forms on their own. The traditional TOCA-R application model is applied through interviews with teachers (Kellam et al., 2011; 2014). However, some studies conducted with self-administered TOCA have shown that it is a feasible alternative to the original structured interview format, with valid results (Koth et al. 2009). In the case of the Brazilian study, given the difficulty of public school teachers making time available during their working hours for the interview with the researchers, as in this modality, and since it takes around 30 minutes to complete the questionnaire for each student, we opted for the auto-fill modality.

The teachers received training to complete the online questionnaires, during which the researchers explained the conceptual framework for the observed behaviors accounted for on MINE and filling instructions. Training took place in April and May 2016. After that, teachers had 15 days to fill in the online forms (baseline). The research team monitored and encouraged through emails and mobile messages those who had not filled the forms in time. Next, the teachers implemented the Elos Program in the experimental classes in the following six months. Finally, the researchers conducted the post-test, November and December 2016, reviewing the teachers' training about the online self-administered questionnaires and following the same procedure as the pre-test.

The Intervention

The National Coordination of Mental Health, Alcohol, and Other Drugs of the BMH was responsible for implementing the Elos Program in Brazilian schools and brought international program developers from the American Institute of Research (AIR) to train national trainers. Each school had an education professional and a health professional from the local Primary Health Unit as coaches trained by national trainers. They served as support for the teachers in their formative process and performed monthly monitoring related to the intervention's evaluation. The teachers and local coaches received book guides that explained each person's roles in the process, the program's conceptual map, the daily intervention step by step, and the instruments they should fill out to be part of the implementation evaluation fidelity.

The Elos Program follows the core elements of GBG, with some cultural adjustments to the procedures and instructions for the Brazilian classrooms. The core elements are: 1) The Educator should divide all the children into heterogeneous teams. 2) During a regular classroom activity, the four rules of social interaction, called Elos Game Agreements, should come into effect: a) follow the instructions of the activities; b) follow the agreed voice level; c) remain in the places agreed; and d) be kind with colleagues and teachers. 3) Teachers and students would jointly define the behaviors that mean breaking the rules. 4) The final goal is, for all teams, to finish the activity following the rules and break as few agreements as possible. In order to win the game, the teams could break the rules four times more within the activity time. 5) The teacher would mediate the interactions among students and give the teams feedback about their behavior. 6) The activity duration may vary between 10 and 30 minutes, and the teacher would define and announce it before each game. 7) At the end of the time set for the Elos' activity, educators and students participated in a moment of recognition for the teams that won the game that day, called the "Elos Super Teams." The game could have one or more winners, and even all teams could win in the same game (Ministério da Justiça, 2016)¹.

The local coaches monitored the implementation every two weeks by visiting the classrooms and having meetings with the teachers. After the sessions, the coaches would complete the online fidelity forms according to their implementation. The researchers evaluated the dosage using two items presented in this fidelity questionnaire. The questions were: a) on average, how many times did you implement the Elos program per week? b) On average, how long did the activity last each time you played? In total, 27 of the 35 teachers responded to these items about the Elos Program implemented dosage. One-third of the teachers used the game for 10 to 20 minutes a time, and 45% played it for 20 to 30 minutes. Among respondents, 22% implemented the Elos program twice a week, and 60% played it three times².

Statistical Analysis

The researchers conducted a descriptive analysis of the data, and absolute and relative frequencies analyzed the categorical variables and reported it in summary measures (mean, quartiles, minimum, maximum, and standard deviation).

The researchers also compared the classes' proportions in the intervention and control group using a Binomial test for one sample and a Chi-Square analysis to compare students' profile distributions (gender, grade, and municipality). The authors also evaluated the internal consistency between the items of each component using Cronbach's Alpha coefficient. The analysis included comparisons of mean scores for Aggressiveness, Disruptive Behavior, and Engagement in Activities, Socialization, and Self-control at baseline - using student's T-test for independent samples.

In the function of ignorability in the treatment as for an observational study, it is crucial to assess whether there were systematic differences between the experimental and control groups. The analysis evaluated the continuous variables utilizing t-test and discrete variables by χ^2, correcting the statistical test's value according to the sample's multilevel design. The null hypothesis test result suggested no systematic differences between the experimental and control groups in the pre-test concerning the variables accounted for in the study. Results strengthen the hypothesis of ignorability and justify interpreting the "difference of the differences" as a causal treatment effect.

The researchers used a Generalized Estimating Equation (GEE) model to verify the effect of the program in each of the five components of the MINE questionnaire (dependent variables) (Zeger & Liang, 1986). The GEE consists of a generalization of the linear models. It allowed incorporating the dependence between the same individual's observations resulting from the repeated measures over time. This study adopted identity link functions, normal marginal distribution, and an interchangeable dependency structure between students' observations. Even though the model assumed normal marginal distribution, it also allowed for the assumption of normality to be relaxed in the distribution of the dependent variables. Besides, the analysis included a robust estimator in calculating the standard error of the estimates. Since there was a violation of the normality assumption for all behavioral variables in the data distribution, it was impossible to use a mixed linear model (also known as a multilevel model or hierarchical linear model). For all statistical tests, the researchers adopted a significance level of 5%. The Program Stata 12 was used to perform statistical analyses.

Ethic Dimension

All procedures in the present study performed the ethical standards of the institutional and national research committees. The Research Ethics Committees of the University of São Paulo (#473.498) and the Federal University of Santa Catarina (#711.377) approved this research.

School principals signed the Free and Informed Consent because they are legal guardians for Brazil's children during school hours. Parents received a letter sent by the School Board and researchers describing the study to be carried out as well as its risks and benefits. If they disagreed with their children's participation in the study, parents should inform the school and researchers about the impediment. The teachers who participated in the survey also responded to the Free and Informed Consent.

Results

Baseline Sample Characteristics

There were 1,731 participating students in both baseline and follow-up data from 67 classes, 35 of which (52.2%) received the intervention, with no difference between proportions by group (p = 0.807). The mean number of students per class was 25.8 students (SD = 4.7 students), with a minimum of eight students and a maximum of 35. The median number of students was 26, showing similarity to the mean value, as reported in Table 1. The age distribution was quite equitable between the experimental and control groups with a variation between 7 and 11 years old, with some students aged between 12 and 14 years old- represented in both groups. The mean age was 8.53 years, with an sd of 1.37.

Table 1.
Distribution of Students by Gender, Grade, and City, according to the Control or Experimental Group: Elos Program Efficacy Evaluation Study in Brazilian Schools, 2016

	Groups				Total
	Control		Experimental
Gender	790	100.0%	941	100.0%	1731	100.0%	0.778
Male	405	51.3%	476	50.6%	881	50.9%
Female	385	48.7%	465	49.4%	850	49.1%
Grade	790	100.0%	941	100.0%	1731	100.0%	0.197
1st grade	226	28.6%	271	28.8%	497	28.7%
2nd grade	144	18.2%	140	14.9%	284	16.4%
3rd grade	228	28.9%	308	32.7%	536	31.0%
4th grade	141	17.8%	172	18.3%	313	18.1%
5th grade	51	6.5%	50	5.3%	101	5.8%
City	790	100.0%	941	100.0%	1731	100.0%	<0.001
SBC	429	54.3%	426	45.3%	855	49.4%
SP	361	45.7%	515	54.7%	876	50.6%

• Note. p - a descriptive level of the Chi-Square test; SBC = São Bernardo do Campo; SP = São Paulo

According to table 1, there were different distributions of location (city) by group (p < 0.001). The experimental group had a higher percentage of students residing in São Paulo (54.7%) than in São Bernardo do Campo (45.3%).

Outcomes and Estimation

The researchers' generated average scores for each factor accounted for the MINE questionnaire and resized them to range from 0 to 100. The study also included each behavior's internal consistencies at each moment, pre-and post-test, using Cronbach's Alpha coefficient, which was high, ranging from 0.944 to 0.800.

There were differences in the mean scores of the following behavior patterns: Disruptive Behavior (p = 0.034), Socialization (p = 0.002), and Self-ontrol (p = 0.009), as presented in Table 2. Accordingly, the analysis indicated that the experimental group had significantly lower means in the scores for Socialization and Self-Control than the control group, with an inverse pattern for the Disruptive Behavior means scores at the beginning of the study. It is interesting to note bias due to the lack of randomization, which may impact the results and indicate the need to develop a future randomized controlled trial for the analyzed program.

Table 2.
Mean and Standard Deviation of the Scores for Target Behaviors at the Beginning of the Elos Program Efficacy Evaluation in Brazilian Schools in 2016

MINE	Pre
Aggressive behavior		0.265
Control	16.6 ± 21.4
Experimental	17.7 ± 20.4
Disruptive behavior		0.034
Control	30.6 ± 24.8
Experimental	33.1 ± 23.5
Engagement in activities		0.804
Control	65.3 ± 28.9
Experimental	65.6 ± 27.6
Socialization		0.002
Control	73.9 ± 19.3
Experimental	71.0 ± 19.8
Self-control		0.009
Control	60.3 ± 23.6
Experimental	57.4 ± 22.8

• Note. N = 790 and 941 respectively for the control and experimental groups; Mean ± SD. - a descriptive level of the student's t-test.

The description of behavior scores, separated by the research stages, control or experimental group, gender, is presented in Table 3 and supported the GEE model's application.

Table 3.
Mean and Standard Deviation of the Scores for Target Behaviors by Group and Moment of Evaluation of the Efficacy of the Elos Program in Brazilian Schools, in 2016, presented according to Gender

MINE	Boys			Girls
	Pre	Post	Post - Pre	Pre	Post	Post - Pre
Aggressive behavior
Control	21.6 ± 24.4	19.7 ± 24.2	-2.0 ± 16.6	11.2 ± 16.0	10.2 ± 16	-1.0 ± 11.3
Experimental	23.8 ± 22.5	18.8 ± 20.5	-5.0 ± 14.6	11.4 ± 15.7	10.4 ± 15	-1.0 ± 9.7
Disruptive behavior
Control	36.2 ± 26.5	32.0 ± 26.0	-4.3 ± 18.0	24.7 ± 21.3	20.5 ± 19.8	-4.2 ± 15.4
Experimental	39.6 ± 24.4	31.6 ± 21.8	-8.0 ± 16.1	26.5 ± 20.5	22.5 ± 19.9	-4.0 ± 14.3
Engagement in activities
Control	59.0 ± 30.1	65.2 ± 29.0	6.2 ± 17.9	71.9 ± 26.1	77.2 ± 26.0	5.3 ± 15.7
Experimental	60.9 ± 28.4	66.1 ± 26.0	5.1 ± 16.2	70.4 ± 25.9	74.3 ± 24.7	3.9 ± 16.4
Socialization
Control	71.7 ± 19.4	75.4 ± 19.4	3.7 ± 14.6	76.2 ± 19.0	80.5 ± 18.1	4.3 ± 14.6
Experimental	70.2 ± 20.1	73.6 ± 19.7	3.5 ± 13.0	71.8 ± 19.4	76.5 ± 18.3	4.7 ± 14.2
Self-control
Control	55.0 ± 23.3	61.8 ± 25.5	6.8 ± 19.2	65.9 ± 22.5	73.5 ± 23.5	7.6 ± 19.1
Experimental	52.7 ± 22.3	61.6 ± 22.8	8.9 ± 17.9	62.2 ± 22.3	68.6 ± 22.5	6.4 ± 17.0

• Note.=405 and 476 respectively for the boys in the control and experimental groups. =385 and 465 respectively for the girls in the control and experimental groups. Mean ± SD

Since the first baseline, differences in behavior scores between boys and girls have already appeared in both groups. The boys systematically presented higher rates of aggressive and disruptive behaviors and lower engagement levels in activities, socialization, and self-control than the girls did, even before the intervention. Table 4 presents these results.

Table 4.
Estimates of GEE Models for the Aggressive and Disruptive Behavior Score for Boys and Girls: Elos Program Efficacy Evaluation in Brazilian Schools, 2016

	Boys (n=881)			Girls (n=850)
	Coefficient	SD		Coefficient	SD
Aggressive Behavior
Experimental Group	2.20	1.59	0.167	0.27	1.09	0.806
Time - post (ref.=pre)	-1.97	0.83	0.017	-1.02	0.57	0.076
Time x Exper Group	-3.05	1.06	0.004	-0.03	0.73	0.966
Grade (ref.=1st grade)
2nd grade	2.56	2.34	0.273	2.06	1.69	0.223
3rd grade	1.96	1.89	0.300	0.24	1.33	0.856
4th grade	-3.84	1.98	0.053	-5.32	1.23	<0.001
5th grade	0.90	3.58	0.801	-0.42	2.39	0.860
Disruptive Behavior
Experimental Group	3.23	1.73	0.062	1.92	1.45	0.185
Time - post (ref.=pre)	-4.27	0.89	<0.001	-4.21	0.79	<0.001
Time x Exper Group	-3.71	1.16	0.001	0.25	1.03	0.807
Grade (ref.=1st grade)
2nd grade	2.49	2.47	0.315	2.87	2.15	0.181
3rd grade	2.68	2.10	0.201	-0.33	1.66	0.841
4th grade	-0.38	2.27	0.868	-2.84	1.80	0.115
5th grade	-0.66	3.73	0.859	-5.89	3.15	0.061

• Nota. Exper = Experimental

Related to aggressive behavior, the effect of the Elos Program was significantly noticeable for the boys, who at the baseline already have profiles with lower psychosocial adequacy. After six months, the analysis indicated a reduction in the score (1.97 points less) in both the control and experimental groups (p = 0.017). However, in the experimental group, an additional reduction of 3.05 (p = 0.004) was noted. For the girls in both groups, however, this score did not vary over time.

Table 4 also reported disruptive behavior results and a significant decrease in boys who participated in the Elos Program. The experimental group and the control group showed the same reduction in this behavior for the girls, with no differences. The research detected a decrease of 4.27 points over time in both the control and experimental group for boys (p <0.001). However, in the experimental group, there was an additional reduction of 3.71 (p <0.001). For the girls, however, there was a reduction of 4.21 points over time in disruptive behavior in both the control group and the experimental group (p <0.001), with no additional variation observed in the experimental group (p = 0.807).

Regarding the three positive aspects of behaviors, engagement in activities, socialization, and self-control, there were no significant differences for the group that received the Elos Program compared to the control group. There were also no significant differences between the genders.

Discussion

The efficacy evaluation of the Elos Program through a non-randomized controlled trial suggested that the program might help decrease aggressive and disruptive behavior among boys. However, it did not present the same effect on girls. Boys showed higher rates of aggressive and disruptive behavior in the first moment, baseline, which might be an initial risk condition for developing antisocial behavior, with clear indications that they were the ones that benefited most from the intervention.

Several other international studies obtained similar results on GBG efficacy and effectiveness with a predominance of results for boys or children with higher initial risk patterns (Dolan et al., 1993; Ialongo et al., 1999; Ialongo et al., 2001; Kellam et al., 1998; Kellam et al., 2008; Kellam et al., 2011; Poduska et al., 2008; Poduska & Kurki, 2014). In GBG follow-up studies, it was verified by Kellam et al. (2011) that GBG reduced drug use, dependence, and high-risk sexual behavior among men that during intervention came from a higher risk profile group. Because these risky behaviors are usually much lower among females, the authors hypothesized that it could be the main reason for the lack of effect of GBG among women.

These results obtained in Brazilian schools reaffirm the discussion that the issue of gender and the question of initial risk pattern for aggressive and disruptive behaviors are significant moderators that act on the efficacy of the GBG program (Kellam et al., 2011; Poduska et al., 2008; Weis et al., 2015). On the other hand, based on these results, it is possible to question whether the Elos /GBG Program is, in fact, a "universal" prevention program. This program proved to be more beneficial for children who initially have a higher risk of antisocial behavior, vulnerabilities, and school difficulties, having a more consistent effect on boys, who have a cultural tendency to show higher rates of such externalizing behaviors.

It is essential to consider that the effectiveness of GBG measured through randomized controlled trials is not consensual in the literature, and there were several contradictions found in terms of gender, outcomes, and participating population (Smith et al., 2019). These results dialogue with the need to better understand the multiple variables' interaction effects that guarantee the program's effectiveness (Joslyn et al., 2019).

The relevance of this article is to measure the program's cultural adaptation effects in a low and middle-income context, as the Latin American countries. The need to balance cultural adaptations and fidelity to the program's core elements is even more challenging when implementing preventive actions via schools' pedagogical infrastructure in these countries. any adaptations were necessary but these discussions were out of the scope of this article's purpose.

This study has some limitations: due to requests of the BMH and Municipal Education Departments regarding the implementation of the program, this study lacks randomization, which may have potentially created an initial bias, as the individuals of the experimental group had more behavioral problems than those of the control group. The authors believe that the schools indicated more "difficult" or "complicated" classes to participate in the Elos Program, bringing an initial bias to the study. However, GEE analysis has attempted to deal with this issue.

Finally, Elos seems to be effective in its target results among boys. These positive results are added to the high acceptability teachers and principals demonstrated for the program by considering it a viable prevention program related to Brazilian schools' mental health. Nonetheless, there is a need for caution in generalizing the results obtained. The authors indicate the need to conduct a new study, with a Randomized Controlled Trial design, in order to assess the effectiveness and, once other studies confirm the results, it would be possible for the Elos Program to become an evidence-based preventive public policy for children's mental health in Brazil.

References

• Bayer, J., Hiscock, H., Scalzo, K., Mathers, M., McDonald, M., Morris, A., Birdseye, J., & Wake, M. (2009). A systematic review of preventive interventions for children's mental health: What would work in Australian contexts? Australian and New Zealand Journal of Psychiatry, 43(8), 695-710. https://doi.org/10.1080%2F00048670903001893
  » https://doi.org/10.1080%2F00048670903001893
• Barrish, H. H., Saunders, M., & Wolf, M. M. (1969). Good Behavior Game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied BehaviorAnalysis, 2, 119-124. https://doi.org/10.1901/jaba.1969.2-119
  » https://doi.org/10.1901/jaba.1969.2-119
• Bowman-Perrott, L., Burke, M. D., Zaini, S., Zhang, N., & Vannest, K. (2016). Promoting positive behavior using the Good Behavior Game: A meta-analysis of single-case research. Journal of Positive Behavior Interventions, 18(3), 180-190. https://doi.org/10.1177/1098300715592355
  » https://doi.org/10.1177/1098300715592355
• Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., ... & Lancet Early Childhood Development Series Steering Committee. (2017). Nurturing care: Promoting early childhood development. The Lancet, 389(10064), 91-102. https://doi.org/10.1016/S0140-6736(16)31390-3
  » https://doi.org/10.1016/S0140-6736(16)31390-3
• Catalano, R.F., & Hawkins, J.D. (1996). The social development model: A theory of antisocial behavior. In J.D. Hawkins(Ed.), Delinquency and Crime: Current Theories (pp 149-197). Cambridge University Press.
• Daelmans, B., Darmstadt, G. L., Lombardi, J., Black, M. M., Britto, P. R ., Lye, S.,...& Richter, L. M . (2017). Early childhood development: The foundation of sustainable development. The Lancet , 389(10064), 9-11. https://doi.org/10.1016/S0140-6736(16)31659-2
  » https://doi.org/10.1016/S0140-6736(16)31659-2
• Dolan, L. J., Kellam, S. G., Brown, C. H., Werthamer-Larsson, L., Rebok, G.W., Mayer, L. S., & Wheeler, L. (1993). The short-term impact of two classroom-based preventive interventions on aggressive and shy behaviors and poor achievement. Journal of Applied Developmental Psychology, 14, 317-345. https://doi.org/10.1016/0193-3973(93)90013-L
  » https://doi.org/10.1016/0193-3973(93)90013-L
• D’Tôlis, P. O. A. (2018). Avaliação da adaptação cultural do programa Elos - Construindo coletivos [Masters dissertation, Universidade Federal de Santa Catarina]. Repositório institucional da UFSC. https://repositorio.ufsc.br/handle/123456789/193636
  » https://repositorio.ufsc.br/handle/123456789/193636
• Embry, D. D. (2002). The Good Behavior Game: A best practice candidate as a universal behavioral vaccine. Clinical Child and Family Psychology Review, 5, 273-297. https://doi.org/10.1023/A:1020977107086
  » https://doi.org/10.1023/A:1020977107086
• Flower, A., McKenna, J., Bunuan, R., Muething, C., & Vega Jr, R. (2014). Effects of the Good Behavior Game on challenging behaviors in school settings. Review of Educational Research, 84(4): 546-571. https://doi.org/10.3102/0034654314536781
  » https://doi.org/10.3102/0034654314536781
• Ford, C. B., Keegan, N., Poduska, J. M., Kellam, S. G., & Litman, J. (2013). Good Behavior Game Implementation Manual American Institute for Research.
• Garcia, D. (2018) Avaliação da fidelidade de implementação de um programa preventivo em saúde mental infantil baseado na escola [Masters dissertation, Universidade Federal de Santa Catarina]. Repositório institucional da UFSC. https://pergamum.ufsc.br/pergamum/biblioteca/index.php
  » https://pergamum.ufsc.br/pergamum/biblioteca/index.php
• Hawkins, J. D., Jenson, J. M., Catalano, R., Fraser, M.W., Botvin, G. J, Shapiro, V. C., Brown, H., Beardslee, W., Brent, D., Leslie, L. K., Rotheram-Borus, M. J., Shea, P., Shih, A., Anthony, E., Haggerty, K. P., Bender, K., Gorman-Smith, D., Casey, E., & Stone, S. (2015). Unleashing the power of prevention [Discussion Paper]. Institute of Medicine and National Research Council. https://nam.edu/wp-content/uploads/2015/06/DPPowerofPrevention.pdf
  » https://nam.edu/wp-content/uploads/2015/06/DPPowerofPrevention.pdf
• Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599-641. https://doi.org/10.1023/A:1022137920532
  » https://doi.org/10.1023/A:1022137920532
• Ialongo, N. S., Poduska, J., Werthamer, L., & Kellam, S. G. (2001). The distal impact of two first-grade preventive interventions on conduct problems and disorder in early adolescence. Journal of Emotional and Behavioral Disorders, 9(3), 146-160. http://doi.org/10.1177/106342660100900301
  » http://doi.org/10.1177/106342660100900301
• Joslyn, P. R., Donaldson, J. M., Austin, J. L., & Vollmer, T. R. (2019). The Good Behavior Game: A brief review. Journal of Applied Behavior Analysis, 52(3), 811-815. https://doi.org/10.1002/jaba.572
  » https://doi.org/10.1002/jaba.572
• Kellam, S. G., Ling, X., Merisca, R., Brown, C.H., & Ialongo, N. (1998). The effect of the level of aggression in the first-grade classroom on the course and malleability of aggressive behavior into middle school. Dev Psychopathol, 10, 165-185. https://doi.org/10.1017/S0954579498001564
  » https://doi.org/10.1017/S0954579498001564
• Kellam, S. G., Brown, C. H., Poduska, J. M., Ialongo, N. S., Wang, W., Toyinbo, P., & Wilcox, H. C. (2008). Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes. Drug and Alcohol Dependence, 95(Suppl 1), S5-S28. http://doi.org/10.1016/j.drugalcdep.2008.01.004
  » http://doi.org/10.1016/j.drugalcdep.2008.01.004
• Kellam, S. G., Mackenzie, A. C., Brown, C. H., Poduska, J. M., Wang, W., Petras, H., & Wilcox, H.C. (2011). The Good Behavior Game and the future of prevention and treatment. Addiction Science and Clinical Practice, 6, 73-84. PMCID: PMC3188824
• Kellam, S. G., Wang, W., Mackenzie, A. C., Brown, C. H., Ompad, D. C., Or, F., & Windham, A. (2014). The impact of the good behavior game, a universal classroom-based preventive intervention in first and second grades, on high-risk sexual behaviors and drug abuse and dependence disorders into young adulthood. Prevention Science, 15(1), 6-18. http://doi.org/10.1007/s11121-012-0296-z
  » http://doi.org/10.1007/s11121-012-0296-z
• Koth, C. W., Bradshaw, C. P., & Leaf, P. J. (2009). Teacher observation of classroom adaptation-checklist: Development and factor structure. Measurement and Evaluation in Counseling and Development, 42(1), 15-30. https://doi.org/10.1177/0748175609333560
  » https://doi.org/10.1177/0748175609333560
• Ministério da Justiça (2016). Prevenção na escola: Guia do educador (Programa Elos: Construindo coletivos) Ministério da Justiça, Secretaria Nacional de Políticas sobre Drogas (SENAD). Disponível em http://bvsms.saude.gov.br/bvs/publicacoes/programa_elos_guia_educador.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/programa_elos_guia_educador.pdf
• Ministério da Saúde (2018). Prevenção ao uso de drogas: Implantação e avaliação de programas no Brasil Ministério da Saúde. Disponível em http://repositorio.unifesp.br/handle/11600/50961
  » http://repositorio.unifesp.br/handle/11600/50961
• Leflot, G., Van Lier, P., Onghena, P., & Colpin, H. (2013). The role of children's on-task behavior in the prevention of aggressive behavior development and peer rejection: A randomized controlled study of the Good Behavior Game in Belgian elementary classrooms. Journal of School Psychology, 51(2), 187-199. http://dx.doi.org/10.1016/j.jsp.2012.12.006
  » http://dx.doi.org/10.1016/j.jsp.2012.12.006
• Lorenzo, F., & Schneider, R. S. (2021). Subsídios para a implementação de programas de prevenção escolar nas séries iniciais. In S. Abreu, S.G. Murta, V.R. Rocha, & N. Pinheiro-Carozzo (Orgs.), A experiência brasileira de prevenção escolar e comunitária do uso de álcool e outras drogas: Registro histórico de adaptação, implementação e avaliação entre 2012 e 2016 (pp. 118-146). Rede Unida.
• Ministério da Justiça. (2016). Prevenção na escola: Guia do educador (Programa Elos: Construindo coletivos) http://bvsms.saude.gov.br/bvs/publicacoes/programa_elos_guia_educador.pdf
  » http://bvsms.saude.gov.br/bvs/publicacoes/programa_elos_guia_educador.pdf
• Ministério da Saúde. (2018). Prevenção ao uso de drogas: Implantação e avaliação de programas no Brasil http://repositorio.unifesp.br/handle/11600/50961
  » http://repositorio.unifesp.br/handle/11600/50961
• Monceau, G. (2008). Implicação, sobreimplicação e implicação profissional. Fractal Revista da Psicologia, 20(1), 19-26. http://doi.org/10.1590/S1984-02922008000100007
  » http://doi.org/10.1590/S1984-02922008000100007
• Nolan, J. D., Houlihan, D., Wanzek, M., & Jenson, W. (2014). The Good Behavior Game: A classroom-behavior intervention effective across cultures. School Psychology International, 35(2), 191-205. https://doi.org/10.1177/0143034312471473
  » https://doi.org/10.1177/0143034312471473
• Poduska, J. M., Kellam, S. G., Wang, W., Brown, C. H., Ialongo, N. S., & Toyinbo, P. (2008). Impact of the Good Behavior Game, a universal classroom-based behavior intervention, on young adult service use for problems with emotions, behavior, or drugs or alcohol. Drug and Alcohol Dependence, 95(Suppl 1), S29-S44. https://doi.org/10.1016/j.drugalcdep.2007.10.009
  » https://doi.org/10.1016/j.drugalcdep.2007.10.009
• Poduska, J. M., & Kurki, A. (2014). Guided by theory, informed by practice: Training and support for the good behavior game, a classroom-based behavior management strategy. Journal of emotional and behavioral disorders 22(2), 83-94. http://doi.org/10.1177/1063426614522692
  » http://doi.org/10.1177/1063426614522692
• Quinn, M. M., Jannasch-Pennell, A., & Rutherford, Jr., R. B. (1995). Using peers as social skills training agents for students with antisocial behavior: A cooperative learning approach. Preventing School Failure: Alternative Education for Children and Youth, 39(4), 26-31. https://doi.org/10.1080/1045988X.1995.9944639
  » https://doi.org/10.1080/1045988X.1995.9944639
• Schneider, D. R., Oliveira, V., Oltramari, L., Grigolo, T., Catão, E., Pereira A. P. D., Cruz, J. I., & Sanchez, Z. (2014). Relatório de avaliação do projeto piloto de implementação e adaptação cultural do jogo Elos em escolas brasileiras em 2013 [Manuscrito não publicado]. Universidade Federal de Santa Catarina, Universidade Federal do Estado de São Paulo.
• Schneider, D. R., Pereira, A. P. D., Cruz, J. I., Strelow, M., Chan, G., Kurki, A., & Sanchez, Z. M.. (2016). Evaluation of the implementation of a preventive program for children in Brazilian schools. Psicologia: Ciência e Profissão, 36(3), 508-519. https://doi.org/10.1590/1982-3703000592016
  » https://doi.org/10.1590/1982-3703000592016
• Schneider, D.R., Kaszubowski, E., Garcia, D., Scaff, L., Moraes-Cruz, R., and Sanchez, Z.M. (2020). Psychometric properties of a Brazilian adaptation of the Teacher Observation of Classroom Adaptation-Revised Scale for Children’s Behavior. Acta Colombiana de Psicología, 23(1), 320-332. https://doi.org/10.14718/ACP.2020.23.1.15
  » https://doi.org/10.14718/ACP.2020.23.1.15
• Smith, S., Barajas, K., Ellis, B., Moore, C., McCauley, S., & Reichow, B. (2019). A meta-analytic review of randomized controlled trials of the Good Behavior Game. Behavior Modification, 45(4), 641-666. https://doi.org/10.1177%2F0145445519878670
  » https://doi.org/10.1177%2F0145445519878670
• Sloboda, Z., & Petras, H. (2014). Defining Prevention Science Springer.
• Weis, R., Osborne, K. J., & Dean. E. L . (2015). Effectiveness of a universal, interdependent group contingency program on children's academic achievement: A countywide evaluation. Journal of Applied School Psychology, 31(3), 199-218. https://doi.org/10.1080/15377903.2015.1025322
  » https://doi.org/10.1080/15377903.2015.1025322
• Zeger, S. L., & Liang, K. (1986). Longitudinal data analysis for discrete and continuous outcomes.Biometrics,42(1), 121-130. https://doi.org/10.2307/2531248
  » https://doi.org/10.2307/2531248

• 1
  Details on the evaluation of cultural adaptation are presented in D’Tôlis (2018).
• 2
  Details on the Elos Program evaluation of fidelity are presented in Garcia (2018).

Publication Dates

• Publication in this collection
  17 Oct 2022
• Date of issue
  2022

History

• Received
  13 Jan 2021
• Accepted
  25 Apr 2021