Executive Functions in the Prediction of Academic Performance in Elementary Education^*

Funções Executivas na Predição de Desempenho Acadêmico no início do Ensino Fundamental

Abstract

The study investigated predictive models of reading and arithmetic based on performance and functional measures of executive functions (EF’s). Ninety-four children (mean age = 6.14y) were evaluated through EF performance tests. Parents and teachers responded to a functional measure of EF. Two years later, children’s reading and arithmetic were evaluated. There were low correlations between EF and later academic performance. For the reading model, inhibition was a relevant variable, with no gain in the prediction with the joint consideration of different EF measures. For arithmetic, attention and delay aversion presented a relevant contribution. When functional measure was considered, the model’s prediction increased. The study indicated skills of interest in identifying children at risk of poor academic performance.

Keywords
regulation; child evaluation; cognition; education; learning

Resumo

O estudo investigou modelos preditivos de leitura e aritmética a partir de medidas de desempenho e funcional de funções executivas (FE). Noventa e quatro crianças (M_idade= 6,14a) foram avaliadas por meio de testes de desempenho de FE. Pais e professores responderam a uma medida funcional. Dois anos depois, leitura e aritmética foram avaliadas. Houve baixas correlações entre FE e desempenho acadêmico posterior. Para o modelo de leitura, inibição foi variável relevante; não houve ganho na predição com a consideração conjunta das diferentes medidas de FE. Para aritmética, atenção e aversão ao adiamento apresentaram contribuição relevante. A consideração da medida funcional aumentou a previsão do modelo. O estudo indicou habilidades de interesse na identificação de crianças em risco de baixo desempenho acadêmico.

Palavras-chave
regulação; avaliação infantil; cognição; educação; aprendizagem

Recent years have been marked by growing interest in the study of a set of skills called executive functions (e.g. Diamond, 2013; Friedman & Miyake, 2017). Within the scope of these investigations, one area of interest has been the relationship or impact of these skills on learning or school performance, notably in the areas of reading and mathematics (Best et al., 2011; Corso et al., 2016; Purpura et al., 2017; Röthlisberger et al., 2013; Toll et al., 2011).

EF’s are a set of high-level cognitive skills (given their regulatory action over other processes) that enable individuals to control their actions, thoughts, and emotions (Friedman & Miyake, 2017). Based on previous evidence (e.g. Friedman & Miyake, 2017), Diamond (2013) suggested three basic EF’s: 1) inhibition, the ability to inhibit inappropriate behaviors (referred to as self-control, which includes the discipline to stay on task, and delay aversion/ gratification, forgoing an immediate pleasure for a greater reward later); it also includes the ability to inhibit thoughts or memories, as well as the ability to control the attention to distractors (also referred to as interference control), which is associated with selective attention; 2) working memory, the ability to mentally sustain and operate information; and 3) cognitive flexibility, the ability to take different perspectives, alternate between rules and adapt to change. Complex EF’s, such as planning or problem solving, would emerge from the integrated action of the basic EF. Furthermore, overlapping inhibition and including some hot aspects, as maintaining optimal levels of emotional and motivational arousal, are understood as regulation or, as in Diamond’s model, self-regulation (Diamond, 2013).

Evidence suggests that EF’s develop throughout childhood and adolescence. It also suggests that EF’s are an important component of academic success, with indications that EF’s are determinants to facilitate learning of diverse school content. Likewise, EF’s also relate to the ability to solve mathematical problems, for abstract reasoning and for social judgments and behavior (Blair & Razza, 2007; Diamond, 2013). In fact, some interesting evidence can be observed in a representative American sample study. Approximately half of Early Childhood Educators indicated that 50% or more of the children in their classrooms experience problems that substantially limit their ability to benefit from early schooling. Although some teachers identified poor academic skills in Early Childhood Education (such as letter or number knowledge) as a source of difficulty for children, more problems related to self-regulation, in particular problems with following instructions and controlling attention, were identified by the teachers as the causes of children not being prepared for school (Rimm-Kaufman et al., 2000).

More recent evidence indicates that EF in preschool age can predict engagement in learning situations during the transition to the 1^st grade of Elementary Education, assessed as learning related behaviors, such as organizing their activities and staying focused on tasks (Nelson et al., 2017).Thus, implication for education is evident, in the sense that knowledge about such abilities and how to assess them could provide teachers tools to address at least some kind of difficulties in their classrooms.

The relevance of EF for schooling, especially in the early years of elementary school, is so consistent that recent policies in Brazil have highlighted the importance of EF developing in students. For example, at the National Evidence-Based Literacy Conference, held in 2019, there was a thematic axis specifically dedicated to presenting evidence on the relationship between EF (cited as cognitive self-regulation) and literacy in the early years of Elementary Education (Brasil, 2019).

In fact, many studies corroborate this association of EF with learning and academic performance. For example, Best et al. (2011), in a sample of children aged five to 17 years old. observed generally moderate and consistent associations between complex EF measures (with planning demands) and academic performance throughout the age groups and different school measures. Their findings support the idea of EF’s as common processes and with contributions for both reading and mathematics. Other studies suggest more specific contributions.

Considering the specified disciplines, there is evidence for the role of EF in reading comprehension, especially in skills such as working memory, planning and inhibition/attentional allocation (Borella et al., 2010; Cutting et al., 2009; Kendeou et al., 2014), with EF also appearing to mediate the effect of socioeconomic level on reading comprehension of students from the 4^th to the 6^th year (Corso et al., 2016). Even in preschool children, EF’s relationship with skills of reading and writing isolated words is already evident (Pazeto et al., 2014). Despite fewer evidence of the role of flexibility on reading, a recent investigation shed light on this question suggesting that poor comprehenders are less able to actively switch focus between the phonological and semantic processes. In this way, an inflexible focus on word recognition could impair more elaborate processes involved in meaning comprehension (Cartwright et al., 2017).

Similarly, the association between performance in mathematics (more specifically arithmetic) and EF at different ages has been consistently highlighted in literature (Bull & Lee, 2014; Gathercole et al., 2004;, Röthlisberger et al., 2013; Toll et al., 2011). Despite that, recent evidence shows, in general, working memory as the main predictor of mathematical performance, with less conclusive findings regarding the other abilities (Bull & Lee, 2014). A point to be highlighted is that most studies in this area are cross-sectional, with little longitudinal evidence (e.g., Toll et al., 2011). In Toll et al.’s study, for example, working memory assessed at the beginning of 1^st grade has been revealed as an important predictor of math difficulties at the end of 1^st and 2^nd grades.

Another example of longitudinal evidence is from Röthlisberger et al. (2013). Authors followed preschool children for three years and found that the EF assessed during the preschool stage (unlike us, the authors used an EF composite score derived from three performance tasks which measure inhibition, working memory and flexibility) predicted substantial variability in mathematical performance (from 9% to 25% of the variance, depending on the cohort and measures considered), reading (from 18% to 24% of the variance, depending on the cohort and measures considered) and writing (from 12% to 17% of the variance, depending on the cohort considered) two years later. Analysis of academic performance profiles according to the previous performance in EF showed that early difficulties in the latter were associated with an academic disadvantage in the initial years of formal schooling. Thus, longitudinal investigations in this area can contribute to the identification of early markers, so as to identify children at risk of learning difficulties in the years to come in Elementary Education (Diamond, 2013; Purpura et al., 2017; Röthlisberger et al., 2013), as well as informing early intervention studies (Dias & Seabra, 2017).

Another point of interest is that most studies performed in this area have used performance tests in the assessment of EF’s. There are some common tasks for EF assessment, such as Go-NoGo tasks (in which a child must emit a response to a target stimulus, but inhibit it in the face of a non-target stimulus) or the Stroop paradigm (in which the child must avoid an automatic response, as reading a word, to emit a less automatic response, as naming the color) (see Dias & Malloy-Diniz, 2020, for a summary of instruments in Brazil). However, there are some criticisms about the ecological validity of EF performance measures (Barkley, 2014; Chaytor et al., 2006; Naglieri & Goldstein, 2014).

One option to these more structured tasks would be the use of functional measures, such as inventories that, in the case of children, can be completed by parents and/or teachers. As already documented with other behavioral inventories (Major & Seabra-Santos, 2014; Seabra-Santos & Gaspar, 2012), it is known that the relationships between EF scales and performance measures, as well as the agreement between respondents in the scales, are generally low, at the most moderate, leading to the conclusion that measures possibly do not measure the same aspects of EF and should be considered complementary (Barkley, 2014; Martoni et al., 2016; Thorell & Catale, 2014; Toplak et al., 2013).

In this sense, this study integrated two questions, that is, (1) we used both types of EF measures, performance and functional, aiming to investigate (2) predictive models of academic performance in a longitudinal study. Specifically, our objective was to investigate predictive models of reading and arithmetic in the 2^nd and 3^rd year of Elementary Education based on the performance and functional measures (as reported by parents and teachers) of EF in a two-year longitudinal study.

Method

Participants

Participants of the study were 94 children (55.3% girls) aged between five and seven years at the start of the study (age in years: M=6.14; SD=0.65; age in months: M=78.94; SD=6.44), recruited from three public schools. In the first year of the study, 42 were enrolled in the final year of Early Childhood Education (Pre-II, age: M=5.67; SD=0.48, 61.9% being girls) and 52 in the 1^st year of Elementary Education (EE) (age: M=6.52; SD=0.51, 50% being girls). There were no children with indicators of intellectual deficiencies (according to the Columbia Mental Maturity Scale; Alves & Duarte, 2001), neurological/psychiatric syndromes or disorders, sensory or motor changes that affected the response to the instruments (according to school records). All 94 children were re-evaluated two years later, when they attended the 2^nd and 3^rd year of EE. All children were speakers of Portuguese as their first language.

The participants’ schools were in medium-low socioeconomic status neighborhoods in a city of the metropolitan region of São Paulo (with approximately 267 thousand inhabitants), Brazil. Household income of 80% of the families was U$ 1,330.00; 20.8% of children’s fathers and 12.5% of their mothers had college degrees.

Instruments

Instruments used at moment 1 (Pre-II and 1^st years)

Columbia Mental Maturity Scale (CMMS; Alves & Duarte, 2001). CMMS evaluates children’s nonverbal reasoning. Children are presented with boards containing 3 to 5 drawings each. Their task is to choose which drawing is different or does not relate to the others. The estimated time of application is 20 minutes. The result in percentiles was used, for both the identification of children with indications of intellectual disability (percentile <4) and for the control in the regression analysis. There are data of reliability and evidence of validity acceptable for the Brazilian population available in the manual.

Simon Task (Trevisan, 2010). This task evaluates EF, it is computerized and consists of three parts, each one with 20 items. In part 1, a figure of either a frog or a butterfly is presented on each screen, randomly and on both sides of the screen. At the bottom of the screen, there are two answer buttons, one with the butterfly figure and the other with the frog figure. The child’s task is to select the button with the same stimulus figure, regardless of which side it appears on. In part 2 of the test, an arrow is presented on each screen, which can point to the left or right and can be on the left or right side of the screen. At the bottom of the screen there are 2 buttons, one on the left and one on the right. The button on the side the arrow points to must be pressed. Part 3 of the test is composed of three conditions, in the first one, a striped circle is displayed on either side of the screen, and the task is pushing the button on the same side the circle appears. In the second condition, a gray circle is displayed on either side of the screen, and the task is pushing the button on the opposite side. In a third condition, there are striped and gray circles; the button must be pushed on the same side for the striped circles and the button on the opposite side for the gray circles. For all items in the three parts of the test, the presentation time is 2500 milliseconds (ms), and the interval between stimuli is 500ms.

The instrument allows four indices to be assessed, both in terms of score (all in the range between 0 and 1) and reaction time (RT), used in the present study: performance in congruent type items (example: left arrow pointing to the left), a simpler measure of selective attention; performance in incongruent type items (example: left arrow pointing to the right), measure of inhibitory control; flexibility index (rule alternation: items that are preceded by the opposite type, that is, congruent preceded by incongruent, or incongruent preceded by congruent) and control index (rule constancy: items with the same rule as the previous ones). Evidence of validity can be found in Trevisan (2010).

Cancellation Attention Test (CAT; Montiel & Seabra, 2012). CAT evaluates the attention ability. It is based on the cancelation paradigm, where there is a target stimulus and a matrix with different stimulus (a page with 300 stimuli). Participants should cancel only the ones equal to the target one. In the first part, a target stimulus (a geometric form) among the other distractors must be cancelled. The same is requested in the second part; however, the target stimulus is formed by two geometric shapes. In the final part, the target stimulus changes with each line. Performance was measured by the number of correct responses (target stimulus cancelled correctly) in the three parts of the test (range between 0 - 109). There is a 1-minute time limit for each part. The instrument has validity evidence and standards available (Seabra & Dias, 2012).

Trail Making Test for preschool children (TMT-PC; Trevisan & Seabra, 2012). This task evaluates the ability of cognitive flexibility. In condition A (control), the child must connect the stimuli (images of five puppies) in order of size. In condition B, images of bones the same sizes as the dogs are introduced, and the child must match the puppies with their appropriate bones, in order of size, connecting them alternately. Performance was measured from the score in sequences (i.e., items connected correctly in an unbroken sequence) in part B of the test (range between 0 - 9). The application is individual, with an average duration of 10 minutes. Evidence of validity and normative data are published for preschool children (Seabra & Dias, 2012a).

Semantic Stroop Test (SST; Trevisan, 2010). This version of Stroop task evaluates the ability of inhibitory control. It is computerized and divided into two parts, with 16 items each. In the first part, the child should name the images displayed on the screen one by one (boy, girl, moon and sun) and, in the second, they should say the opposite noun of the semantic pair (Example: say “boy” for the image of a “girl”). Interference performances (performance in part 2 minus performance in part 1 of the test), both in terms of score (range between -1 and 1) and reaction time (RT), were used. The application is individual, with an average duration of 15 minutes. Evidence of validity can be found in Trevisan (2010).

Inventory of Difficulties in Executive Functions, Regulation and Delay Aversion (IFERA-I; Trevisan et al., submitted). IFERA-I is a functional scale that evaluates the EF in day-to-day situations of a child through the observation of parents and teachers. It consists of 28 items, on a Likert-type scale from 1 to 5, divided into 5 subscales: Working Memory - WM (five items; range between 5 and 25; e.g., The child has difficulty remembering the several steps of a task or an activity. For example, in a school task, they forget some steps), Inhibitory Control - IC (six items; range between 6 - 30; e.g.: The child does things without thinking first about what can happen), Cognitive Flexibility - CF (five items; range between 5 - 25; e.g., The child has difficulty finding a new or different way to solve a problem when they have no way out), besides some more hot aspects related with EF, as Delay Aversion - DA (5 items; range between 5 - 25; e.g., When the child wants something, they expect it to happen immediately) and State Regulation - RE (seven items; range between 7 - 35; e.g., The child has trouble starting an uninteresting task, they need help or more time for it). Higher scores in the IFERA-I indicate greater difficulty for the child in the different domains. Scores of each subscale and of the total (range between 28 and 140) were used. The instrument has good psychometric properties in the Brazilian context (Trevisan et al., submitted). In a recent study, IFERA-I’s domains have been revealed as important predictors of behavior indices in children aged three to six (Dias et al., 2017). For our sample, IFERA-I showed appropriate reliability, with Cronbach’s Alpha of .96 for teachers’ responses and .83 for parents’ responses.

Instruments used at moment 2 (2^nd and 3^rd years)

Word and Pseudoword Reading Competence Test (WPRCT; Seabra & Capovilla, 2010). This measure evaluates children’s recognition of isolated words. It has 70 items, each one consisting of an image and a written element (either a word or pseudoword). Children must read the words silently and then judge whether the written word corresponds to the image (thus, throughout the article, ‘reading’ refers more specifically to the word recognition component). The application lasts approximately 30 minutes. Data on the psychometric and normative characteristics are provided in the test manual. The total score was used (range between 0 - 70).

Arithmetic Test (AT - Seabra et al., 2013). This test contains six subtests that allow the evaluation of aspects of numerical processing and calculation, including reading and writing of numbers, writing of increasing and decreasing numerical sequences, comparing numerical magnitude, calculation of operations presented in writing and orally (including the four arithmetic operations), and word problem solving (one for each arithmetic operation). The estimated time of application is 25 minutes. The total score was used (range between 0 - 60). Evidence of validity, reliability and normative data are reported in the study of Seabra et al. (2013).

Procedure

After the study was approved by the Research Ethics Committee (CAAE no. 0001.0.272.000-11), the participating institutions were contacted, with the consent forms subsequently signed. The study was presented in a meeting with children’s parents or acting parent, and the consent forms were signed.

During moment 1 of the study, the authorized children gave their own consent and were evaluated in their schools and during a regular period, individually, with the Simon, CAT, TMT-PC, SST and CMMS instruments. Data collection occurred in five sessions with a mean duration of 15 minutes each. The parents and teachers were asked to respond to the IFERA-I (with a 96.8% teacher return and 79.8% parent return). This stage of collection occurred at the end of the school year, moment 1of the study (October and November), when the children attended the Pre-II and 1^st year. After 2 years, moment 2 started (between October and November). The same children (now in the 2^nd and 3^rd year) were assessed with AT and WPRCT, collectively, in two 30-minute sessions.

Data analysis

Descriptive statistics of children’s performance in the EF tests (Pre-II and 1^st years) and reading and arithmetic tests (2^nd and 3^rd year) were calculated. Initially, in order to decide whether the measures should be considered together as a single EF measure, it was verified if there was a high correlation among several EF measures (parents and teachers’ reports and tests scores). Therefore, partial correlations analysis were conducted, controlling for age (in months), between the EF tests and parents and teachers’ responses to the IFERA-I (results not presented in the article). This procedure revealed different patterns of correlations, generally low, or at the most moderate, and guided the decision to continue the other analysis considering the performance and functional measures, as well as the responses of both respondents (correlations inter-responders were in the range between .36 and .44, all significant), separately. Consecutively, a partial correlation analysis was conducted, controlling by age, between children’s performances in the EF measures and reading and arithmetic measures two years later, as well as between the responses of parents and teachers to the IFERA-I and children’s performances in reading and arithmetic two years later.

In order to test prediction models, a hierarchical multiple linear regression analysis was conducted for each outcome (reading and arithmetic), including age and nonverbal reasoning as predictor variables in the first step and children’s performances in the EF tests (sequences in part B of the TMT-PC; total correct responses in the CAT; score and RT in Part 2 of the SST; interference score and RT in the SST, score and RT for the congruent and incongruent items and the flexibility and control indexes in the Simon) in the second step. The Stepwise variable selection method was chosen, so that the analysis itself selected the relevant variables for each model. Subsequently, this procedure was repeated. However, the predictor variables inserted in the second step were the IFERA-I indices (IC, WM, CF, DA, RE and total), answered by parents and teachers. Again, the Stepwise variable selection method was used. Based on the previous regressions results, a new regression analysis was conducted for each outcome. The predictor variables included in the first block were age and nonverbal reasoning and, in the second block, the indices that composed the previous models for reading (interference score in the SST, RT in the Flexibility index of the Simon and the WM subtest of the IFERA-I answered by teachers) and arithmetic (scores in the CAT, congruent item scores of the Simon and the DA subtest of the IFERA-I answered by teachers). The Enter method for selection of variables was used. The design allowed the verification of prediction models from performance tests, a functional measure and whether there was gain in the prediction from the joint consideration of the measures.

Results

Table 1 shows the sample's performance in the EF measures (Moment 1: Pre-II and 1^st year) and reading and arithmetic (Moment 2: 2^nd and 3^rd year). Considering the IFERA-I, answered by parents and teachers, Table 1 presents the means obtained by both respondents.

Table 1.
Descriptive Statistics of the Performances in the EF Tests and of the Scores of Parents and Teachers in the IFERA-I at moment 1 (Pre-II and 1^st year) and of Reading and Arithmetic at moment 2 (2^nd and 3^rd year)

Performance tests	M	SD		Report: IFERA-I		M	SD
Measures at Moment 1
CMMS (percentile)	80.15	20.45		IC	Teachers	2.42	0.88
TMT-PC - part B sequences	3.56	2.24			Parents	2.61	0.82
CAT - total correct responses	53.80	15.00		WM	Teachers	2.23	0.91
SST - interference score	-0.03	0.06			Parents	2.12	0.87
SST - interference RT	0.35	0.20		CF	Teachers	2.29	0.73
Simon - congruent items	0.92	0.11			Parents	2.33	0.88
Simon - incongruent items	0.93	0.09		DA	Teachers	2.41	0.86
Simon - Flexibility	0.93	0.10			Parents	2.97	1.08
Simon - Control	0.92	0.13		RE	Teachers	2.40	0.86
Simon - congruent items RT	1.50	0.38			Parents	2.69	1.04
Simon - incongruent items RT	1.44	0.41		Total	Teachers	2.35	0.81
Simon - Flexibility RT	1.45	0.37			Parents	2.56	0.79
Simon - Control RT	1.57	0.53
Measures at Moment 2
WPRCT	59.86	8.16
AT	33.60	11.33

Table 2 shows the relationships between children’s performance in the EF tests and, two years later, in the reading and arithmetic measures. Reading performance (two years later) was related, with low magnitude, positively with SST interference score and negatively with the RT in the Simon’s flexibility index. A higher number of relations, also of low magnitude, was observed with future performance in arithmetic, positive with sequences in part B of the TMT-PC, correct responses in the CAT and scores in all the indexes of the Simon Task; and negative with RT in the incongruous items of the Simon, in addition to a marginally significant trend, also negative, with RT in the Simon’s flexibility index.

Table 2.
Matrix of Correlations between EF Measures (performance tests and functional measure) at Moment 1 and in the Reading and Arithmetic Measures at Moment 2, controlling by age (in months)

Measures of EF performance
		TMT-PC-seqB	CAT	SST-interf	SST-interf-RT	Simon-Con	Simon-Incon	Simon-Flex	Simon-Cont	Simon-Con RT	Simon-Incon RT	Simon-Flex RT	Simon-Cont RT
WPRCT	r	-0.09	0.14	0.29**	0.02	0.12	0.05	0.05	0.02	-0.09	-0.18	-0.22*	-0.04
AT	r	0.27**	0.40***	0.14	0.03	0.36***	0.35***	0.33**	0.21*	-0.09	-0.30**	-0.20 #	-0.15
Functional measure of EFs
		IFERA-I - Teachers						IFERA-I - Parents
		IC	WM	CF	DA	RE	Total	IC	WM	CF	DA	RE	Total
WPRCT	r	-0.20 #	-0.24*	-0.08	-0.15	-0.13	-0.14	-0.03	-0.18	-0.04	0.02	0.01	-0.03
AT	r	-0.13	-0.27**	-0.20 #	-0.29**	-0.24*	-0.16	0.01	-0.11	-0.12	-0.12	-0.08	-0.12

• Notes.^*p < .05 / ^** p < .01 / ^*** p < .001 / # p > .05 < .065; TMT-PC-seqB: score in sequences in part B of the Trail Making Test for preschool children; CAT: total number of correct responses in the Cancellation Attention Test; SST-interf: interference score in the Semantic Stroop Test; SST-interf-RT: interference reaction time in the Semantic Stroop Test; Simon-Con: performance in the congruent type items of the Simon Task; Simon-Incon: performance in the incongruent type items of the Simon Task; Simon-Flex: flexibility index of the Simon Task; Simon-Cont: control index of the Simon Task; Simon-Con RT: reaction time in the congruent type items of the Simon Task; Simon-Incon RT: reaction time in the incongruent type items of the Simon Task; Simon-Flex RT: reaction time in the items that present rule alternation in the Simon Task; Simon-Cont RT: reaction time in the items that present rule constancy in the Simon Task; WPRCT: Word and Pseudoword Reading Competence Test; AT: Arithmatic Test.

Table 2 also presents the relations between the responses of parents and teachers to the IFERA-I and children’s performance in reading and arithmetic measures. It must be remembered that higher scores in the IFERA-I indicate greater difficulty for the child in each domain. Considering the teachers’ responses, there was a negative relation of low magnitude between the WM index and future reading performance and a marginal trend in the same direction for the IC index. The WM, DA, RE and marginally CF indices presented low magnitude negative relations with the performance in arithmetic two years later. There was no relation between parents’ responses and children’s future performance.

Considering only children’s performance in the EF tests, in addition to controlling by age and nonverbal reasoning, explanatory models of reading and arithmetic were tested. All the models had a satisfactory fit (p<0.001). Regarding reading, age and nonverbal reasoning were able to explain 11.2% of the variance in the WPRCT performance. Also, the inclusion of the EF measures (specifically measures of inhibition and flexibility, interference score in SST and RT in the flexibility index of the Simon, respectively) raised the predictive power of the model to 23.1%, overriding the contribution of age and nonverbal reasoning. For arithmetic, 34% of the variance in the AT performance was explained by age and reasoning, and the inclusion of EF measures (especially attention, i.e., CAT and congruent items from Simon) increased the explanatory power of the model to 46.3%, overriding the contribution of reasoning. Age continued to have a significant contribution to the model.

The same procedure was repeated considering the responses of parents and teachers to the IFERA-I. Again, all models had a satisfactory fit (p<0.005). After the step of controlling by age and reasoning, it was verified that the inclusion of the WM index, according to the teachers’ responses, was able to contribute, in a modest but significant way, to the reading model, increasing its explanatory power to 13.4%. Similarly, the DA index, also according to the teachers’ responses, increased the explanatory power of the arithmetic model to 41.3%.

From the results of the regressions conducted with the measures of the performance tests and the IFERA-I, new analyses were conducted, inserting as predictor variables the indexes that composed the previous models for each outcome, including performance measures and reports. The resulting models are shown in Table 3. For reading (fit of the model with p<0.001), there was no gain in the prediction in relation to the first model tested (only with the performance tests), with the highest regression index being associated to the interference score in the SST, and the contributions of the Simon and WM of the IFERA-I becoming marginally significant. In turn, for the arithmetic model (fit of the model with p<0.001), adding the performances in the tests and teachers’ responses to the IFERA-I (the DA index) led to an explanatory power of 57.2% of the variance.

Table 3
Prediction Models of the Performance in Reading and Arithmetic, with control of Age (in months) and nonverbal Reasoning, from the Variables selected between children’s Performances in EF tests and Responses of Parents and Teachers to the IFERA-I

Reading
	Beta	t	p	R2	Adjusted R2
(Constant)		3.712	0.000	0.276	0.233
Age	0.216	2.139	0.035
Nonverbal reasoning	0.134	1.396	0.166
SST - interference score	0.281	2.986	0.004
Simon - Flexibility RT	-0.184	-1.818	0.073
IFERA-I - WM - Teachers	-0.176	-1.823	0.072
Arithmetic
	Beta	t	p	R2	Adjusted R2
(Constant)		-3.801	0.000	0.596	0.572
Age	0.380	4.788	0.000
Nonverbal reasoning	0.036	0.481	0.632
CAT	0.326	4.003	0.000
Simon - congruent items	0.285	3.745	0.000
IFERA-I DA- teachers	-0.155	-2.081	0.040

Discussion

The study aimed to investigate predictive models of reading and arithmetic skills in children in the 2^nd and 3^rd year of EE based on EF performance and functional measures (as reported by parents and teachers) in the preschool and 1^st year of EE. In agreement with literature, the results also revealed low magnitude relations between EF performance tests and the responses of parents and teachers to the IFERA-I (Barkley, 2014; Martoni et al., 2016; Thorell & Catale, 2014; Toplak et al., 2013), as well as low concordance between the respondents in the inventory (Major & Seabra-Santos, 2014; Martoni et al., 2016; Seabra-Santos & Gaspar, 2012), justifying the use of each of those measures (instead of a composite score) in the analysis.

Several significant relations were evidenced between children’s performance in the EF tests and scores attributed to the IFERA-I by the teachers and the reading and arithmetic skill levels two years later. These results indicate a clear trend that preschool and 1^st years children who show better performance in EFs also tend to present better academic performance in future grades, which is consistent with literature (Blair & Razza, 2007; Diamond, 2013; Röthlisberger et al., 2013; Toll et al., 2011).

Considering the performance tests, in all cases, children with better scores in the corresponding EF test indices and with faster RT tended to present better performance in reading and arithmetic. Specifically, although the performance in reading also established a relation with the EF measures, a greater number and more consistent relations were found with the arithmetic test indices. In general, reading performance was associated with previous performance in a measure of inhibition (control of interference) and flexibility. Arithmetic was associated with previous performances in attention, flexibility and inhibition (inhibition of response).

It is possible to hypothesize about the role of such processes in reading and arithmetic performance. For example, inhibition has a role in preventing impulsive behavior but also in the control of attention, allowing the selection of important information and protecting working memory content by suppressing distractors. Also, inhibition can allow suppression of competing responses. These processes help keeping a child on the task and probably allow more effective processes in working memory. About flexibility, it is possible that competences such as arithmetic or even word recognition require some different processes; therefore, being able to switch between them effectively could allow a better performance. For example, for arithmetic, the child must recognize numbers or signs and, in the same task, apply calculation procedures or, another example, translate a word problem in an arithmetic operation. For reading, even for word recognition, processes for letter recognition and decoding occur at the same time with those involved in semantic processing.

Considering the functional measure, no association with reading and arithmetic performance was observed when considering parents’ responses, which may suggest that teachers’ evaluation was more sensitive and discriminative (Martoni et al., 2016). Given the teachers’ responses, the children evaluated in the preschool and 1^st year with greater working memory difficulties and, marginally, inhibitory control difficulties, tended to perform worse in reading two years later. Again, more relations were observed with performance in arithmetic, suggesting that children with greater difficulties in working memory, delay aversion, regulation, and, marginally, flexibility, are more prone to difficulties in arithmetic two years later. The greater number of associations evidenced with the arithmetic measure in comparison to the reading measure may be linked to the demands involved in each of these tasks. That is, we indeed expected more relations with arithmetic performance, once our task included not only number processing but also calculation, which demands applying procedures, performing operations, and even solving written problems. All of these require collecting relevant information, choosing strategies, conforming on rules, staying on task, and manipulating information.

Hence, when considering some findings, such as those of Best et al. (2011), who argue that EF are common processes with contributions for both reading and mathematics, it should be taken into consideration that these authors used complex EF measures (they focused on the Planning scale of the Cognitive Assessment System, which require the child to create, apply and monitor the effectiveness of a plan) and that the use of more specific measures on this construct can help elucidate associations and specific demands (e.g. Purpura et al., 2017). In an attempt to better understand these demands and from this first analysis, the study tested predictive models.

Considering reading, children’s performances in indices of inhibition and flexibility showed a significant contribution (accounting for 23.1% of the variance in the outcome, controlling for age and nonverbal reasoning). In turn, scores attributed by the teachers to the working memory index also had a significant contribution to the model (explaining 13.4% of the variance in the outcome, controlling by age and nonverbal reasoning). However, the combined consideration of children’s performance and teachers’ responses to the IFERA-I did not lead to an increase in the prediction (23.3%), with the contribution of the flexibility performance measure and the functional measure of working memory being only marginally significant and the interference control measure being the most relevant to the model. This finding suggests that the consideration of functional measures did not provide an incremental contribution to the variance already explained by the performance measures in reading prediction.

This result may be related to the reading measure used, which evaluated more specific processes of word recognition. It is recognized that there is a greater demand for EF in more complex measures, such as reading comprehension (Borella et al., 2010; Corso et al., 2016; Kendeou et al., 2014). However, as our sample was in the early years of EE, it was judged to be more pertinent to evaluate more basic processes of reading, hence the selection of the ability to recognize words. Interestingly, even though word recognition and reading comprehension are distinct processes (although consistently associated, e.g. r=0.76 and p<0.001 according to Seabra & Dias, 2012b in a study with Brazilian children) the results of the present study are in line with those of Borella et al. (2010), which evidenced the role of inhibition in the processes of reading comprehension. Specifically, Borella et al. found that poor comprehenders have problems with the elimination of no longer relevant information from memory. Our results suggest that executive functions, and specifically inhibition here, may also play a role in the most basic word recognition processes. This understanding is supported by the findings of Purpura et al. (2017) who, in a sample of preschool children, found an association between inhibition and skills such as knowledge of letters and their sounds and discrimination of letters and words.

The specific role inhibition has for word recognition (or more specifically for our measure of word recognition) can only be postulated. Analogue to Borella et al. (2010), it is possible that inhibiting no relevant information and maintaining as active only the relevant ones to answer the item is also important even for more basic processes as word recognition, at least at the moment when such ability is not automatized yet, as probably in the case of our children. Also, interference control is related with the attentional focus control, very demanded in our task. For example, some items have semantic (as the correct word RADIO but paired with an image of a telephone) or visual changes (as the Kangaroo item: It is written ‘CANCURU’; the correct spelling is CANGURU) that may go unnoticed for a more inattentive reader.

In turn, considering the performance in arithmetic, children’s performance in attention indices increased the predictive power of the model to 46.3% (a gain of 12.3% in relation to that explained only by age and nonverbal reasoning). In a very similar way, the scores in the delay aversion index based on teachers’ responses also contributed to an explanation of 41.3% of the variance in arithmetic. However, unlike what was observed in reading, the joint consideration of children’s performance and scores attributed by the teachers increased the explanatory power of the model even more, overriding the contribution of reasoning. That is, in the case of arithmetic performance, the delay aversion index of the IFERA-I had an incremental contribution to children’s performance in the attention indices, reaching an explanatory model of almost 60%. It is likely that the delay aversion measure, considering the fact that it established a higher relation with performance in arithmetic, is mediating the effects of other indices of the IFERA-I. Therefore, it cannot be said that other skills do not contribute to the arithmetic performance, but rather that their contribution should be included in the more robust/relevant contribution of the delay aversion index.

Thus, performances in attention and the evaluation by the teacher of delay aversion seem to be relatively sensitive indices of arithmetic performance two years later. In part, this result corroborates the data of Purpura et al. (2017) who, in a preschool sample, found that performance in inhibition measures were associated with most measures of mathematics. It should be remembered that attention is considered an aspect of inhibition, insofar as it enables control of the attentional focus and inhibition of distractors (Diamond, 2013). Similarly, delay aversion has been considered in some studies as an inhibition index (ability to inhibit an immediate temptation in favor of something more important) (e.g., Schoemaker, Mulder et al., 2013). In this way, children who are more capable of focusing their attention may be more effective in selecting relevant information and staying on task despite the fact that no immediate reward revealed more successful in the arithmetic task.

In general, the results for reading are consistent with the previous finding of a longitudinal study that; however, used a composite index of EFs. In the present study, predictions for reading between 13.4% and 23.3% were established. The findings of Röthlisberger et al. (2013) were able to explain between 18% and 24% of the variance in measures of reading speed and comprehension. For mathematics, however, the present study findings extrapolate previous findings (in the study by Röthlisberger et al., the prediction ranged from 9% to 25% of the variance, depending on the cohort and measures considered).

A point worth mentioning is about the differential predictive capability of parents and teachers. One hypothesis for such finding could be related with the nature of the IFERA-I’s items. In this sense, items capture some behavior samples that could manifest in different ways in different contexts (home versus school) or it could be the case that, due to demands of school environment, teachers may be in best conditions to observe and rate such behaviors (see examples of IFERA-I’s item in the Method section). Suchlike discussion is also boosted by Martoni et al. (2016). Also, literature supports that functional measures can capture different aspects from those assessed by performance tests (Barkley, 2014; Naglieri & Goldstein, 2014). In this way, we can infer that the role of EF is not just to support the cognitive operations underlying academic performance (as more directed assessed by performance tests), but also to enable the child to benefit from instruction by allowing them to pay attention during class, to switch comfortably between tasks, to be able to wait for their turn and stay on task, for example. These can be referred to as learning-related behaviors, which can facilitate learning diverse school content, and can possibly even mediate EF effects on learning (Nelson et al., 2017).

The study has limitations, such as the number of participants, which made it impossible to separate analysis by school level, which perhaps would have allowed more refined performance predictions. Other limitations refer to the sample by convenience from public schools only and the absence of other controls, such as socioeconomic level, which could also be a variable of impact on the outcomes of interest. It is worth noting that our conclusions are limited by the specific measures we use, mainly in the case of our finding for reading. That is, the potential of the study to draw conclusions on reading competence is quite limited, and one should consider that the specific contribution of the study in this area is in relation with a singular reading component, word recognition. Nevertheless, given the educational level we evaluate and reading proficiency in Brazilian children, this appears to be the most appropriate component to investigate. On the other hand, our arithmetic measure seems to be a more complete one in the sense it covers number processing and calculation domains. Future research could look deeper into such components and investigate EF specific contributions to number processing abilities, such as counting, or more complex skills, such as calculation, and word problem solving, for instance.

Despite these limitations, the originality of the study in the use of different types of measures (performance and functional) and verification of its complementarity in the investigation of academic performance predictive models are highlighted. The scarcity of longitudinal design predictive studies in the national context should also be highlighted. Future studies should investigate EF predictive power on more complex academic outcomes, such as reading comprehension, and throughout more advanced school levels and with other measures, including neurobiological ones.

Finally, the findings allow us to draw some implication for the field of EF measurement in children. Indeed, there is some substrate to affirm that performance tests and our functional measure are not assessing the same aspects of EF (Barkley, 2014; Naglieri & Goldstein, 2014; Martoni et al., 2016; Thorell & Catale, 2014; Toplak et al., 2013). Performance tests capture underlying specific processes whereas functional measures allow looking at global aspects of functioning in contexts of interest, including such learning-related behaviors. When predicting children’s learning outcomes both should be considered, above all for more complex outcomes (as our arithmetic measure instead of a more basic word recognition one). For parsimony, and specifically for predicting academic performance, teachers’ reports should be preferred rather than parents’.

Also, our results dialogue with educational practices to inform the relevance of considering EF in childhood and understanding the relations between such abilities and learning, and schooling process. This is one of the objectives of the so-called School Neuropsychology (Fonseca et al., 2020).

The results illustrate the predictive power of EF on performances in reading and arithmetic two years later, overriding the contribution of nonverbal reasoning, corroborating the role of these skills in learning and academic performance (Blair & Razza, 2007; Diamond, 2013; Rimm-Kaufman et al., 2000). The findings highlight the incremental value of using functional measures for the specific prediction of arithmetic performance. The results also indicate skills of interest in the preschool and 1^st year of EE that can be monitored and evaluated (as well as stimulated) for the early identification of children at risk of poor academic performance.

References

• Alves, I., & Duarte, J. (2001). Escala de Maturidade Mental Columbia [Columbia Mental Maturity Scale] Casa do Psicólogo.
• Barkley, R. A. (2014). The Assessment of Executive Functioning Using the Barkley Deficits in Executive Functioning Scales. In S. Goldstein & J. A. Naglieri (Eds.), Handbook of Executive Functioning (pp. 245-264). Springer Science and Business Media.
• Best, J. R., Miller, P. H., & Naglieri, J. A. (2011). Relations between Executive Function and Academic Achievement from Ages 5 to 17 in a Large, Representative National Sample. Learning and Individual Differences, 21(4), 327-336. https://doi.org/10.1016/j.lindif.2011.01.007
  » https://doi.org/10.1016/j.lindif.2011.01.007
• Blair, C., & Razza, R. P. (2007). Relating Effortful Control, Executive Function, and False Belief Understanding to Emerging Math and Literacy Ability in Kindergarten. Child Development, 78(2), 647-63. https://doi.org/10.1111/j.1467-8624.2007.01019.x
  » https://doi.org/10.1111/j.1467-8624.2007.01019.x
• Borella, E., Carretti, B., & Pelegrina, S. (2010). The Specific Role of Inhibition in Reading Comprehension in Good and Poor Comprehenders. Journal of Learning Disabilities, 43(6), 541-552. https://doi.org/10.1177/0022219410371676
  » https://doi.org/10.1177/0022219410371676
• Brasil, Ministério da Educação. (2019). Portaria nº 1.460, de 15 de agosto de 2019 Diário Oficial da União, Edição: 159, Seção: 1, Página: 28. https://www.in.gov.br/web/dou/-/portaria-n-1.460-de-15-de-agosto-de-2019-211216061
  » https://www.in.gov.br/web/dou/-/portaria-n-1.460-de-15-de-agosto-de-2019-211216061
• Bull, R., & Lee, K. (2014). Executive Functioning and Mathematics Achievement. Child Development Perspectives, 8(1), 36-41. https://doi.org/10.1111/cdep.12059
  » https://doi.org/10.1111/cdep.12059
• Cartwright, K. B., Coppage, E., Lane, A., Singleton, T., Marshall, T., & Bentivegna, C. (2017). Cognitive flexibility deficits in children with specific reading comprehension difficulties. Contemporary Educational Psychology, 50, 33-44. https://doi.org/10.1016/j.cedpsych.2016.01.003
  » https://doi.org/10.1016/j.cedpsych.2016.01.003
• Chaytor, N., Schmitteredgecombe, M., & Burr, R. (2006). Improving the ecological validity of executive functioning assessment. Archives of Clinical Neuropsychology, 21(3), 217-227. https://doi.org/10.1016/j.acn.2005.12.002
  » https://doi.org/10.1016/j.acn.2005.12.002
• Corso, H. V., Cromley, J., Sperb, T., & Salles, J. (2016). Modeling the relationship among reading comprehension, intelligence, socioeconomic status, and neuropsychological functions: The mediating role of executive functions. Psychology & Neuroscience, 9, 32-45. https://doi.org/10.1037/pne0000036
  » https://doi.org/10.1037/pne0000036
• Diamond, A. (2013). Executive functions. Annual Review of Psychologys, 64, 135-168. https://doi.org/10.1146/annurev-psych-113011-143750
  » https://doi.org/10.1146/annurev-psych-113011-143750
• Dias, N. M., & Malloy-Diniz, L. F. (2020). Funções Executivas: modelos e aplicações Pearson.
• Dias, N. M., & Seabra, A. G. (2017). Intervention for executive functions development in early elementary school children: effects on learning and behaviour, and follow-up maintenance. Educational Psychology, 37(4), 468-486. https://doi.org/10.1080/01443410.2016.1214686
  » https://doi.org/10.1080/01443410.2016.1214686
• Dias, N. M., Trevisan, B. T., Leon, C. B. R., Prust, A. P., & Seabra, A. G. (2017). Can Executive Functions Predict Behavior in Preschool Children? Psychology & Neuroscience, 10(4), 383-393. https://doi.org/10.1037/pne0000104
  » https://doi.org/10.1037/pne0000104
• Fonseca, R. P., Miranda, M. C., & Seabra, A. G. (in press). Neuropsicologia Escolar Pearson.
• Friedman, N. P., & Miyake, A. (2017). Unity and Diversity of Executive Functions: Individual Differences as a Window on Cognitive Structure.Cortex , 86, 186-204. https://doi.org/10.1016/j.cortex.2016.04.023
  » https://doi.org/10.1016/j.cortex.2016.04.023
• Gathercole, S. E., Pickering, S. J., Knight, C., & Stegmann, Z. (2004). Working memory skills and educational attainment: evidence from national curriculum assessments at 7 and 14 years of age. Applied Cognitive Psychology, 18(1), 1-16. https://doi.org/10.1002/acp.934
  » https://doi.org/10.1002/acp.934
• Kendeou, P., van den Broek, P., Helder, A., & Karlsson, J. (2014). A Cognitive View of Reading Comprehension: Implications for Reading Difficulties. Learning Disabilities Research & Practice, 29(1), 10-16. https://doi.org/10.1111/ldrp.12025
  » https://doi.org/10.1111/ldrp.12025
• Major, S., & Seabra-Santos, M. J. (2014). Parents and/or Teachers? Inter-Ratter Agreement on Preschoolers Social-Emotional Assessment. Psicologia: Teoria e Pesquisa, 30, 373-383.
• Martoni, A. T., Trevisan, B. T., Dias, N. M., & Seabra, A. G. (2016). Funções executivas: relação entre relatos de pais, de professores e desempenho de crianças. Temas em Psicologia, 24(1), 173-188. https://doi.org/10.9788/TP2016.1-12
  » https://doi.org/10.9788/TP2016.1-12
• Montiel, J. M., & Seabra, A. G. (2012). Teste de Atenção por Cancelamento. In A. G. Seabra & N. M. Dias (Eds.), Avaliação Neuropsicológica Cognitiva: Atenção e funções executivas (pp. 57-66). Memnon.
• Naglieri, J. A., & Goldstein, S. (2014). Assessment of Executive Function Using Rating Scales: Psychometric Considerations. In S. Goldstein & J. A. Naglieri (Eds.), Handbook of Executive Functioning (pp. 159-170). Springer Science and Business Media.
• Nelson, T. D., Nelson, J. M., James, T. D., Clark, C. A. C., Kidwell, K. M., & Espy, K. A. (2017). Executive control goes to school: Implications of preschool executive performance for observed elementary classroom learning engagement. Developmental Psychology, 53(5), 836-844. https://doi.org/10.1037/dev0000296
  » https://doi.org/10.1037/dev0000296
• Pazeto, T. C. B., Seabra, A. G., & Dias, N. M. (2014). Executive functions, oral language and writing in preschool children: Development and correlations. Paidéia, 24(58), 213-221. https://doi.org/10.1590/1982-43272458201409
  » https://doi.org/10.1590/1982-43272458201409
• Purpura, D. J., Schmitt, S. A., & Ganley, C. M. (2017). Foundations of mathematics and literacy: The role of executive functioning components. Journal of Experimental Child Psychology, 153, 15-34. https://doi.org/10.1016/j.jecp.2016.08.010
  » https://doi.org/10.1016/j.jecp.2016.08.010
• Rimm-Kaufman, S. E., Pianta, R. C., & Cox, M. J. (2000). Teachers’ judgments of problems in the transition to kindergarten. Early Childhood Research Quarterly, 15(2), 147-166. https://doi.org/10.1016/s0885-2006(00)00049-1
  » https://doi.org/10.1016/s0885-2006(00)00049-1
• Röthlisberger, M., Neuenschwander, R., Cimeli, P., & Roebers, C. M. (2013). Executive Functions in 5- to 8-Year Olds: Developmental Changes and Relationship to Academic Achievement. Journal of Educational and Developmental Psychology, 3(2), 153. https://doi.org/10.5539/jedp.v3n2p153
  » https://doi.org/10.5539/jedp.v3n2p153
• Schoemaker, K., Mulder, H., Deković, M., & Matthys, W. (2013). Executive functions in preschool children with externalizing behavior problems: A meta-analysis.Journal of abnormal child psychology,41(3), 457-471. https://doi.org/10.1007/s10802-012-9684-x
  » https://doi.org/10.1007/s10802-012-9684-x
• Seabra-Santos, M. J., & Gaspar, M. F. F. (2012). Pais, educadores e testes: estão de acordo na avaliação de aptidões de crianças pré-escolares? Psicologia: Reflexão e Crítica, 25(2), 203-211. https://doi.org/10.1590/S0102-79722012000200001
  » https://doi.org/10.1590/S0102-79722012000200001
• Seabra, A. G., & Capovilla, F. C. (2010). Teste de Competência de Leitura de Palavras e Pseudopalavras - TCLPP Memnon.
• Seabra, A. G., & Dias, N. M. (2012a). Avaliação Neuropsicológica Cognitiva: atenção e funções executivas (1st ed.). Memnon.
• Seabra, A. G., & Dias, N. M. (2012b). Reconhecimento de palavras e compreensão de leitura: dissociação e habilidades linguístico-mnemônicas preditoras. Neuropsicologia Latinoamericana, 4(1), 43-53.
• Seabra, A. G., Dias, N. M., & Capovilla, F. C. (2013). Avaliação Neuropsicológica Cognitiva: leitura, escrita e aritmética Memnon.
• Thorell, L. B., & Catale, C. (2014). The Assessment of Executive Functioning Using the Childhood Executive Functioning Inventory (CHEXI). In S. Goldstein & J. A. Naglieri (Eds.), Handbook of Executive Functioning (pp. 359-366). Springer Science and Business Media.
• Toll, S. W., van der Ven, S., Kroesbergen, E., & Van Luit, J. E. (2011). Executive functions as predictors of math learning disabilities. Journal of Learning Disabilities, 44(6), 521-532. https://doi.org/10.1177/0022219410387302
  » https://doi.org/10.1177/0022219410387302
• Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Practitioner Review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry, 54(2), 131-143. https://doi.org/10.1111/jcpp.12001
  » https://doi.org/10.1111/jcpp.12001
• Trevisan, B. T. (2010). Atenção e controle inibitório em pré-escolares e correlação com indicadores de desatenção e hiperatividade [Master’s Dissertation, Universidade Presbiteriana Mackenzie]. Biblioteca Digital de Teses e Dissertações Mackenzie. http://tede.mackenzie.br/jspui/handle/tede/1537
  » http://tede.mackenzie.br/jspui/handle/tede/1537
• Trevisan, B. T., Berberian, A. A., Dias, N. M., Roama-Alves, R. J., & Seabra, A. G. (submitted). Development and psychometric properties of the Difficulties in Executive Functions, Regulation and Delay Aversion Inventory (IFERA-I). Avaliação Psicológica
• Trevisan, B. T., & Seabra, A. G. (2012). Teste de trilhas para pré-escolares. In A. G. Seabra & N. M. Dias (Eds.), Avaliação Neuropsicológica Cognitiva: Atenção e funções executivas (pp. 92-100). Memnon.

• ^*
  Acknowledgments: CNPq - Research productivity grant to NM Dias and AG Seabra; CAPES - APPP Doctoral Scholarship.

Publication Dates

• Publication in this collection
  21 Jan 2022
• Date of issue
  2022

History

• Received
  12 Sept 2019
• Accepted
  07 Sept 2020