Self-Regulation for Reading Comprehension: Assessment of Strategies and Time Management

Autorregulação para Compreensão de leitura: Avaliação de Estratégias e Gerenciamento do Tempo

ABSTRACT

Self-regulation of learning strategies and time management can foster reading comprehension. This research investigated the content validity of three scales aimed at Middle School students: the Reading Strategies (EE-CL), the Time Planning and Organization (EOT-L), and the Procrastination (EP-L). Three researchers and 16 students evaluated the scales. The theoretical and practical relevance of the EE-CL and EOT-L was verified, but not of the EP-L, whose remaining items were included in the EOT-L. In the internal structure investigation (N= 522 students), the EE-CL presented a unifactorial structure and the EOT-L, two factors, with reasonable reliability estimates. Both scales may help identify specific problems of students' self-regulation aimed at strategies and time management for reading comprehension and support further research.

KEYWORDS:  self-management; study habits; reading; middle school; psychological assessment

RESUMO

A autorregulação das estratégias de aprendizagem e do gerenciamento do tempo fomentam a compreensão de leitura. Nesta pesquisa investigou-se a validade de conteúdo das escalas Estratégias para Leitura (EE-CL), Planejamento e Organização do Tempo (EOT-L) e Procrastinação (EP-L), destinadas ao Ensino Fundamental II. Participaram três pesquisadores e 16 alunos. Identificou-se a pertinência teórica e prática da EE-CL e EOT-L, mas não da EP-L, cujos itens remanescentes foram incluídos na EOT-L. Na análise da estrutura interna (N= 522 estudantes) a EE-CL apresentou uma estrutura unifatorial e a EOT-L, dois fatores, com estimativas adequadas de fidedignidade. Essas escalas podem auxiliar na identificação de problemas na autorregulação das estratégias e do gerenciamento do tempo na compreensão de leitura, bem como subsidiar novas pesquisas.

PALAVRAS-CHAVE:  autogestão; hábitos de estudo; leitura; ensino fundamental; avaliação psicológica

Self-regulation of learning directed to strategies and time management is associated with performance in reading. Therefore, evaluating the levels of students' self-regulation in these two variables enables the proposition of improvements in teaching practices aimed at reading and reading comprehension (Gilakjani & Sabouri, 2016; White & DiBenedetto, 2015). To support the assessment of self-regulation of these skills in Middle School students, this study aimed to investigate the psychometric properties of three instruments: the Reading Comprehension Strategies Scale (Escala Estratégias para Compreender a Leitura - EE-CL), the Reading Time Organization Scale (Escala Organização do Tempo para Leitura- EOT-L), and the Reading Procrastination Scale (Escala Procrastinação para Ler- EP-L).

These scales were built to assess two of the six dimensions of self-regulation for learning proposed by Zimmerman and Risemberg (1997): method and time management. The other dimensions of self-regulation refer to motivation, self-perceived behavior, physical environment, and the social environment (Schunk & Usher, 2013; White & DiBenedetto, 2015; Zimmerman & Risemberg, 1997). The concept of reading comprehension that founded the scales came from Kintsch and Rawson (2013). These researchers define reading comprehension as a process involving different processing levels - from the linguistic level (e.g., decoding and word recognition) to the situational model (integration of text content with prior knowledge). The basis for the degree of reading comprehension development for Middle School students was based on the Common National Curriculum Base (Base Nacional Comum Curricular [BNCC]; Ministério da Educação, 2017) and the reference matrices in the Portuguese language of the Basic Education Assessment System (Sistema de Avaliação da Educação Básica [SAEB]; Ministério da Educação, 2018).

The EE-CL assesses the method dimension, which encompasses strategies that may facilitate reading comprehension (Schunk & Usher, 2013; Zimmerman & Risemberg, 1997). The EOT-L measures two key processes of the time management dimension: planning and organization of reading time. The EP-L measures problems in self-regulation expressed through reading activity procrastination.

Learning strategies involve the intersection of cognitive and metacognitive level procedures that help the performance of school activities by facilitating access to information, decision-making, and content assimilation (Schunk & Usher, 2013; Zimmerman & Risemberg, 1997). In tasks that require reading comprehension, students are more likely to be successful when they achieve self-regulation in strategies. Self-regulated students assess whether they are applying the strategies correctly and consider their level of effectiveness to decide whether they should continue to use them, they should adjust them, or whether they need to seek other strategies (Leopold & Leutner, 2015).

Associations have been found between reading comprehension and elaboration strategies, based on the selection of information, and metacognitive strategies, emphasizing planning, evaluation, and regulation (Pinto et al., 2016). In samples of American (7^th to 12^th grade) and Spanish (Secondary Education) students, classified as good readers, there was a predominance of a more extensive and more diversified repertoire of strategies applied to reading (Denton et al., 2015; Pinto et al., 2016). These results were compatible with the conclusions from a literature review conducted by Gilakjani and Sabouri (2016). The study by Pinto et al. (2016) identified that students with greater reading ability, compared to less skilled students, more often reported the use of elaboration strategies, centered on the selection of information, and regulation strategies, referring to adjustments in the reading speed according to the complexity of the material being read. Strategies are more effective for reading comprehension performance when they occur through self-regulated (White & DiBenedetto, 2015; Zimmerman & Risemberg, 1997), like Leopold and Leutner (2015) found in an intervention study with German 10th-grade students, focusing on didactic texts.

In turn, time management involves estimating the period needed to carry out an activity, which involves planning and organizing activities around a deadline previously established by the students or others (Schunk & Usher, 2013; White & DiBenedetto, 2015). Self-regulated students mobilize their time based on task demands and the results they want to achieve. As with learning strategies, self-regulation of this skill involves students’ self-evaluation of how they plan and organize time and their impressions about the repercussions of these self-assessments concerning the results obtained (Schunk & Usher, 2013; Zimmerman & Risemberg, 1997).

Problems in self-regulation such as lack of autonomy lead to procrastination, characterized by the tendency of students to systematically postpone academic activities even though they are aware of the negative consequences of this action (Lenggono & Tentama, 2020; Zacks & Hen, 2018; Ziegler & Opdenakker, 2018). A longitudinal study with Dutch students (1^st year of Secondary Education) suggests that low self-efficacy, metacognitive self-regulation, and effort regulation are predictors of procrastination (Ziegler & Opdenakker, 2018).

Furthermore, problems in self-regulation that reflect the misuse of learning strategies, time management, and procrastination can be overcome through interventions aimed at, for example, encouraging self-assessment of the procedures adopted during reading and through guidance on how to manage time (Leopold & Leutner, 2015; Xu, 2016). Before any intervention, however, it is crucial to develop diagnostic assessments of students’ self-regulation abilities. The goal of this study was to provide instruments that may allow such diagnostic assessments.

The main aim of this study was to investigate the evidence based on test content and the evidence based on the internal structure of the EE-CL, EOT-L, and EP-L. Another aim was to estimate the scales’ reliabilities. It was hypothesized that the content of the scales would have theoretical relevance and practical relevance to assess the strategies applied to reading, time management (planning and organization), and procrastination. Middle School students would be able to understand their content and functioning. They also would recognize the situations presented in the items as representative of their daily academic lives (AERA et al., 2014). Concerning the structure of the scales, we expected to find unifactorial solutions and that these would present adequate reliability indices.

Content validity evidence study - part 1: Judges’ Analysis

Method

Participants

The sample consisted of three researchers (judges) with expertise in Psychological and Educational Assessment and with the constructs of self-regulation for learning and reading comprehension of primary education students. The judges have been working for a mean of 16.66 years (SD = 13.87 years) in the role of researchers and professors in higher education institutions in Brazil and Portugal.

Instrument

Judges’ Evaluation Protocol. The protocol assessed the content of the EE-CL (22 items), EOT-L (13 items), and EP-L (11 items) using four criteria: language clarity, practical relevance, theoretical relevance - answered on a Likert scale, and dimension theoretical - answered on a dichotomous scale (Yes or No). The theoretical aspects of the scales were evaluated based on the six dimensions of self-regulation proposed by Zimmerman and Risemberg (1997). The protocol also had a section for judges to comment on the scales.

Data collection procedure

The Research Ethics Committee approves the project from the Universidade São Francisco (Authorization No. 3.263.350). The study with the judges was carried out remotely using Google Forms. To gain access to the Judge Evaluation Protocol, the judges had to accept the consent terms.

Data analysis procedure

Microsoft Excel® software.We apply the Content Validity Coefficient (CVC) calculation, as recommended by Hernández-Nieto (2002). In the validation criteria, clarity of language, practical relevance, and theoretical relevance, items with values of CVCc ≥ .80 are considered adequate, and factors or the full-scale (CVCtotal ≥ .80). Fleiss’ Kappa values (k) ≥ .40 are considered adequate at the theoretical dimensional (Brennan & Prediger, 1981). The judges’ observations were analyzed qualitatively, emphasizing the convergence between the theoretical aspects that underlie the concepts of learning strategies and time management for reading comprehension from the perspective of self-regulation dimensions (Zimmerman & Risemberg, 1997) with the Middle School context. The judges also assessed the quality of item construction (AERA, APA, & NCME, 2014). The quantitative and qualitative criteria of the CVCc and CVCtotal supported the maintenance, reformulation, and exclusion of items.

Results

Table 1 shows that the EE-CL obtained CVCt > .80 in the three validation criteria, and thek= .88 was qualified as almost perfect. Item 17 was reformulated due to Constant Content Validity Coefficient (CVCc) > .80. Regarding exclusions, in item 11, the judges highlighted a failure in constructing the sentence - incomplete content. Item 22 was classified as not very representative of behaviors observed in Middle School students.

Table 1
Reading Comprehension Strategies Scale (EE-CL): Content Validity Coefficient

Items Reading Comprehension Strategies Scale (EE-CL)	CVCc			k
	CL	PR	TR	TD
1. Mark the *** of the text	.96	.96	.96	1
2. I look for the *** words	.96	.96	.96	1
3. I read the complex parts *** than once	.89	.96	.96	1
4. I read the most complicated *** aloud	.89	.83	.89	1
5. I move forward or backward *** better	.96	.96	.96	1
6. I summarize the complex *** text	.96	.96	.96	1
7. I relate the content read *** read	.96	.96	.96	1
8. I interpret the graphics*** in the text	.83	.83	.83	1
9. I make notes in essential *** text	.96	.96	.96	1
10. I organize the primary *** the text	.96	.96	.96	1
11. I search *** texts	.83	.83	.83	.33
12. I compare information *** of text	.89	.89	.89	1
13. I identify the *** in the text	.89	.89	.89	1
14. I relate the content *** my daily life	.89	.89	.89	1
15. I search for *** the text	.83	.96	.96	1
16. I read the *** the text	.96	.83	.89	1
17. I identify the textual genre *** reading	.76	.76	.76	.33
18. I try to find out *** topic covered	.89	.89	.89	1
19. I imagine the scenarios *** in the story	.89	.89	.89	1
20. I write with my own words *** text	.96	.96	.96	1
21. I think about questions *** text	.96	.96	.96	1
22. I ask the teacher *** I read	.89	.83	.83	1
CVCt	.87	.87	.87
k (full scale)	.88
Agreement between judges for the full scale	93.94%
Original item	Reformulated item		Justifications
Item 17. I identify the textual genre when starting to read	Item 17. I identify the type of text when reading		CVCc < .80; k, poor
Deleted items
Item 11. I search for similar texts	k, poor; Judges' observations
Item 22. I ask the teacher if I understand what I read	Judges' observations

• Subtitle. EE-CL = Escala Estratégias para Compreender a Leitura; CVCc = Constant Content Validity Coefficient; k = Kappa Fleiss; CL = Clarity of Language; PR = Practical Relevance; TR = Theoretical Relevance; TD = Theoretical Dimension; CVCt = Total Content Validity Coefficient.

The EOT-L and EP-L had CVCt > .80 and k = 1, classified as perfect (see Table 2 and Table 3). Regarding the reformulations, in item 12 of the EOT-L, the judges indicated an object (diary) that the students might not have. Item 14 of the EOT-L and items 2, 4, and 6 of the EP-L changes were intended to promote their intelligibility.

Table 2
Reading Time Organization Scale (EOT-L): Content Validity Coefficient

Items Reading Time Organization Scale		CVCc			k
		CL	PR	TR	TD
1. I set aside *** week to read		.96	.96	.96	1
2. The time it takes *** the text		.96	.83	.96	1
3. Before starting to read *** the text		.96	.96	.96	1
4. I take longer *** complicated texts		.96	.96	.96	1
5. I take less time *** easy texts		.96	.96	.96	1
6. I find it challenging *** to read		.96	.96	.96	1
7. The time the teacher *** the text		.96	.96	.96	1
8. I know how to *** read		.96	.96	.96	1
9. I take time *** at home		.96	.96	.96	1
10. I first read the *** texts		.89	.89	.89	1
11. I better understand *** noon, or night		.89	.89	.89	1
12. I mark the readings *** a diary		.96	.83	.89	1
13. I stop reading *** tired		.89	.89	.89	1
14. I take a break *** the text		.89	.89	.89	1
CVCt		.90	.88	.90
k (full scale)		1
Agreement between judges for the full scale		100%
Original item	Reformulated item	Justifications
Item 12. I mark the readings I need to do in a diary	Item 12. I mark the readings I need to do in a diary or elsewhere	Judges' observations
Item 14. I take a break from reading when it is hard to understand the text	Item 14. I take a break from reading when it is hard to understand the text (Dou uma parada na leitura quando está difícil de entender o texto)

• Subtitle. EOT-L = Escala Organização do Tempo para Leitura; CVCc = Constant Content Validity Coefficient; k = Kappa Fleiss; CL = Clarity of Language; PR = Practical Relevance; TR = Theoretical Relevance; TD = Theoretical Dimension; CVCt = Total Content Validity Coefficient.
• Note. Item 14 appears in both languages because the change made to the item in Portuguese did not change the writing of the item in English.

Table 3
Reading Procrastination Scale (EP-L): Content Validity Coefficient

Items Reading Procrastination Scale		CVCc			k
		CL	PR	TR	TD
1. I leave to read *** last minute		.96	.96	.96	1
2. I leave to read *** read today		.96	.96	.96	1
3. My readings *** late		.96	.96	.96	1
4. It is hard to go back *** interrupted		.89	.89	.89	1
5. I stop reading *** time		.89	.89	.89	1
6. Even with time to read *** later		.96	.96	.96	1
7. I just find time *** I want		.89	.96	.96	1
8. I do not have time *** things to do		.89	.89	.89	1
9. I always leave to read *** the test		.89	.96	.96	1
10. It is difficult *** to read		.96	.96	.96	1
11. I do not want to go back *** to understand		.96	.96	.96	1
CVCt		.89	.89	.92
k (full scale)		1
Agreement between judges for the full scale		100%
Original item	Reformulated item	Justifications
Item 2. I leave to read the texts I need to read today	I leave to read the texts that I need to read immediately	Judges' observations
Item 4. It is hard to go back to reading when I am interrupted	It is hard to go back to reading when I stop in the middle of reading
Item 6. Even with time to read, I leave it for	Even though I have time, I leave it to read later

• Subtitle. EP-L = Escala Procrastinação para Ler; CVCc = Constant Content Validity Coefficient; k = Kappa Fleiss; CL = Clarity of Language; PR = Practical Relevance; TR = Theoretical Relevance; TD = Theoretical Dimension; CVCt = Total Content Validity Coefficient.

Content validity evidence study - part 2: Target audience study

Method

Participants

Sample of 16 students from the 6^th to 9^th year of Middle School at a school located in the state of São Paulo. The students were distributed evenly regarding school years and sex. The students had no history of year repetition, with a minimum age of 11 and a maximum of 15 years (M = 13; SD = 1.31).

Instrument

Target Audience Assessment Protocol. This instrument assessed the students’ understanding of the statement, the answer key options (Part 1), and the EE-CL, EOT-L, and EP-L. The content of the items was measured by intelligibility and relevance to the school reality experienced by the students.

Data collection procedure

After authorization to carry out the target audience study from the school, parents/guardians consented to the participation of students in the research by signing the consent form. The students agreed to participate by also signing a consent form.

To respond to the Target Audience Assessment Protocol, students must be able to read without difficulties since the purpose of the instrument is to assess the scales. Because of this, the pedagogical coordinator was asked to select students who did not have reading difficulties. This selection was made by the pedagogical coordinator based on the exchange of information about the students with the Portuguese language teacher.

The interviews were conducted in two stages to avoid the fatigue effect on the students and took place individually in a semi-structured interview format. Parts I and II of the protocol were applied at an interval of one to two weeks.

Data analysis procedure

The statement's evaluation, the answer key options, and the EE-CL, EOT-L, and EP-L items were computed using Absolute Frequency (AF). The open-ended responses were analyzed qualitatively to ensure that the content of the items was compatible with students’ cognitive level and ensure their understanding of the functioning of the scales. The combination of quantitative and qualitative results supported the reformulation and exclusion of items. In addition, the researchers also based their decisions on theoretical aspects of self-regulation for learning and the instrument construction guides (AERA et al., 2014; Borsa & Seize, 2017).

Results

Concerning Part I of the Target Audience Assessment Protocol, it was found that the difficulties highlighted by the students in evaluating the statements and labels of the answer keys of the EE-CL, EOT-L, and EP-L referred to the particularities of this task (see Tables 3, 4, and 5). This hypothesis was confirmed during the interviews. Students were used to taking tests (e.g., National Assessment of School Achievement [Avaliação Nacional do Rendimento Escolar], Ministério da Educação, 2018). However, they had never played the role of psychological test evaluators. This problem was overcome through additional guidance on the role of students in the target audience study. The purpose of this procedure was to ensure that students understood the aim of the task.

Based on students' comments in Part II of the Target Audience Assessment Protocol, two items of the EE-CL were reformulated, and two items were excluded - see Table 4. In item 5, the students presented a specific difficulty assimilating the passage that indicates "moving back and forth within a text".

Table 4
Reading Comprehension Strategies Scale (EE-CL): Target Audience Study

Part I: Statement and the answer key
Understanding the instructions for completing the scale	AF = 16 (100%)
2. Identification of inaccurate passages	AF = 1 (6.28%)
3. Unknown word identification	AF = 2 (12.5%)
4. Understanding answer key labels	AF = 16 (100%)
Part II: Items
1. Identification of similar content	AF = 4 (25%)
2. Difficulty understanding	AF = 0 (0%)
Original item	Reformulated item
Item 3. I read the complex parts of the text more than once (Leio mais de uma vez os trechos difíceis do texto)	I read, more than once, the complex parts of the text (Leio, mais de uma vez, os trechos difíceis do texto)
Item 16. I read the title of the text (Identifico o tipo do texto ao ler)	When reading, I identify the type of text (Ao ler, identifico o tipo do texto)
Deleted items	Justifications
Item 1. Mark the essential parts of the text	Students opted for item 9 due to the similarity of content “I make notes in essential parts of the text”
Item 5. I move forward or backward through the text to understand it better	The difficulty of item content

• Subtitle. EE-CL = Escala Estratégias para Compreender a Leitura; AF = Absolute Frequency

Table 5 indicates the reformulation of one EOT-L item. In item 6, the word encontrar (find) was replaced by arrumar (arrange). Five items were excluded. In particular, items 2, 4, 5, and 14 require students’ metacognitive reasoning to associate the items with the organization of time based on the complexity of the reading comprehension task. This type of reasoning is a characteristic of task self-monitoring, another aspect of self-regulation linked to time management, but which is not the focus of EOT-L. In item 13, a pause in reading was not classified as a way to manage time. Students perceived this behavior as a demonstration of giving up due to physical fatigue and lack of motivation.

Table 5
Reading Time Organization Scale (EOT-L): Target Audience Study

Part I: Statement and the answer key
Understanding the instructions for completing the scale	AF = 16 (100%)
2. Identification of inaccurate passages	AF = 1 (6.28%)
3. Unknown word identification	AF = 0 (0%)
4. Understanding answer key labels	AF = 14 (87.5%)
Part II: Items
1. Identification of similar content	AF = 2 (12.5%)
2. Difficulty understanding	AF = 0 (0%)
Original item	Reformulated item
Item 6. I find it challenging to find time to read (Sinto dificuldade de encontrar um tempo para ler)	I find it challenging to find time to read (Sinto dificuldade de arrumar um tempo para ler)
Deleted items	Justifications
Item 1. I set aside a few hours a week to read	Students opted for item 8 due to the similarity of content “I know how to organize the time to read”
Item 2. The time it takes to read depends on the difficulty of the text	The difficulty of item content
Item 4. It takes me longer to understand complicated texts
Item 5. It takes less time to understand accessible texts
Item 14. I take a break from reading when it is hard to understand the text
Item 13. I stop reading when I am tired	Content refers to demotivation

• Subtitle. EOT-L = Escala Organização do Tempo para Leitura; AF = Absolute Frequency.
• Note. After reformulation, item 6 changed in Portuguese, but not in English.

Regarding the EP-L, Table 6 shows the exclusion of six items. The contents of items 5, 8, and 10 cover situations experienced by the students, however, which may not be under their control and possibly not entirely related to procrastination. Consequently, students could achieve maximum scores in these items because they have other obligations that would make it impossible to carry out the reading required by the school, not because they have beliefs related to procrastination. A negative consequence of keeping them on the scale would be the misinterpretation that students have self-regulation problems in managing time for reading activities, classifying them as procrastinating students. In items 7 and 11, the students gave a motivational connotation, mainly to the term “enjoy” (item 7), understood as pleasure and effort, beyond the scope of the scale. After the exclusions, the EP-L had five items (see Table 3, items 1, 2, 4, 6, and 9). These items were relocated to EOT-L as a potential factor to assess problems in self-regulation to manage the time for activities that involve reading.

Table 6
Reading Procrastination Scale (EP-L): Target Audience Study

Part I: Statement and the answer key
Understanding the instructions for completing the scale	AF = 16 (100%)
2. Identification of inaccurate passages	AF = 2 (12.5%)
3. Unknown word identification	AF = 0 (0%)
4. Understanding answer key labels	AF = 14 (87.5%)
Part II: Items
1. Identification of similar content	AF = 8 (50%)
2. Difficulty understanding	AF = 0 (0%)
Deleted items	Justifications
Item 3. My readings are always late	Students opted for item 1 due to the similarity of content “I leave to read the school texts at the last minute”
Item 5. I stop reading for lack of time	Content moves away from the student's school routine
Item 8. I do not have time to read school texts because I have so many things to do
Item 10. It is hard to find time to read
Item 7. I just find time to read what I enjoy	Content refers to demotivation
Item 11. I do not want to go back to reading texts that are difficult to understand

• Subtitle. EP-L = Escala Procrastinação para Ler; AF = Absolute Frequency.

Steps 1 and 2: Discussion

Based on the results of steps 1 and 2 of this study, it was concluded that the EE-CL (18-item version) and EOT-L (8-item version) presented evidence of content validity. In the target audience study (step 2), this evidence was supported for only five items of the EP-L, which the students qualified as representative of difficulties in managing time to perform activities that require reading. Therefore, it was conjectured that these items refer to problems in students’ self-regulation of time management, emphasizing procrastination (Zacks & Hen, 2018; Ziegler & Opdenakker, 2018).

This study also revealed that the quality of the test construction reflects on the results of investigations of its psychometric properties, given that problems in the elaboration of items can compromise the intelligibility and representativeness of the measure (AERA et al., 2014; Carvalho & Ambiel, 2017). For example, item 11 of the EE-CL, “I search for similar texts,” was evaluated as an incomplete sentence in the judges’ analysis. Item 8 from the EP-L, “I do not have time to read school texts because I have too many things to do,” and item 10, “It is difficult to find time to read,” presented situations that may distance themselves from the reality evaluated. Some items mentioned related constructs, such as motivation, in items 7, “I only find time to read what I enjoy,” and 11, “I do not want to go back to reading texts that are difficult to understand,” of the EP-L. Therefore, this study also served to improve the EE-CL, EOT-L, and EP-L items by submitting them to the appreciation of experts in the evaluated constructs and the target audience of these scales (AERA et al., 2014; Borsa & Seize, 2017).

The importance of the items being understandable is also emphasized, considering the intelligibility of the wording and the consonance between the reality portrayed in their content and that experienced by the students (AERA et al., 2014; Toland & Usher, 2015). This premise underlies, for example, the exclusion from the EOT-L of items 4, “I take longer to understand complicated texts” and 5, “I take less time to understand easy texts,” in which time management appears implicitly. It is recognized that self-regulation manifests itself in the metacognitive aspects present in self-assessment and self-monitoring of actions and thoughts underlying time management (Schunk & Usher, 2013; Ziegler & Opdenakker, 2018). However, for the target audience of this scale, it was observed that situations involving planning and organization of time were better understood if presented objectively.

Finally, the items from the EP-L were relocated to the EOT-L, focusing on the difficulty of time management expressed mainly by problems in self-regulation, which tend to be associated with unfavorable academic behaviors for learning, as is the case in procrastination (Zacks & Hen, 2018; Ziegler & Opdenakker, 2018). Therefore, these items translate into low levels of self-regulation in managing time for tasks that require reading comprehension.

Validity evidence study based on the Internal Structure

Method

Participants

The sample consisted of 522 students enrolled in the four years of Middle School, with a minimum age of 10 and a maximum of 18 years (M = 12.72;SD = 1.26). Among these students, 280 (53.64%) were female, and 90 students (17.24%) had repeated years. The students attended three schools in the state of São Paulo.

Instruments¹

Reading Comprehension Strategies Scale (Escala Estratégias para Compreender a Leitura - EE-CL). This scale assesses the self-regulation of Middle School students through the repertoire of strategies used to promote reading comprehension. The EE-CL has 18 items. The students responded to the items through four response options: Never (Nunca), Seldom (Pouco), Often (Algumas Vezes), and Always (Sempre).

Reading Time Organization Scale (Escala Organização do Tempo para Leitura- EOT-L). The 13 items of the EOT-L assess self-regulation through students’ perceptions of their time management for activities that require reading. The scale’s answer key is a 4-point Likert-type scale, with options ranging from Not True (Nada Verdadeiro) to Completely True (Totalmente Verdadeiro).

Data collection procedure

After obtaining authorization from the schools, students who brought the consent form signed by one of their parents/guardians and signed the consent terms were considered eligible to participate in the study. One student was over 18 years old. So, this student signed the consent form for people of legal age. The scales were applied in pencil and paper format, collectively, during class time.

Data analysis procedure

The data were analyzed using the Factor (Lorenzo-Seva & Ferrando, 2020) and MPlus (version 7.11; Muthen & Muthen, 2012).

Factorial retention of EE-L and EOT-L. We used the Parallel Analysis, having as criterion values of mean-variance of real data greater than the mean of variance of the random data. For the EE-CL, the hypothesis was that a unifactorial structure. Therefore, we also analyzed the unidimensionality indicators: Unidimensional Congruence (UniCo) > .95, Explained Common Variance (ECV) > .85, and the Mean of Item Residual Absolute Loadings (MIREAL) < .30 (Damásio & Dutra, 2017).

To analyze the internal structure of the EE-CL, we used the Confirmatory Factor Analysis (CFA). For EOT-L, we apply Exploratory Structural Equation Modeling (ESEM). We used the Weighted Least Square Mean and Variance Adjusted (WSLMV) estimation method in both analyses. In CFA, we apply the Geomin oblique rotation method (Muthén & Muthén, 2012).

Interpretation of the models’ plausibility. The Root Mean Square Error of Approximation: RMSEA ≤ .90 values qualify as an acceptable fit; the Confirmatory Fit Index and Tucker-Lewis Index: CFI and TLI ≥ .80 values are qualified as a moderate fit. In ESEM, the ratio of χ ² /dftest < .05 was also considered (Marôco, 2014). Factor loadings ≥ .40 was the cutoff point established for the maintenance of the item in the models tested.

Internal consistency was assessed using the Composite Reliability (cr) index, calculated using the online calculator available at http://www.thestatisticalmind.com/calculators/comprel/composite_reliability.htm. The interpretation of these indices was based on values ≥ .70 (Peterson & Kim, 2013).

Results and Discussion

The results of the parallel analysis for the Reading Comprehension Strategies Scale (EE-CL) indicated the retention of one factor, with the mean-variance of the real data (37.91%) being more significant than the mean of the explained variance of the random data (10.57%). The MIREAL value = .22 also indicated a unifactorial structure.

Two models were tested for the EE-CL. The first model yielded χ ² /df = 4.11; RMSEA = .08 (90% CI 0.071 - 0.084); CFI = .89; TLI = .87. The item “I read the title of the text” was excluded as it presented a factor loading of less than .40. Table 7 shows that in the second model of the EE-CL, with 17 items, theχ ² /dfwas adequate, and the fit indices were qualified as good, which denotes the validity evidence based on the scale’s internal structure. This result was corroborated by thecrindex, which indicated that the scale has reasonable reliability estimates.

Table 7
Reading Comprehension Strategies Scale (EE-CL): Internal Structure

Items Reading Comprehension Strategies Scale	CFA2
	F1
1. I look for *** the words	.59
2. I read, more *** the text	.56
3. I read the most *** text aloud	.40
4. I summarize the complex *** text	.60
5. I relate the content *** read	.65
6. I interpret the graphics*** the text	.47
7. I make notes *** the text	.55
8. I organize the primary *** text	.67
9. I compare information *** of text	.57
10. I identify the relationships *** the text	.60
11. I relate the content *** daily life	.52
12. I search for the main *** text	.61
13. When reading, *** of text	.48
14. I try to find out *** topic covered	.58
15. I imagine the scenarios *** the story	.40
16. I write with my own *** the text	.49
17. I think about questions *** text	.59
Plausibility indices of the CFA2 model
χ2/gl = 3.65; RMSEA = .071 (.064 - .079); CFI = .91; TLI = .90
Reliability index
cr = .88

• Subtitle. EE-CL = Escala Estratégias para Compreender a Leitura.
• Note1. Values in bold indicate factor loading above .40.
• Note2. Contact the first author of this paper to check the possibility of accessing the scales.

The 17 items of the EE-CL encompass strategies to facilitate reading comprehension through procedures that involve cognitive and metacognitive aspects centered on repetition, summarization, and association of information with the text itself or with other reading materials (Gilakjani & Sabouri, 2016; Pinto et al., 2016). These strategies are identified in students recognized as being good readers (Pinto et al., 2016). The EE-CL scores indicate the repertoire of strategies used by students to carry out reading activities. Therefore, the scale scores indicate the level of self-regulation based on mental images and the self-instruction underlying these procedures (Wigfield, Klauda, & Cambria, 2015; Zimmerman & Risemberg, 1997). It is also considered that having a more extensive and diverse number of strategies present in EE-CL represents greater chances of presenting proficient reading comprehension (Denton et al., 2015; Gilakjani & Sabouri, 2016; Pinto et al., 2016).

For the Reading Time Organization Scale (EOT-L), the parallel analysis indicated a structure with two factors - mean values of variance of the real data of 33.16% and 15.51% higher than the mean of the explained variance random data, 20.45%, and 13.99%. Two models were analyzed to investigate the internal structure of the EOT-L. Three items that presented factor loadings below .40 (χ ² /df = 2.57; RMSEA = .05 [90% CI 0.044 - 0.066]; CFI = .95; TLI = .93) were excluded from the first model, these being, “Before I start reading, I think about the time it will take to read the text” and “It is difficult to read again when I stop in the middle of reading,” which were grouped in F1 and “The time the teacher usually gives to read is enough for me to understand the text,” which loaded in F2.

The second model (10 items) proved plausible (see Table 8), with an adequateχ ² /df ratio, RMSEA classified as good and CFI/TLI as very good. This result gives the EOT-L evidence of validity based on the internal structure. The items did not show cross-factor loadings above .40 between the factors. Item 10, designed to assess the absence of self-regulation in time organization, became part of F1, corresponding to procrastination. Both factors consisted of five items each, F1 being named the Procrastination factor (items 1, 2, 4, 7, and 10) and F2 Time Organization (items 3, 5, 6, 8, and 9). The composite reliability indices of the EOT-L factors were qualified as adequate, indicating that the structure tested has reliability estimates. The correlation between F1 and F2 was -.29 (p< .001).

Table 8
Reading Time Organization Scale (EOT-L): Internal Structure

Items Reading Time Organization Scale	ESEM2
	F1	F2
1. I leave to read *** last minute	.70	-.11
2. I leave to read *** read immediately	.70	.01
3. I know how to organize *** read	-.01	.75
4. Even though I have time*** later	.58	-.25
5. I take time *** home	-.10	.72
6. I first read *** important texts	.14	.43
7. I always leave *** near the test	.60	.05
8. I better understand *** or night	.28	.55
9. I mark the readings *** elsewhere	.07	.57
10. It is hard *** to read	.45	-.18
Plausibility indices of the ESEM2 model
χ2/gl = 2.61; RMSEA = .063 (.041 - .086); CFI = .97; TLI = .95
Reliability index
cr: F1 = .75; F2 = .75

• Subtitle. EOT-L = Escala Organização do Tempo para Leitura; F1 = Factor 1, Procrastination; F2 = Factor 2, Time Organization.
• Note1. Values in bold indicate factor loading above .40.
• Note2. Contact the first author of this paper to check the possibility of accessing the full scales.

As expected, the Procrastination factor (F1) contains aspects that allude to problems in self-regulation of time management linked to academic procrastination in the performance of reading activities. These items focus on postponing activities and emphasizing the difficulty of planning, being in line with studies that report the manifestation of these behaviors in procrastinating students (Zacks & Hen, 2018; Ziegler & Opdenakker, 2018).

The items in the Time Organization Factor (F2) involve students’ beliefs about their ability to organize reading tasks in an environment outside the class, the use of instructional materials to manage commitments, metacognitive reasoning about times of the day that facilitate reading comprehension, and priority setting through the ranking of activities. These situations express self-regulation, emphasizing planning and time management to carry out activities that require reading comprehension (Schunk & Usher, 2013; White & DiBenedetto, 2015; Ziegler & Opdenakker, 2018).

This study found that the EE-CL and EOT-L have good initial psychometric properties for investigating self-regulation strategies and time management for activities requiring reading comprehension for Middle School students. Future studies with these scales will assess whether their structure remains plausible in samples from other Brazilian regions and private schools. This proposition is mainly directed to the EOT-L to evaluate the plausibility of the two-factor model in other samples, as we did not foresee this configuration when we built the scale.

It is expected that the EE-CL and EOT-L will contribute to the identification of aspects to be improved in the context of fundamental education, either through timely and specific interventions to reverse self-regulation problems or in the design of pedagogical practices to prevent reading comprehension difficulties as a result of lack of students’ skills to apply reading strategies and manage their time. Furthermore, these scales can also be applied throughout the pedagogical interventions to assess the aspects that need to be changed. This type of evaluation may help to optimize the effectiveness of interventions.

References

• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing American Educational Research Association.
• Brennan, R. L., & Prediger, D. J. (1981). Coefficient kappa: Some uses, misuses, and alternatives.Educational and Psychological Measurement,41(3), 687-699. https://doi.org/10.1177/001316448104100307
  » https://doi.org/10.1177/001316448104100307
• Borsa, J. C., & Seize, M. M. (2017). Construção e adaptação de instrumentos psicológicos: caminhos possíveis. Em B. F. Damásio & J. C. Borsa (Orgs.). Manual de desenvolvimento de instrumentos psicológicos, (pp. 15-38). Vetor Editora.
• Carvalho, L. F., & Ambiel, R. A. M. (2017). Construção de instrumentos psicológicos. Em B. F. Damásio & J. C. Borsa (Orgs.). Manual de desenvolvimento de instrumentos psicológicos, (pp. 39-56). Vetor Editora.
• Damásio, B. F. & Dutra, D. F. (2017). Análise fatorial exploratória: um tutorial com o software Factor. Em B. F. Damásio & J. C. Borsa (Orgs.). Manual de desenvolvimento de instrumentos psicológicos, (pp. 241-266). Vetor Editora.
• Denton, C. A., Wolters, C. A., York, M. J., Swanson, E., Kulesz, P. A., & Francis, D. J. (2015). Adolescents’ use of reading comprehension strategies: Differences related to reading proficiency, grade level, and gender.Learning and Individual Differences,37, 81-95. https://doi.org/10.1016/j.lindif.2014.11.016
  » https://doi.org/10.1016/j.lindif.2014.11.016
• Ferraz, A. S. (2022). Autorregulação para a compreensão de leitura no ensino fundamental II: Proposta de avaliação a intervenção (Doctoral Thesis in Psychology). Universidade São Francisco. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.usf.edu.br/galeria/getImage/427/2810491657915438.pdf
  » chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://www.usf.edu.br/galeria/getImage/427/2810491657915438.pdf
• Gilakjani, A. P., & Sabouri, N. B. (2016). How can students improve their reading comprehension skill.Journal of Studies in Education, 6(2), 229-240. http://dx.doi.org/10.5296/jse.v6i2.9201
  » http://dx.doi.org/10.5296/jse.v6i2.9201
• Hernández-Nieto, R. A. (2002). Contributions to Statistical Analysis BookSurge Publishing.
• Kintsch, W., & Rawson, K. A. (2013). Compreensão. In M. J. Snowling, & C. Hulme (Orgs.). A ciência da leitura, (pp. 227-244). Editora Penso: Porto Alegre.
• Lenggono, B., & Tentama, F. (2020). Construction measurement of academic procrastination of eleventh grade high school students in Sukoharjo. International Journal of Scientific and Technology Research, 9(01), 454-459. http://eprints.uad.ac.id/id/eprint/20077
  » http://eprints.uad.ac.id/id/eprint/20077
• Leopold, C., & Leutner, D. (2015). Improving students’ science text comprehension through metacognitive self-regulation when applying learning strategies.Metacognition and Learning,10(3), 313-346. https://doi.org/10.1007/s11409-014-9130-2
  » https://doi.org/10.1007/s11409-014-9130-2
• Lorenzo-Seva, U., & Ferrando, P. J. (2020). Factor Analysis. Recuperado de https://psico.fcep.urv.cat/utilitats/factor/
  » https://psico.fcep.urv.cat/utilitats/factor/
• Marôco, J. (2014).Análise de equações estruturais: Fundamentos teóricos, software & aplicações ReportNumber.
• Ministério da Educação, Brasil (2017). Base Nacional Comum Curricular Retrieved from http://basenacionalcomum.mec.gov.br/images/BNCC_publicacao.pdf
  » http://basenacionalcomum.mec.gov.br/images/BNCC_publicacao.pdf
• Ministério da Educação, Brasil(2018). Prova Brasil: Apresentação Retrieved from http://portal.mec.gov.br/prova-brasil
  » http://portal.mec.gov.br/prova-brasil
• Muthén, L. K., & Muthén, B. O. (2012). Mplus User’s Guide: Statistical Analysis with Latent Variables (7th ed.). Muthén & Muthén.
• Peterson, R. A., & Kim, Y. (2013). On the relationship between coefficient alpha and composite reliability. Journal of Applied Psychology, 98(1), 194-198.
• Pinto, N. S., Martínez, A. I. M., & Jiménez-Taracido, L. (2015). Estrategias de aprendizage, comprensión lectora y rendimiento académico en Educación secundaria, Psicologia Escolar e Educacional, 20(3), 447-456. http://dx.doi.org/10.1590/2175-3539/2015/02031011
  » http://dx.doi.org/10.1590/2175-3539/2015/02031011
• Schunk, D. H., & Usher, E. L. (2013). Barry J. Zimmerman’s theory of self-regulated learning. In H., Bembenutty, A., Kitsantas, & T. J., Cleary (Eds.). Applications of self-regulated learning across diverse disciplines: A tribute to Barry J. Zimmerman, (pp. 1-28). Information Age Publishing.
• Toland, M. D., & Usher, E. L. (2016). Assessing mathematics self-efficacy: How many categories do we really need?. The Journal of Early Adolescence, 36(7), 932-960. https://doi.org/10.1177/0272431615588952
  » https://doi.org/10.1177/0272431615588952
• White, M. C., & DiBenedetto, M. K. (2015). Self-regulation and the Common Core: Application to Ela Standards Routledge.
• Wigfield, A., Klauda, S. L., & Cambria, J. (2015). Influences on the development of academic self-regulatory processes. Em M. C. White, & M. K.DiBenedetto (Orgs.). Self-regulation and the Common Core: Application to Ela Standards, (pp. 33-48). Routledge.
• Xu, Z. (2016). Just do it! Reducing academic procrastination of secondary students.Intervention in School and Clinic,51(4), 212-219. https://doi.org/10.1177/1053451215589178
  » https://doi.org/10.1177/1053451215589178
• Zacks, S., & Hen, M. (2018). Academic interventions for academic procrastination: A review of the literature.Journal of Prevention & Intervention in the Community,46(2), 117-130. https://doi.org/10.1080/10852352.2016.1198154
  » https://doi.org/10.1080/10852352.2016.1198154
• Ziegler, N., & Opdenakker, M. C. (2018). The development of academic procrastination in first-year secondary education students: The link with metacognitive self-regulation, self-efficacy, and effort regulation.Learning and Individual Differences,64, 71-82. https://doi.org/10.1016/j.lindif.2018.04.009
  » https://doi.org/10.1016/j.lindif.2018.04.009
• Zimmerman, B. J., & Risemberg, R. (1997). Self-Regulatory Dimensions of Academic Learning and Motivation. In G. D. Phye (Ed). Handbook of Academic Learning, 105-125. Academic Press. https://doi.org/10.1016/b978-012554255-5/50005-3
  » https://doi.org/10.1016/b978-012554255-5/50005-3

• 1
  The scales investigated here comprise the Multidimensional Battery of Self-Regulation for Reading Comprehension (Ferraz, 2022), which covers the assessment of six dimensions of self-regulation, namely, motivation, method, time management, self-perceived behavior, the physical environment, and the social environment.

Publication Dates

• Publication in this collection
  20 Oct 2023
• Date of issue
  2023

History

• Received
  08 July 2021
• Accepted
  10 Nov 2021