Factor structure of the Rutter Teacher Questionnaire in Portuguese children

Estrutura fatorial do Questionário de Rutter para Professores numa amostra de crianças portuguesas

ORIGINAL ARTICLE

Factor structure of the Rutter Teacher Questionnaire in Portuguese children

Estrutura fatorial do Questionário de Rutter para Professores numa amostra de crianças portuguesas

Ana Telma Pereira^I; Berta Rodrigues Maia^I, Mariana Marques^I; Sandra Carvalho Bos^I; Maria João Soares^I; Ana Gomes^II; António Macedo^I; Maria Helena Pinto de Azevedo^I

^IInstitute of Medical Psychology, School of Medicine, University of Aveiro, Aveiro, Portugal

^IIDepartment of Educational Sciences, University of Aveiro, Campus Universitário de Santiago, Aveiro, Portugal

^{Correspondence}

ABSTRACT

OBJECTIVE: To examine the factor structure of the Rutter Teacher Questionnaire in Portuguese primary school children.

METHOD: The Rutter Teacher Questionnaire, a 26-item scale covering a variety of behavioral problems, was completed by teachers of 877 children, aged 6 to 11 years. Data were subjected to factor analysis using the principal components solution with varimax rotation.

RESULTS: The factorial analysis in total sample revealed three factors explaining 38.88% of the total variance. The factors contained items representing hyperactivity/conduct (Factor 1), anxious/depressive (Factor 2) and truancy/stealing (Factor 3). The highest correlations between factors scores were for Factor 1 and Factor 3. These Factors scores were higher in boys than girls and correlated with lower social class. All three Factors scores correlated with school performance. The comparison between separate factorial structures for the samples of boys and girls revealed a considerable overlap.

CONCLUSIONS: The pattern of the items contained on Factor 1 appears to be related with the category of hyperkinetic conduct disorder used by the International Classification of Diseases-10. Results suggest that the Portuguese language version of the Rutter Teacher Questionnaire possesses good psychometric properties and may be considered a useful instrument for measuring children's behavior problems.

Descriptors: Portugal; Questionnaires; Factor analysis, statistical; Child behavior; Evaluation studies

RESUMO

OBJETIVO: Analisar a estrutura fatorial do Questionário de Rutter para Professores numa amostra de crianças portuguesas do 1º Ciclo do Ensino Básico.

MÉTODO: O questionário, constituído por 26 itens que avaliam problemas do comportamento, foi preenchido pelos professores de 877 crianças (6-11 anos). As respostas foram sujeitas a uma análise fatorial, por meio do método de componentes principais com rotação ortogonal varimax. RESULTADOS: Na amostra total, a estrutura fatorial resultou em três fatores que explicam 38,88% da variância total e que foram denominados: problemas de hiperatividade/conduta (Fator 1), ansiedade/depressão (Fator 2) e vadiagem/furto (Fator 3). A correlação entre os fatores 1 e 3 foi a mais elevada. As pontuações fatoriais foram significativamente mais elevadas nos rapazes do que nas raparigas e apresentaram uma relação inversa com a classe social e com o rendimento escolar. As estruturas fatoriais realizadas separadamente para rapazes e raparigas revelaram grandes similitudes.

CONCLUSÕES: Os itens do Factor 1 parecem relacionar-se com o distúrbio hipercinético da conduta proposto pela Classificação Internacional de Doenças-10. Os resultados sugerem que a versão portuguesa do Questionário de Rutter para Professores apresenta parâmetros psicométricos adequados, podendo ser útil na avaliação dos problemas de comportamento das crianças.

Descritores: Portugal; Questionários; Análise fatorial; Comportamento infantil; Estudos de avaliação

Introduction

The Behavior Questionnaire developed by Rutter¹ for completion by teachers (Rutter Scale B2) is a well known screening instrument for epidemiological studies of children with emotional and behavioral disorders.²

In a recent study on the validity of child psychiatric screening methods using ROC analysis,³ it was found that the Rutter Teacher Questionnaire (RB2) was the most valid and had the best overall power to discriminate psychiatric disturbances in comparison with the Parent Questionnaire (Scale A2)⁴ and the Children Depression Inventory.³ In fact, teachers are important and reliable informants about children problematic behaviors and in a considerable amount of times are them who first notice the need of a further evaluation.⁵

The 26-item Rutter Scale B2¹ includes three subscales: emotional, antisocial and hyperactive. The scale was applied to Finnish⁶ and Chinese populations⁷ and its psychometric properties were studied using factor analysis. The Finish version⁶ was based on teachers reports of 5,871 children (mean age 8.5 years) and resulted in a 4-factor solution structure, with Externalizing, Hyperactive, Internalizing and School Refusing factors, while the Chinese version,⁷ using teachers ratings of 3,069 children (7-year old boys), found a 3-factor solution with Antisocial, Hyperactive and Neurotic factors.

In a cross-sectional survey of sleep-wake patterns in school-aged children, the Portuguese version of the Rutter Scale B2 was used to assess behavioral disturbances.⁸ Its inter-rater and retest reliability has already been found to be satisfactory.⁹ However, to our knowledge the factorial structure has not been studied so far. As factorial analysis should always be tested when an instrument is going to be used in a new population as a form of construct validation,¹⁰ the aim of this study was to explore the Rutter Scale B2 factorial structure in a large sample of Portuguese children. It would be expected that the factor structure of the Portuguese version of the Rutter Scale B2 would be similar to the ones obtained in previous studies.

Moreover, as conduct, attention and hyperactivity problems are more frequently observed in boys and emotional disorders in girls, and since associations between low socioeconomic classes, poor schooling and children behavioral problems are often described in the literature,¹¹ a separate factor analysis by gender was conducted and associations between social class, poor schooling and emotional/disruptive behaviors were explored.

Method

1. The Rutter Scale B2

The Rutter Scale B2¹ for completion by teachers has 26 items concerning the child's behavior at school and the teacher is asked for each to indicate whether it "certainly applies" (scored = 2), "applies somewhat" (scored = 1) or "doesn't apply" (scored = 0). A total deviance score is derived from the sum of scores for the individual items; the scores may range from zero to 52. In addition to the total score, three subscores may be obtained from the sum of the scores for certain items. The emotional or neurotic subscore is a summation of the scores of four items (often worried, miserable, fearful and tears on arrival at school), the conduct or antisocial subscore is obtained from the sum of scores of six items (destructive, fights, disobedient, lies, steals and bullies) and the hyperactivity subscore is a result of the sum of scores of three items (restless/ overactive, poor concentration and fidgety/squirmy). Rutter found that a cumulative score of nine points or more on the total scale is an indicator of the presence of some psychiatric disorders.

In order to assess teacher's knowledge and educational performance of their pupils the following items were added to the Scale: 1) How well do you know this child? (Answers: Very well/ Moderately well/Not very well); and 2) How do you rate this child concerning his/her school performance? (Answers: very poor = 1; poor = 2; within average = 3; good = 4; very good = 5).

The Portuguese version of the Teacher Scale B2 was a translation of the original questionnaire done by a Portuguese psychiatrist with experience and training in psychometrics and in child behavioral problems.⁹ All correlation coefficients computed for the test-retest (total score r = 0.77, emotional subscore = 0.70, conduct subscore = 0.71) and inter-rater (total score r = 0.56, emotional subscore = 0.44, conduct subscore = 0.66) reliability study were highly significant (p < 0.001). Correlation coefficients of individual items ranged from 0.31 (item 22 often complains of pains and aches) to 0.81 (item 4 often destroys own or others' belongings). The levels of significance gave values of p < 0.001 for all items, except for item 22, p < 0.01.

2. Subjects and procedure

The project was reviewed and approved by the Regional Director of Education, which serves as the Institutional Human Subject Protection Committee for the schools.

A total of 1,381 children (grades 1 to 4), of both genders, were enrolled during the school year of 1994-1995 when the epidemiologic survey of sleep-wake patterns was conducted. A previous paper gives full details of methods, which will be summarized here.¹² All 10 schools located in a parish of the city of Coimbra agreed to participate after explaining the aims of the project to all teachers involved in the study. This particular parish was selected because it is the most populous in the city and is considered representative of all social-cultural and economic groups. The ratings were made in the last term of the school year (April-July 1995). Responses to the teachers Scale B2, which were voluntary and dependent upon permission from parents, were obtained for 877 children (response rate 63.50%).

3. Statistical analyses

All statistical analyses were carried out using SPSS for Windows, version 13.0. Factor analysis was performed on the 26-item scale using principal factor solution with varimax rotation. The strategies used to extract the number of factors were 1) the Kaiser criteria,¹³ which determines that components with eigenvalues lower than one should be excluded and 2) the scree test of Cattell criteria,¹⁴ which implies the retention of all components in the sharp descent part of the plot before the eigenvalues start to level off, where line changes slope.¹⁵ Cronbach's alpha was used to assess internal consistency of the factors. Spearman's rho coefficient (r_s), Pearson's product moment correlations (r) and t-tests were carried out as appropriate.

Results

1. Sample characteristics

The sample consisted of 877 Caucasian children (50.9% Girls, n = 446). Their mean age was 7.90 years, SD=1.300 (range: 6-11 years). There were no gender differences in mean age (Girls: M = 7.91, SD = 1.324; Boys: M = 7.89, SD = 1.277; t = 0.194, df = 875, p = 0.846, NS). The vast majority of the children were attending the first three school grades during the survey (77.4%). With respect to social background, as indicated by parental occupational status, the distribution of the sample was as follows: 23.4% Social class I (higher), 11.2% Social Class II, 43.1% Social Class III, 22.4% Social Class IV (lower).

With regard to the question 'How well do you know this child?' most of the teachers (96.2%) referred that they knew their students very well or reasonably well.

Teacher ratings of school performance are shown in Table 1. The lower the score, the poorer is the performance. The total sample mean score was 3.64 (SD = 0.92; range = 1-5). Significant gender differences were found with respect to school performance. Boys had lower mean levels than girls (M = 3.72, SD = 0.92 vs. M = 3.56, SD = 0.91; t = 2.531, p = 0.012).

2. Factor analysis

The factorial analysis revealed a meaningful three factors solution explaining 38.88% of the total variance. Items loading above 0.40 on each factor are shown in Table 2. Cronbach's alpha coefficients obtained for each factor were: Factor 1 (10 items) = 0.880, Factor 2 (five items) = 0.655 and Factor 3 (five items) = 0.584.

The first factor (F1: Hyperactivity/conduct) explained 23.34% of the total variance and consisted of items representing hyperactivity/ conduct problems. Four out of the 10 items loading on F1 are shared with the original Rutter conduct subscore (destructive, fights, disobedient and bullies) and three items correspond to the original Rutter hyperactivity subscore (restless/overactive, poor concentration and fidgety/squirmy). Factor 2 (F2: Anxious/depressive) explained 9.48% of the total variance and contained items appearing to reflect anxious/depressive behaviors (fearful, miserable, worried, solitary, aches and pain). Three of these items (fearful, miserable and worried) were included by Rutter in his emotional/neurotic subscore. Factor 3 (F3: Truancy/stealing) explained 6.06% of the total variance and consisted of items reflecting truancy/stealing behaviors (lies, truants, steal, off school for trivial reasons and sucks thumb). Two items of this factor (lies and steals) are part of the Rutter conduct/anti-social subscore.

Six items had loadings below 0.40: twitches, bites nails, other speech disorders, tears on arrival at school, wets or soils and stutter.

The mean Factors scores were as follows: F1, M = 2.93 (SD = 3.723, range = 0-20); F2, M = 1.15 (SD = 1.512, range = 0-10) and F3, M = 0.39 (SD = 0.946, range = 0-8). Boys scored significantly higher than girls in F1 (M = 3.81, SD = 4.135 vs. M = 2.08, SD = 3.049, t = -7.000, df = 789.775, p = 0.000) and F3 (M = 0.469, SD = 1.097 vs. M = 0.316, SD = 0.768, t = -2.378, df = 767.456, p = 0.018). With regard to F2 there were no significant gender differences (M = 1.09, SD = 1.529 vs. M = 1.22, SD = 1.496, t = 1.221, df = 875, p = 0.222).

Spearman's Correlations coefficients (r_s) between factors were all positively significant. F1 was strongly correlated with F3 in both genders (boys, r_s = 0.412; girls, r_s = 0.377) and moderately correlated with F2 (boys, r_s = 0.233; girls, r_s = 0.293). The correlation between F2 and F3 was small (boys, r_s = 0.113; girls, r_s = 0.146). With respect to the overall sample, the correlation between F1 and F3 was strong (r_s = 0.411), between F1 and F2 was moderate (r_s = 0.221) and between F2 and F3 was small (r_s = 0.174).

3. Factorial analysis by gender

A factor analysis of the Rutter Scale B2 data was performed separately for boys and girls. Three factors were equally obtained for boys and girls and the respective structures showed great similarities with those of the total sample (Tables 3, 4, 5).

In relation to Hyperactivity/conduct factor (F1), the 10 items loading on the boys subsample were exactly the same as the items loading on F1 in the total sample. In the female subsample, F1 was slightly different from the other two factor structures. The item poor concentration instead of loading on F1 (as in the total sample and in the boys subsample) loaded on F3; the item twitches, which showed a considerable loading on girls F1, was not present in any factor of the other two structures (Table 3).

With respect to items loading on Anxious/depressive factor (F2) almost all items were the same in the three samples. The exception was the item sucks thumb that loaded on F2 in the boys subsample but not in the total or female samples (Table 4).

Concerning Truancy/stealing factor (F3), four items were the same in the three structures: lies, off school for trivial reasons, truants and steals. However, an extra item loaded on this factor in the total sample (sucks thumb), as well as in the girls subsample (poor concentration) - Table 5.

4. Correlation analysis with factor scores and sociodemographic variables

No significant correlations (r = Pearson correlation coefficient) were found between factors scores (mean scores on items loading in each factor) and age (maximum correlation was r = -0.082).

The correlations between social class and F2 were not significant in the overall sample, as well as in the boys and the girls subsamples. However, F1 was positively and significantly correlated with social class (boys, r_s = 0.175; girls, r_s = 0.187; overall, r_s = 0.161), as well as with F3 (boys, r_s = 0.222; girls, r_s = 0.203; overall, r_s = 0.209), indicating that in both genders the lower social class was related to higher hyperactivity/conduct and truancy/stealing scores.

Concerning the association between school performance and the three factors, significant and negative correlations were observed, indicating that high levels of behavioral disturbance were associated with poor schooling: F1 (boys, r_s = -0.233; girls, r = -0.359; overall, r= -0.312); F2 (boys, r= -0.118; girls, r = -0.206; overall, r= -0.158); F3 (boys, r= -0.269; girls, r_s = -0.275; overall, r_s = -0.278).

Discussion and conclusions

The factor analysis of the Portuguese Rutter Teacher Questionnaire revealed three distinct factors: hyperactivity/conduct factor (F1), anxious/depressive factor (F2) and truancy/stealing factor (F3).

The largest component obtained was for F1, which accounted for 23.34% of the variance, and consisted predominantly of a mixture of hyperactivity (restless/overactive, poor concentration and fidgety/ squirmy) and conduct behaviors (destructive, fights, disobedient, lies, steals and bullies). The internal consistency of this factor was high as the alpha coefficient obtained was 0.88.

The second factor (F2) contained items reflecting anxious/ depressive behaviors (fearful, miserable, worried, solitary, aches and pain) and accounted for 9.48% of the variance. F3 consisted of items reflecting truancy/stealing behaviors (lies, truants, steals and off school for trivial reasons) and accounted for 6.06% of the variance. The alpha coefficient for F2 (five items) was 0.66 and for F3 (five items) was 0.58.

As expected, the highest correlation coefficients were found between F1 (hyperactivity/conduct factor) and F3 (truancy/stealing factor) as both contained items representing externalizing problems.

Although there were no significant correlations between factor scores and age, it is important to note that our sample did not include adolescents.

There was also no significant correlations of F2 scores (anxious/ depressive, internalizing problems) with social class. Significant correlations although modest were found between F1 and social class, as well as between F3 and social class indicating that higher scores on externalizing problems were associated with lower social class in both genders, which confirms previous findings.¹¹ Although the significant association found between poor schooling and behavioral disturbances has also been reported in other studies,¹⁶ it may be exacerbated by the fact that the two variables were rated by the same person (the teacher).

When comparing the Portuguese RB2 factor structure with the original Rutter subscales¹ and with the Finnish⁶ and the Chinese⁷ factor structures, a few similarities and some differences were observed. It is important to note that the Rutter's subscales were not derived via factor analysis and that the Finnish and Chinese research groups used a more recent version of the RB2 (a few items were renamed).¹⁷

A considerable correspondence across cultures was observed with respect to the emotional/internalizing factor. In fact, the Portuguese F2 (anxious/depressive factor) was similar to the Rutter emotional/neurotic subscore, the Finnish internalizing factor and the Chinese neurotic factor. The major difference was in the Portuguese hyperactive/conduct factor as this factor included items which were part of two distinct factors, hyperactive factor and conduct/antisocial behavior factor, in other Countries (Finland and China).^6,7

Five out of the 10 items of the Portuguese hyperactive/conduct F1 (fidgety/squirmy, destructive, disobedient, poor concentration and restless/o322veractive) are part of the hyperactive Finnish factor and three items (fidgety, restless and cannot settle/poor concentration) were part of the Rutter hyperactivity subscore and the Chinese hyperactive factor. Although some items of the Portuguese hyperactive/conduct factor belong to the hyperactive factor of other Countries, four items of this factor (destructive, fights, disobedient and bullies) are part of the Rutter conduct/anti-social subscore and four items (fights, irritable, bullies and not liked) are part of the Finnish externalizing factor. The majority of the Portuguese Rutter scale B2 hyperactive/conduct factor items (destructive, fights, disobedience, bullies, irritable, not liked by others and fussy) belong to the Chinese anti-social factor. This last difference could be in part explained by the fact that Chinese families and teachers are more concerned about discipline and self-control.¹⁸

Moreover, items of the truancy/stealing Portuguese factor F3 (lies, off school for trivial reasons, truants, steals and sucks thumb) were not present in the Chinese Rutter scale B2 study. Two items of this factor (lies and steals) were part of the Finnish externalizing factor and part of the Rutter conduct/anti-social subscore, and two other items (truants and off school) were part of the Finnish school refusal factor.

Thus, although a considerable correspondence exists across cultures with respect to the emotional/internalizing factor, a major difference exists concerning the Portuguese hyperactive/conduct factor as this factor congregates items which are separated in two distinct factors (hyperactive factor and conduct/antisocial behavior factor) in other Countries.

The pattern of the items loading on the Portuguese F1 (hyperactive/ conduct factor) appears to be related to the category of hyperkinetic conduct disorder used by the ICD-10.¹⁹ Indeed, it has consistently been observed that hyperkinetic disorder (ICD-10)/Attention deficit hyperactivity disorder (DSM-IV)²⁰ and conduct disorder commonly co-occur, and family and twin study findings suggest that much of this overlap is due to a common genetic etiology.²¹

In conclusion, the factor structure of the Portuguese Rutter scale B2 is novel and reflects the hyperkinetic conduct disorder described in ICD-10¹⁹ as hyperactive and conduct problems emerged as a single factor in our study. Although concurrent validity was not explored, findings of the present study together with the results on the inter-rater and retest reliability analysis suggest that the Portuguese language version of Rutter Scale B2 possesses good psychometric properties and may be considered a useful instrument for measuring children's behavior problems.

Acknowledgements

The cooperation of school head masters, teachers, and parents is gratefully acknowledged. No potential conflicts of interest.

Submitted: May 13, 2008

Accepted: September 6, 2008

References

• 1. Rutter MA. Children's Behavior Questionnaire for completion by the teachers: preliminary findings. J Child Psychol Psychiatry 1967;8(1):1-11.
• 2. Elander J, Rutter M. An update on the status of the Rutter Parents' and Teachers' Scales. Child Psychol Psychiatry Rev 1996;1(1):31-5.
• 3. Kresanov K, Tuominen J, Piha J, Almqvist F. Validity of child psychiatric screening methods. Eur Child Adolesc Psychiatry 1998;7(2):85-95.
• 4. Rutter M, Graham PH. Psychiatric disorder in 10 and 11 year-oldchildren. J R Soc Med 1966;59:382-7.
• 5. Achenbach TM, McConaughy SH, Howell CT. Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychol Bull 1987;101(2):213- 32.
• 6. Kumpulainen K, Räsänen E, Henttonen I, Moilanen I, Piha J, Puura K, Tamminen T, Almqvist F. Children's behavioural/emotional problems: a comparison of parents' and teachers' reports for elementary school-aged Children. Eur Child Adolesc Psychiatry 1999;8(Suppl. 4):41-7.
• 7. Ho TP, Patrick WL, Leung ESL, Taylor E, Bacon-Shone J, Mak FL. Establishing the constructs of childhood behavioral disturbances in a Chinese population: a questionnaire study. J Abnorm Child Psychol 1996;24(4):417-31.
• 8. Clemente VO. Epidemiologia dos Distúrbios do Sono em Crianças do Ensino Básico [dissertation]. Faculty of Psychology: Coimbra University; 1997.
• 9. Azevedo MHP, Barreto MM, Faria MM, Robalo MM. Questionário comportamental para ser completado por professores: estudo de fidedignidade. Psiquiatr Clin. 1986;7:203-7.
• 10. Loewenthal KM. An introduction to psychological tests and scales. 2nd edition. London: Psychology Press; 2001.
• 11. Almqvist F, Kumpulainen K, Ikaheimo K, Linna SL, Henttonen I, Huikko E, Tuompo-Johansson E, Aronen E, Puura K, Piha J, Tamminem T, Rasanen E, Moilanen I. Behavioural and emotion symptoms in 8-9 year old children. Eur Child Adolesc Psychiatry. 1999;8(Suppl 4):7-16.
• 12. Ferreira AM, Clemente V, Gozal D, Gomes A, Pissarra C, César H, Coelho I, Silva CF, Azevedo MH. Snoring in Portuguese primary school children. Pediatrics 2000;106(5):E64.
• 13. Kaiser HF. The varimax criterion for analytic rotation in factor analysis. Psychometrika. 1958;23:187-200.
• 14. Cattell RB. The scree test for the number of factors. Multivariate Behav Res. 1966;1:245-76.
• 15. Kline P. An easy guide to factor analysis. Routledge: London and New York; 1994.
• 16. Ekblad S. The Children behaviour questionnaire for completion by parents and teachers in a Chinese sample. J Child Psychol Psychiatry. 1990;31(5):775-91.
• 17. Rutter M, Tizard J, Whitmore K. Education, health and behaviour. London: Longman; 1970.
• 18. Luk ESL, Leung PW, Ho T. Cross-cultural/ethnic aspects of childhood hyperactivity. In: Sandberg S, editor. Hyperactivity disorders. Cambridge: Cambridge University Press; 2002. p. 64-98.
• ¹⁹ World Health Organization. The ICD-10 Classification of Mental and Behavioral Disorders Diagnostic Criteria for Research. Geneve: World Health Organization; 1994.
• ²⁰ American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th edition. Washington: APA; 1994.
• 21. Faraone SV, Biederman J, Jetton JG, Tsuang MT. Attention deficit disorder and conduct disorder: longitudinal evidence for a familial subtype. Psychol Med 1997;27(2):291-300.

Publication Dates

• Publication in this collection
  17 Dec 2008
• Date of issue
  Dec 2008

History

• Accepted
  06 Sept 2008
• Received
  13 May 2008