Child Behavior Checklist (CBCL),Youth Self-Report (YSR) and Teacher's Report Form(TRF): an overview of the development of the original and Brazilian versions

Child Behavior Checklist (CBCL), Youth Self- Report (YSR) y Teacher's Report Form (TRF): una visión general sobre el desarrollo de las versiones originales y brasileñas

Child Behavior Checklist (CBCL), Youth Self-Report (YSR) e Teacher's Report Form (TRF): uma visão geral sobre o desenvolvimento das versões originais e brasileiras

REVIEW REVISÃO

Child Behavior Checklist (CBCL),Youth Self-Report (YSR) and Teacher's Report Form(TRF): an overview of the development of the original and Brazilian versions

Child Behavior Checklist (CBCL), Youth Self-Report (YSR) e Teacher's Report Form (TRF): uma visão geral sobre o desenvolvimento das versões originais e brasileiras

Child Behavior Checklist (CBCL), Youth Self- Report (YSR) y Teacher's Report Form (TRF): una visión general sobre el desarrollo de las versiones originales y brasileñas

Isabel A. Bordin^I; Marina M. Rocha^II; Cristiane S. Paula^I,III; Maria Cristina T. V. Teixeira^III; Thomas M. Achenbach^IV; Leslie A. Rescorla^V; Edwiges F. M. Silvares^II

^IDepartamento de Psiquiatria, Universidade Federal de São Paulo, São Paulo, Brasil

^IIInstituto de Psicologia, Universidade de São Paulo, São Paulo, Brasil

^IIIPrograma de Pós-graduação em Distúrbios do Desenvolvimento, Universidade Presbiteriana Mackenzie, São Paulo, Brasil

^IVCenter for Children Youth and Families, University of Vermont, Burlington, U.S.A.

^VBryn Mawr College, University of Pennsylvania, Bryn Mawr, U.S.A.

^{Correspondence}

ABSTRACT

The Achenbach System of Empirically Based Assessment (ASEBA) for school-age children includes three instruments for assessing emotional and/or behavioral problems: Child Behavior Checklist (CBCL), completed by parents, Youth Self-Report (YSR), completed by adolescents and Teacher's Report Form (TRF), completed by teachers. This review article gives detailed information on the development of these forms in the United States and Brazil, describing the main changes to the items, scales and score cut-off points in original versions between 1991 and 2001, as well as the process involved in the translation, back-translation and cultural adaptation of the original questionnaires to develop the current official Brazilian versions of the CBCL, YSR and TRF. The utility of these tools for research and clinical practice is highlighted, mentioning epidemiological studies and evaluation of interventions conducted in Brazil. Researchers' and clinicians' doubts regarding the correct use of the current official Brazilian versions are answered, giving examples of frequently asked questions relevant to the Brazilian context.

Child; Adolescent; Mental Health; Questionnaires; Evaluation

RESUMO

O Sistema de Avaliação de Base Empírica de Achenbach para crianças/adolescentes em idade escolar inclui três instrumentos para avaliar problemas emocionais e/ou comportamentais: Child Behavior Checklist (CBCL) [pais], Youth Self-Report (YSR) [adolescentes] e Teacher's Report Form (TRF) [professores]. Este artigo de revisão fornece informações detalhadas sobre o desenvolvimento desses instrumentos nos Estados Unidos e no Brasil, descrevendo as principais alterações em itens, escalas e pontos de corte na pontuação, ocorridas nas versões originais de 1991 a 2001, e o processo de tradução, retrotradução e adaptação cultural dos questionários originais para desenvolver as atuais versões brasileiras oficiais do CBCL, YSR e TRF. A utilidade desses instrumentos em pesquisa e na prática clínica é salientada, mencionando estudos epidemiológicos e de avaliação de intervenções conduzidos no Brasil. Pesquisadores e clínicos são instruídos a respeito do uso correto das atuais versões brasileiras oficiais, dando exemplos de perguntas frequentes, relevantes para o contexto brasileiro.

Criança; Adolescente; Saúde Mental; Questionários; Avaliação

RESUMEN

El sistema de evaluación de base empírica de Achenbach para niños/adolescentes en edad escolar incluye tres instrumentos para evaluar problemas emocionales y/o de comportamiento: Child Behavior Checklist (CBCL) [padres], Youth Self-Report (YSR) [adolescentes] y Teacher's Report Form (TRF) [profesores]. Este artículo de revisión proporciona información detallada sobre el desarrollo de estos instrumentos en los Estados Unidos y en Brasil, describiendo las principales alteraciones en ítems, escalas y puntos de corte en la puntuación, que se realizaron en las versiones originales de 1991 a 2001, y el proceso de traducción, retrotraducción y adaptación cultural de los cuestionarios originales, con el fin de desarrollar las actuales versiones brasileñas oficiales del CBCL, YSR y TRF. La utilidad de estos instrumentos en investigación y en la práctica clínica se resalta mencionando estudios epidemiológicos y de evaluación de intervenciones, llevados a cabo en Brasil. Investigadores y personal clínico son instruidos en lo que se refiere al uso correcto de las actuales versiones brasileñas oficiales, dando ejemplos de preguntas frecuentes y relevantes para el contexto brasileño.

Niño; Adolescente; Salud Mental; Cuestionarios; Evaluación

Introduction

In 2008, Holmbeck et al. ¹ published an evidence-based review of measures of psychosocial adjustment and psychopathology, with a specific focus on their use in the field of pediatric psychology. The authors found only eight measures in the category "broad-band rating scales" (i.e., measures with a broad coverage of psychological adjustment constructs) that could be considered well-established forms of assessment. Two of these eight measures were the Achenbach System of Empirically Based Assessment (ASEBA) for school-age children published in 1991 ^2,3,4 and 2001 ⁵. Holmbeck et al. ¹ defined "well-established forms of assessment" as measures with the following characteristics: (1) being the focus of at least two peer-reviewed articles by different investigators; (2) being described with sufficient detail to allow critical evaluation and replication (e.g., measure and manual provided or available upon request); and (3) availability of detailed information indicating good validity and reliability in at least one peer-reviewed article.

ASEBA school-age instruments include the Child Behavior Checklist (CBCL), the Youth Self-Report (YSR) and the Teacher's Report Form (TRF). These questionnaires are high quality standardized screening measures of child and adolescent emotional/behavioral problems and social competencies developed by Achenbach used with parents, adolescents and teachers, respectively. The ASEBA offers a comprehensive approach to assessing adaptive and maladaptive functioning of children and adolescents. It is widely used in mental health services, schools, medical settings, child and family services, public health agencies, child guidance, training, and research. There are translations of ASEBA instruments in over 80 languages and more than 7,000 publications report the use of ASEBA materials involving the work of 9,000 authors from over 80 cultural groups and societies (http://www.aseba.org, accessed on 25/Jul/2011).

The current review article gives an overview of the origin and development of the ASEBA approach, describes the original ASEBA school-age instruments developed in 1991 and 2001 at the University of Vermont (Burlington, USA), and highlights the main changes in items, scales and score cut-off points that occurred between 1991 and 2001. This review also gives an overview of the development of the ASEBA school-age instruments in Brazil, presenting studies conducted to evaluate their psychometric properties, and describes the development process of the most recent official Brazilian versions of the CBCL/6-18, YSR/11-18 and TRF/6-18, including translation, back-translation and cultural adaptation of the original questionnaires. This review also highlights the utility of these tools for child and adolescent mental health research and clinical practice, mentioning examples of epidemiological studies and evaluations of interventions conducted in Brazil. Information is also given to instruct clinicians and researchers about the correct use of the current official Brazilian versions, with examples of frequently asked questions relevant to the Brazilian context.

The origin and development of the ASEBA approach: an overview

The ASEBA approach originated in the 1960s with Achenbach's efforts to develop a more differentiated picture of child and adolescent psychopathology than that provided by the prevailing diagnostic system. At that time, the American Psychiatric Association's Diagnostic & Statistical Manual (DSM) provided only two categories of childhood disorders: Adjustment Reaction of Childhood and Schizophrenic Reaction, Childhood Type (http://www.aseba.org, accessed on 25/Jul/2011). In 1966, Achenbach was interested in investigating the diversity of symptoms that could bring children to psychiatric treatment. Data was collected on 1,000 psychiatric patients based on documented mental health cases and descriptions available in the literature that led to the production of a preliminary child behavior checklist ⁶. This list was later adapted to collect information from parents to serve as a basis for the development of the CBCL ⁷.

The child behavior profile derived from the CBCL was initially standardized for 6 to 11-year-old boys ⁷, and later expanded to both genders in the six to 16 years age range ⁸. In 1983, Achenbach & Edelbrock published the Manual for Child Behavior Checklist/4-16, and in 1991 a revised version of the CBCL was published for 4 to 18-year-old with no significant changes in the content of items compared to the 1983 version ². The most relevant changes in the 1991 version involving the child behavior profile included the establishment of a borderline range for scale T-scores and the development of the "cross-informant syndromes" to combine information from parents, youth and teachers ⁹.

The pre-1991 profiles for scoring the CBCL, YSR and TRF were developed separately. The pre-1991 syndrome scales functioned well for describing and assessing patterns that were empirically derived for specific sex/age groups as seen by a particular type of informant (parent, youth or teacher). The 1991 editions of the CBCL, YSR and TRF were designed to improve both the conceptual structure and the practical application of empirically based assessment by focusing more precisely on the syndromes that were common to both sexes and different age ranges, according to reports by each type of informant ^2,3,4.

In 2001, the CBCL/4-18 was revised to make it applicable to both genders in the six to 18-years age range (CBCL/6-18). DSM-oriented scales were also created to help users coordinate empirically based assessment with the diagnostic categories of DSM, and ASEBA Windows software was introduced to facilitate data entry, T-score calculation and profile printing ⁵ (http://www.aseba.org, accessed on 25/Jul/2011).

The original ASEBA school-age instruments (USA)

English versions from 1991: CBCL/4-18, YSR/11-18 and TRF/5-18

The CBCL/4-18, YSR/11-18 and TRF/5-18 are standardized measures of child and adolescent emotional/behavioral problems and social competencies that are completed by parents, adolescents and teachers, respectively. The social competence items of the CBCL/4-18 ² and YSR/11-18 ³ provide scores for three narrow-band scales (I. Activities, II. Social, III. School), and one broad-band scale for Total Social Competence. The adaptive functioning items of the TRF/5-18 provide scores for the child's performance in academic subjects and four adaptive characteristics (dedication to school work, appropriateness of behavior in school, ability to learn and happiness) ⁴. Behavior problem items provide scores for nine narrow-band scales or syndromes (I. Withdrawn, II. Somatic Complaints, III. Anxious/Depressed, IV. Social Problems, V. Thought Problems, VI. Attention Problems, VII. Delinquent Behavior, VIII. Aggressive Behavior, and IX. Sexual Problems), and three broad-band scales. (1. Internalizing Behavior Problems, that correponds to the sum of subscales Withdrawn, Somatic Complains and Anxious/Depressed; 2. Externalizing Behavior Problems, that corresponds to the sum of subscales Delinquent Behavior and Aggressive Behavior; and 3. Total Behavior Problems). Syndromes I to VIII are applicable to children aged four to 18 years and are derived from the three questionnaires, while syndrome IX is restricted to four to 11-year olds and is scored only from the CBCL/4-18. All three instruments have adequate psychometric properties ^2,3,4.

• Cross-informant syndromes

Eight cross-informant syndromes (I. Withdrawn, II. Somatic Complaints, III. Anxious/Depressed, IV. Social Problems, V. Thought Problems, VI. Attention Problems, VII. Delinquent Behavior, VIII. Aggressive Behavior) were derived from analyses of the CBCL/4-18, YSR/11-18 and TRF/5-18. The cross-informant syndromes reflect patterns of problems that are common to ratings by the different kinds of informants (parents, youth, and teachers). The items that constitute each cross-informant syndrome are those present in the respective syndrome of at least two of the three ASEBA school-age forms ⁹.

English versions from 2001: CBCL/6-18, YSR/11-18 & TRF/6-18

The three ASEBA school-age instruments, CBCL/6-18, completed by parents, YSR/11-18, completed by adolescents, and TRF/6-18, completed by teachers and other school staff, are standardized screening questionnaires internationally used to identify emotional/behavioral problems and social competencies in children and adolescents ⁵. The time frame for CBCL/6-18 and YSR/11-18 responses is usually the past six months, but the period of time may vary according to different study objectives. Because teachers may need to make periodic reassessments during the school-year, the time frame for administering the TRF/6-18 is usually the last two months. The CBCL/6-18 and YSR/11-18 were designed to be self-administered, but can be applied by a trained interviewer if parents have not completed elementary school or if adolescent are deficient in reading skills or reading comprehension.

• Items and scales

The three questionnaires have a similar structure comprising two sections: one for social competence/adaptive functioning and another for behavior problems (behavior profile). The social competence section of the CBCL/6-18 and YSR/11-18 includes 20 items scored from zero to four according to pre-established rules described in the manual ⁵. These social competence items provide scores for three narrow-band scales (Activities, Social, School), and one broad-band scale for Total Social Competence. The social competence section assesses: (1) children's involvement in activities (how much time they spend on sports, hobbies or games, and performance compared to same age peers; how active they are in the organizations, clubs, teams or groups to which they belong; how well they carry out jobs or chores); (2) social interaction patterns (how many close friends they have, how frequently they meet with friends, how well they get along with family members and other children, how independent they are when playing or working alone); and (3) school performance (performance in academic subjects, academic or other problems in school). This section also investigates illness and disability, major concerns and best things about the child/adolescent. The adaptive functioning items of the TRF/6-18 provide scores for the child's performance in different academic subjects on a scale of one (far below grade) to five (far above grade), and four adaptive characteristics on a scale of one to seven, including the pupil's dedication to school work ("How hard is he/she working?"), appropriateness of behavior in school ("How appropriately is he/she behaving?"), his/her ability to learn ("How much is he/she learning?"), and his/her current mood state ("How happy is he/she?"). Respondents are required to fill out information about the child's current academic performance, but data on achievement and ability tests are optional since this kind of information is not always available.

The behavior profile section of the three instruments comprises 118 items that can be scored as zero (not true), one (somewhat or sometimes true) or two (very true or often true). These items provide scores for eight narrow-band scales or syndromes (Anxious/Depressed, Withdrawn/Depressed, Somatic Complaints, Social Problems, Thought Problems, Attention Problems, Rule-Breaking Behavior, and Aggressive Behavior), and three broad-band scales (Internalizing Behavior Problems, Externalizing Behavior Problems, and Total Behavior Problems). The names given to these syndromes reflect the content of their items and were chosen from a familiar vocabulary to facilitate communication among mental health professionals and other questionnaire users. These are empirically derived syndromes identified by factor analysis, and must not be used as psychiatric diagnoses. Items from the syndromes or subscales Anxious/Depressed, Withdrawn/Depressed and Somatic Complaints are components of the Internalizing scale, while items from syndromes or subscales Rule-Breaking Behavior and Aggressive Behavior are components of the Externalizing scale. The Total Problem scale includes items from all syndromes. Furthermore, additional factor analyses of the TRF/6-18 Attention Problems syndrome produced two subscales: Inattention and Hyperactivity-Impulsivity. For all three instruments, raw scores are transformed into T-scores that indicate whether subjects present deviant behaviors or deficiencient competencies in relation to norms for their age and gender.

• Differences in items among the three instruments

Regarding social competence/adaptive functioning items, parents and teachers are required to inform about grade retention and if the child/adolescent is attending a special class or special school. Adolescents are not asked these questions, since they might not know, or be willing to give, the correct information, but are asked to describe their concerns regarding school. Regarding the TRF/6-18, respondents are asked to give information about their contact with the child/adolescent in school before giving information about the pupil. Questions about the number of months they are in contact with the pupil, how well they know him/her and how much time per week he/she spends in their class or service are examples of items that provide a perspective about the context in which the students is being evaluated. Teachers are also asked specific questions regarding the child's adaptive functioning, as described above.

Differences in the behavior profile section (Table 1) reflect contextual differences. Parents are asked to evaluate a few behaviors that are specific to the home environment (e.g., disobedient at home, sleeps less than most kids, wets the bed), while teachers are asked to evaluate a few behaviors specific to the school environment (e.g., disturbs other pupils, breaks school rules, has difficulty learning, sleeps during class). Consequently, the CBCL/6-18 and TRF/6-18 have 95 common items and 23 different items. The differences between the CBCL/6-18 and YSR/11-18 consist of 14 items in the YSR/11-18 that evaluate socially desirable characteristics, such as "I like to be fair to others" or "I try to help other people when I can". Items unique to the YSR/11-18 may replace CBCL/6-18 items that are not appropriate for the age range 11-18 years (e.g., thumb-sucking, wets self during the day), and provide scores for a new scale on positive qualities.

• Changes in items from 1991 to 2001 versions

Although no changes were made in the social competence items of the three instruments, modifications in format occurred from 1991 to 2001. Regarding the behavior profile, changes were made to six CBCL/6-18 items, six YSR/11-18 items and three TRF/6-18 items (Table 2). The reasons for these modifications were the low prevalence of this type of behavior in childhood and adolescence or the irrelevance of items in identifying psychopathology ⁵.

• T-score cut-off points

Raw scores derived from the social competence and behavior problem sections are transformed into T-scores to allow comparison with children from the same gender and age. T-score cut-off points for narrow-band and broad-band scales determine the degree of deviance from normality, categorizing children as clinical, borderline or non-clinical. The clinical category corresponds to low scores for social competence and high scores for emotional/behavioral problems, while the opposite applies to the non-clinical category. The borderline category spans an intermediate range of T-scores that indicates the need of follow-up the child or adolescent to identify a possible increase in symptoms and/or decrease in competence over time. It is interesting to note that children and adolescents may present T-scores in the clinical range for individual syndromes, while not presenting T-scores in the clinical range for the internalizing, externalizing or total problem scale.

Table 3 shows T-score cut-off points for the social competence and behavior profile narrow-band and broad-band scales of the 1991 and 2001 versions of the ASEBA school-age forms.

• Cross-informant comparisons

The ASEBA software introduced in 2001 makes it possible to print bar graphs that provide side-by-side comparisons of scores from up to eight informants for each narrow-band and broad-band scale from the behavior profile. These graphs make it easy for users to identify important similarities and differences between a child's scale scores according to parents, youth and teachers ⁵. It is interesting to note that certain problems may be consistently reported by multiple informants (e.g., externalizing problems reported by parents and teachers), whereas other problems may be reported by only one informant (e.g., internalizing problems reported only by youth).

The ASEBA software introduced in 2001 also makes it possible to identify correlations between scores from each pair of informants compared to correlations between large reference samples of similar pairs of informants ⁵. However, modest cross-informant correlations found by a number of studies and confirmed by meta-analyses ^10,11 show that no one source of assessment data can substitute the others. Comprehensive assessment therefore requires comparisons of data from multiple sources ⁶.

• DSM-oriented scales

When applying the CBCL/6-18, YRS/11-18 or TRF/6-18, problem items can be scored based on DSM-oriented scales, as well as on empirically based syndromes. The DSM-oriented scales were constructed by having experts from many cultures identify ASEBA problem items that they judged to be very consistent with particular diagnostic categories from the Diagnostic and Statistical Manual of Mental Disorders, 4^th Edition (DSM-IV) ¹². Items that were identified by a substantial majority of experts as being consistent with a particular diagnostic category were used to construct a DSM-oriented scale representing that category (http://www.aseba.org, accessed on 25/Jul/2011).

The DSM-oriented scales scored from the CBCL/6-18, YRS/11-18 and TRF/6-18 are: Affective Problems (items rated as very consistent with Dysthymia and Major Depressive Disorder); Anxiety Problems (items rated as very consistent with Generalized Anxiety Disorder, Separation Anxiety and Specific Phobia); Attention Deficit/Hyperactivity problems (items rated as very consistent with Inattentive and Hyperactive-Impulsive types of ADHD); Conduct Problems; Oppositional Defiant Problems; and Somatic Problems (items rated as very consistent with Somatization Disorder and Somatoform Disorder) ⁵. Like the syndrome scales, the DSM-oriented scales are scored by summing the 0-1-2 ratings of the constituent items, raw scores are transformed into T-scores based on normative data, and T-scores are displayed on profiles and in cross-informant comparisons (http://www.aseba.org, accessed on 25/Jul/2011).

Users should note that even scores in the clinical range for specific DSM-oriented scales are not directly equivalent to a DSM diagnosis. High scores in specific DSM-oriented scales suggest problems in specific areas, and identify children that deserve further mental health evaluation to confirm the need of psychiatric and/or psychological assistance.

• Psychometric properties of the original ASEBA school-age instruments from 2001

The CBCL/6-18, YSR/11-18 and TRF/6-18 (original versions in English) have good test-retest reliability and internal consistency ⁵. Mean test-retest reliabilities for empirically based syndromes for the CBCL/6-18, YSR/11-18 and TRF/6-18 were 0.90, 0.82, and 0.90, respectively. For DSM-oriented scales, test-retest reliabilities were 0.88, 0.79, and 0.85, respectively. Mean test-retest reliabilities for the CBCL and YSR competence scales were 0.90 and 0.88, respectively, whereas mean test-retest reliability was 0.90 for the TRF adaptive functioning items. Internal consistencies of problem scales as measured by Cronbach's alphas ranged from 0.72 to 0.97 for the CBCL, 0.67 to 0.95 for the YSR, and 0.72 to 0.97 for the TRF. Highest alphas were generally found for the three broad-band scales plus the longer narrow-band scales, such as Aggressive Behavior on all three forms and Attention Problems and DSM-Attention Deficit Hyperactivity Problems on the TRF.

The CBCL/6-18, YSR/11-18 and TRF/6-18 have strong criterion-related validity ⁵. Both competence/adaptive scales and problem scales significantly differentiated between demographically-matched referred and non-referred children. For example, referral status explained the high percentage of variance of the scores in Total Competence for the CBCL and YSR and in Adaptive Functioning for the TRF, after effects of age, SES, and ethnicity had been partialed out (36%, 28%, and 29% for the CBCL YSR, and TRF, respectively). Similarly, referral status explained the high percentages of the variance of the scores in Total Problems, after partialing out age, SES and ethnicity (36%, 15%, and 26% for the CBCL, YSR, and TRF, respectively). Referred children showed significantly higher scores on all problem items and scales for all three instruments. Odds ratios also showed that the ASEBA forms significantly differentiated between referred and non-referred children when problem scores were dichotomized into normal or borderline/clinical ranges.

Development of the ASEBA school-age instruments in Brazil: an overview

ASEBA instruments cannot be translated to other languages without the approval of the author of the original questionnaires. In Brazil, as in other countries with only one official language, it is important to have only one approved translation of each instrument in order to standardize assessment. The use of exactly the same questionnaire in different studies conducted in the same country avoids bias when interpreting differences in study results. In addition, translated questionnaires must use words that can be equally understood in different regions of the same country regardless of the level of education of the population. In countries with more than one official language, one translation of each instrument for each language is required to access different populations in their native language.

In Brazil, researchers and clinicians interested in applying the ASEBA school-age instruments should use the most recent (2010) and official Brazilian versions of CBCL/6-18, answered by parents, YSR/11-18, answered by adolescents, and TRF/6-18, answered by teachers (license number at the University of Vermont: 207-12-04-06, 2010). Each one of these three questionnaires was created by merging two pre-existing Brazilian versions of the original ASEBA school-age instruments from 2001 developed at the Department of Psychiatry of the Federal University of São Paulo (UNIFESP, acronym in Portuguese) in 2002, and at the Department of Clinical Psychology of the University of São Paulo (USP, acronym in Portuguese) in 2005. These three instruments in Portuguese (official Brazilian versions) correspond to the latest English versions of the CBCL/6-18, YSR/11-18 and TRF/6-18 (Copyright 2001 T. Achenbach ASEBA, University of Vermont 1 South Prospect St., Burlington, VT 05401-3456) ⁵, and are available upon request from USP Psychology Institute [[e-mail]].

Brazilian versions of the ASEBA school-age instruments from 1991

The first Brazilian version of CBCL/4-18 was developed at UNIFESP from the original 1991 English version ² with the author's permission. This version was translated into Portuguese and blindly back-translated to English by a professional bilingual translator native from USA. Simple terms and idiomatic expressions were used to facilitate the correct understanding of items among low-educated mothers, without distorting the content of original items. Translation and cultural adaptation of this Brazilian version benefited from corrections and suggestions from Achenbach.

Validity studies demonstrated the high sensitivity of this Brazilian version of CBCL/4-18 compared to "gold standard" psychiatric diagnoses based on the International Statistical Classification of Diseases and Related Health Problems, 10^th revision (ICD-10) ¹³, and the DSM-IV ¹²criteria. Initial findings from a validity study ¹⁴ showed high sensitivity of this Brazilian version of CBCL/4-18 when compared with ICD-10 psychiatric diagnoses made by an experienced child psychiatrist blind to CBCL/4-18 results. In a random sample of low-income pediatric outpatients (n = 49, 4-12 years), the CBCL/4-18 was administered to mothers by a trained lay interviewer, due to the mothers' low educational level, and 80.4% of children with one or more ICD-10 psychiatric diagnoses were in the borderline or clinical range (T-score ≥60) for the total behavior problem scale. Considering all children with ICD-10 psychiatric diagnoses, this Brazilian version of CBCL/4-18 correctly identified 100% of severe cases, 95% of moderate cases and 75% of mild cases. High sensitivity of CBCL/4-18 was also shown in a consecutive sample of children and adolescents (n = 78) scheduled for a first appointment at the mental health outpatient clinic of the Federal University of Rio de Janeiro (UFRJ, acronym in Portuguese) ¹⁵. This university outpatient clinic is a public service that provides assistance free of charge, and typically assists children from low-income families. Because sources of referral include health professionals, schools, social services, and parents themselves, the group of children scheduled for a first appointment was heterogeneous in terms of psychopathology, including children without disorders and clinical cases of different severity levels. CBCL/4-18 data was compared to results from a semi-structured psychiatric interview based on DSM-IV criteria [The Schedule for Affective Disorders and Schizophrenia for School-Age Children/Present and Lifetime Version (K-SADS-PL)]. The authors noted that 89.7% of children with one or more psychiatric disorders obtained a T-score in the borderline or clinical range (≥ 60) and 82.8% obtained a T-score in the clinical range (> 63) on the CBCL/4-18 total problem scale. In the above mentioned studies, T-scores were calculated with reference to American normative data, given that norms are not available in Brazil.

Brazilian versions of the ASEBA school-age instruments from 1991 were also developed at USP based on the versions from Portugal, as suggested by Achenbach, with some minor changes in wording to overcome cultural gaps in language, and have been used in research since 1991 mainly to evaluate school-age children's behavior problems pre and post psychological interventions. Although the use of these questionnaires appeared in presentations at scientific meetings ^16,17, the first paper mentioning their use was published only five years later ¹⁸. Since then, many studies have been conducted and published in either Brazilian ^19,20,21,22 or international ^23,24,25,26 journals. The majority of these studies have focused on the evaluation of patients in clinical practice and conducted in university psychological centers; few studies examined the validity of instruments applied. The same is true for the content of most books ^27,28 and individual chapters of books ^29,30 produced by the USP research team.

Brazilian versions of the ASEBA school-age instruments from 2001

Brazilian versions of the original 2001 ASEBA school-age instruments (CBCL/6-18, YSR/11-18 and TRF/6-18) were developed at UNIFESP and USP. The UNIFESP versions have mainly been used in epidemiological studies involving child and adolescent mental health, including cross-sectional studies ^{31,32,33,34,35} and longitudinal studies ^36,37. For instance, the CBCL/6-18 was applied in studies examining the association between child and adolescent mental health problems and severe physical punishment ³³, in research that identified gender differences in aggressiveness among children and adolescents at risk for schizophrenia ³², and also in studies that examined behavior problems in children with rare genetic disorders ^38,39. Both the CBCL/6-18 and the YSR/11-18 were used in a study that estimated the prevalence of mental health problems in a population sample of children and adolescents and verified the correspondent local service capacity to assist those in need of mental health care ³¹, and also in a study that identified environmental factors associated with adolescent antisocial behavior ³⁴. It is important to mention that the CBCL/6-18 is one of the most commonly used mental health screening tools in Brazilian epidemiological studies involving children and/or adolescents ³⁵.

The USP versions have been used in studies to evaluate the results of treatment, as well as to assess behavioral problems shown by children referred to psychological services and/or observed in the general population ^40,41,42. A number of validation studies of the 2001 ASEBA forms in Brazil have been conducted using the USP versions, especially the YSR. Preliminary data highlight the following attributes of this form: discriminative capacity due to the fact that the questionnaire allows researchers to distinguish between youths referred to mental health services and youths from the general population (Rocha MM, personal communication); and convergent validity – supported by correlations between YSR and a Brazilian inventory developed by Del Prette & Del Prette ⁴³ to evaluate social skills in adolescents (Inventário de Habilidades Sociais para Adolescentes). The factor model of eight syndrome-scales of both CBCL and YSR has been tested, with results supporting the applicability of this model for Brazilian children and adolescents ^44,45.

A comparative study of the UNIFESP and USP versions of the YSR showed a high degree of agreement between both translations, with no statistical differences regarding the answers from both forms, indicating that small differences in the wording of items had very little effect on the understanding of item content ⁴⁶.

Development of the most recent official Brazilian versions of the ASEBA school-age instruments

The current official Brazilian versions of ASEBA school-age instruments include: CBCL/6-18 (Inventário de Comportamentos para Crianças e Adolescentes entre 6 e 18 Anos), YSR/11-18 (Inventário de Autoavaliação para Adolescentes de 11 a 18 Anos) and TRF/6-18 (Inventário de Comportamentos para Crianças e Adolescentes entre 6 e 18 Anos – Formulário para Professores).

After one year of frequent meetings and team work, it was possible to complete the following tasks: (1) to compare each item of the CBCL/6-18, YSR/11-18 and TFR/6-18 from UNIFESP and USP versions (in Portuguese) and find the best translation for the unified versions; (2) to guarantee that items present in more than one instrument receive the same translations as occuring in the original versions in English; (3) to discuss the different options of Portuguese words with similar meanings in English in order to choose simple terms that are understandable to low-educated adults and younger adolescents without distorting the content of original items; (4) when translating the instruments, information from focus groups conducted with low-educated mothers was taken into account to avoid terms in Portuguese that were not well understood by group participants even if linguistically equivalent to the items in English; (5) to have a blind back-translation of the final draft of the three questionnaires to English by a professional bilingual translator native to the USA; (6) to compare each item of the back-translation with the original version to make the appropriate corrections to Portuguese terms in order to guarantee total coherence of meanings in both languages; and (7) to have input from Achenbach and Rescorla regarding corrections and suggestions for the final content of the current official Brazilian versions of the CBCL/6-18, YSR/11-18 and TRF/6-18.

Focus groups with low-educated mothers of school-age children were conducted by a psychologist and a social worker to help us identify misunderstandings regarding the content of specific items translated into Portuguese. Participants were residents from the metropolitan area of the city of São Paulo in the Southeastern Region of Brazil, including mothers born in the State of Minas Gerais and migrants from the Northeast Region of the country. In all cases, the translation was linguistically correct, but sometimes it was necessary to replace certain words by synonyms (i.e., more simple and popular terms) or the item had to be rephrased to ensure correspondence of meaning with the item in English. Changes were made to few items during the translation and back-translation process before and after information gathering with group participants. For instance, two different translations to Portuguese of the original item "Drinks alcohol without parents' approval" were discussed with group participants: (1) "Toma bebida alcoólica sem a aprovação dos pais", and (2) "Toma bebida alcoólica sem a permissão dos pais". The word permission (permissão) was preferable to the word approval (aprovação), since "no parental permission" was clearly understood as parental opposition to the child's drinking behavior (behavior not authorized by the parents) while "no parental approval" could be understood as parental disagreement but not prohibition. A second example concerns the original item "Fears going to school". Focus groups pointed out that the fact that many children living in violent neighborhoods were afraid of being assaulted on their way to school does not mean that they were afraid of the school environment. Therefore, this item was translated to Portuguese as "Medo da escola" (fear of school). A final example refers to the item "Daydreams or gets lost in his/her thoughts" that had to be translated as "Vive no mundo da lua ou perde-se em seus pensamentos". The term in Portuguese "devaneio" corresponds to daydreaming but could not be used because it was unknown by most low-educated adults and adolescents. The popular expression "vive no mundo da lua" (lives on the moon) was a more appropriate translation since it was clearly understandable to everyone regardless of educational or socioeconomic conditions.

When compared to the original English versions of the ASEBA school-age instruments, the current official Brazilian (Portuguese) versions of CBCL/6-18, YSR/11-18 and TRF/6-18 respect the principles of semantic equivalence (items with similar meaning in both languages), criteria equivalence (items with the same interpretation in both languages) and conceptual equivalence (items corresponding to the same theoretical construct in both languages) ⁴⁷. All final decisions regarding translation of items to Portuguese were based on consensus among all professionals involved in the development of the current official Brazilian versions of the ASEBA school-age instruments (all are authors of the current article). Since Brazil is a large country of great diversity, words and expressions not equally understood throughout the country were avoided or synonyms were provided (e.g., item 101 from CBCL/6-18 – "mata aula, cabula, gazeteia" as three regional terms meaning truancy, skips school).

Utility of the ASEBA school-age instruments for research and clinical practice

ASEBA school-age instruments are useful tools for epidemiological studies (including cross-sectional and follow-up studies), the evaluation of interventions, and for clinical practice involving the mental health of children and adolescents.

When aiming to identify children and adolescents with emotional and/or behavioral problems, it is important to consider the need to obtain information from more than one informant. It is important to note that while youth self-reports may reveal suicide ideation and other internalizing symptoms frequently unknown by parents and teachers, they may not reveal certain behaviors such as stealing or drug use, among other externalizing behaviors usually observed by parents and/or teachers.

Epidemiological studies and evaluation of interventions

ASEBA instruments can be used in epidemiological studies involving community samples, school-based samples and clinical samples of school-aged children from all socioeconomic strata. Based on previous validity studies ^14,15,44,45, it is reasonable to say that the most recent official Brazilian versions of the ASEBA school-age instruments are valid for screening mental health problems in Brazilian children and adolescents, even those with low educated parents and living in disadvantaged areas.

Questionnaires can be self-administered having parents, youth or teachers as informants or, if necessary, these instruments can be applied (in interviews) by trained lay people. Their use in research is therefore much less expensive than clinical evaluations by psychiatrists or psychologists. Therefore, when planning to conduct large epidemiological studies aimed to evaluate the mental health of children and adolescents, the use of a valid screening measure, at least in the first phase of the study, is recommended. When the identification of psychiatric disorders is required, all potential clinical cases (positive in the screening phase) and a random sample of non-clinical subjects (negative in the screening phase) are usually referred for further clinical evaluation (diagnostic phase).

ASEBA forms can also be used in follow-up studies, since they can show the persistence of certain behaviors, the appearance of new ones, and the disappearance of previous ones from one point in time to another. ASEBA forms have therefore been used in follow-up studies to register the influence of various factors on the course of psychopathology ³⁶, and to verify responses to different interventions such as clinical treatment involving medication ⁴⁸, educational programs ⁴⁹, psychotherapy ¹⁹, and training in social skills ⁵⁰.

Since the CBCL/6-18, YSR/11-18 and TRF/6-18 register degrees of deviance from normality of emotions and behaviors over time according to the opinion of parents, youth and teachers, respectively, they are useful instruments for testing whether the use of specific services contributes to a decrease in psychopathology symptoms or a reduction in behavioral deviance among children and adolescents in different environments (e.g., home, school).

Clinical practice

Although clinical interviews remain the main tool used by most clinicians, the use of standardized forms in clinical settings is increasing in different countries. Publications worldwide report the use of ASEBA forms not only in mental health and pediatric clinics, but also in schools. These forms can be completed at intake as a standardized measure of problems present before treatment. This measure can be used as a baseline for future comparisons to identify behavioral changes over time. This first assessment provides an overview of the child's behavior profile, indicates the need for intervention and guides decisions regarding the most appropriate treatment for specific cases. ASEBA forms can be used as an interview guide, a screening tool and an instrument to reassess children at regular intervals to assess treatment outcomes and the effectiveness of service delivery ⁵.

Correct use of the official Brazilian versions of the CBCL/6-18, YSR/11-18 and TRF/6-18

The time taken to complete school-age ASEBA forms varies depending on the educational level of the informant (understanding of questions) and the severity of child mental health problems (the greater the number of symptoms the longer it takes to complete the form). In general with low-educated informants, the behavior problem section of questionnaires takes 10 to 15 minutes to complete.

CBCL/6-18

When applying the CBCL/6-18 to a heterogeneous sample including low-educated parents (the majority of the Brazilian adult population), all questionnaires should be administered by trained interviewers (that could be lay people). In this case, Achenbach and Rescorla ⁵ suggest that a copy of the form should be given to the respondent so that he/she may follow the questions asked. Self-reporting is an appropriate method for informants who have completed at least eighth grade (mandatory basic education according to Brazilian law). The CBCL/6-18 should be answered by the biological parents or guardians, or caregivers who know the child well and have preferably been residing with her/him for at least the last six months.

YSR/11-18

The YSR/11-18 was designed to be filled out by youths with at least fifth grade level reading skills. When administering the YSR/11-18 in schools, it is generally necessary to complete the form in the classroom. Ethically, permission must be obtained from the school and a written informed consent must be obtained from parents (with the youth assent) and the presence of a professional is recommended to coordinate the process and answer pupils' questions. Interference from the group on individual responses and access to other class mates' answers must be avoided. The YSR/11-18 can be administered orally to those pupils with poor reading skills.

TRF/6-18

When administering the TRF/6-18 in Brazil, the most commonly used method is self-administration since teachers from all Brazilian regions are expected to have completed at least eighth grade. However, exceptions must be taken into account in more disadvantaged areas where less-educated women or men may work as teachers in the initial grades of elementary school. The respondents must know the child well and preferably have been in close contact with the pupil for at least the last two months. If the teacher has known the child for less than two months, she/he should be encouraged to complete the TRF, even if guessing is necessary to answer certain questions. Although not ideal, this can add some important information to the global evaluation of the child's behavior. In such cases, the teacher should be aware that a score of zero does not require complete certainty of the non-occurence of the problem; it may only indicate that the respondent does not know if it occurs.

Frequently asked questions regarding the CBCL/6-18, YSR/11-18 and TRF/6-18

According to the authors' experience in training lay persons and professionals, frequently asked questions involving the ASEBA school-age instruments reflect common doubts of interviewers regarding the administration of questionnaires and rating of items. Table 4 presents five examples of frequently asked questions with respective answers.

Scoring data using the Assessment Data Manager (ADM)

Achenbach and his team from the University of Vermont strongly recommend the use of the Assessment Data Manager (ADM) to score data obtained from ASEBA forms. The ADM is a rigorously tested scoring software for ASEBA instruments. The easiest way to use the ADM for scoring is to enter the responses from the questionnaire (raw scores) directly into the ADM. The software will then generate T-scores for all narrow-band and broad-band scales of the CBCL/6-18, YSR/11-18 and TRF/6-18 and allows the user to print the social competence and behavior profiles for all three instruments separately, and to print graphs to visualize cross-informant comparisons. In addition, ADM network licensing is available if simultaneous access for multiple users is needed (http://www.aseba.org, accessed on 25/Jul/2011).

Conclusion

The most recent official Brazilian versions of the ASEBA school-age forms and profiles are appropriate for use in studies involving mental health screening measures of Brazilian children and adolescents, even those with low-educated parents and living in disadvantaged areas. The official Brazilian versions of the CBCL/6-18, YSR/11-18 and TRF/6-18 are useful for clinical practice, training and research involving Brazilian children and adolescents from all socioeconomic strata; however, validation studies are still required.

The possibility of using a single set of forms in Brazil is a very important achievement for Brazilian psychiatry and psychology. The development of unified Brazilian versions of the ASEBA school-age instruments was fundamental to enhance the quality and value of all future work in the country because all researches will be able to use exactly the same instruments, allowing adequate comparison of multiple study results.

The next necessary methodological step following the development of these versions of the ASEBA school-age instruments concerns the assessment of its psychometric properties when applied to Brazilian children and adolescents residing in different regions of the country.

Contributors

All authors collaborated in the conception and writing of this review article. The final version was approved by all authors.

Acknowledgments

The authors are grateful to Daniel Graça Fatori de Sá (psychologist) for his contribution to the discussions on the translation of the ASEBA school-age form items to Portuguese; to Luis Augusto Rohde (child and adolescent psychiatrist and researcher) for revising the current official Brazilian versions of the CBCL, YSR and TRF in the final phase of development; and to Daniel Graça Fatori de Sá and Conceição Martins dos Reis (social worker) for conducting the focus groups.

Submitted on 05/Dec/2011

Final version resubmitted on 13/Aug/2012

Approved on 03/Oct/2012

• 1. Holmbeck GN, Thill AW, Bachanas P, Garber J, Miller KB, Abad M, et al. Evidence-based assessment in pediatric psychology: measures of psychosocial adjustment and psychopathology. J Pediatr Psychol 2008; 33:958-80.
• 2. Achenbach TM. Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington: Department of Psychiatry, University of Vermont; 1991.
• 3. Achenbach TM. Manual for the Youth Self-Report and 1991 profile. Burlington: Department of Psychiatry, University of Vermont; 1991.
• 4. Achenbach TM. Manual for the Teacher's Report Form and 1991 profile. Burlington: Department of Psychiatry, University of Vermont; 1991.
• 5. Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles. Burlington: Research Centre for Children, Youth and Families, University of Vermont; 2001.
• 6. Achenbach TM. The classification of children's psychiatric symptoms: a factor-analytic study. Psychol Monogr 1966; 80:1-37.
• 7. Achenbach TM. The child behavior profile: I. Boys aged 6-11. J Consult Clin Psychol 1978; 46:478-88.
• 8. Achenbach TM, Edelbrok C. The child behavior profile: II. Boys aged 12-16 and girls aged 6-11 and 12-16. J Consult Clin Psychol 1979; 47:223-33.
• 9. Achenbach TM. Integrative guide for the CBCL/4-18, YSR and TRF profiles. Burlington: Department of Psychiatry, University of Vermont; 1991.
• 10. Achenbach TM, McConaughy SH, Howell C. Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychol Bull 1987; 101:213-32.
• 11. Achenbach TM, Krukowski RA, Dumenci L, Ivanova MY. Assessment of adult psychopathology: meta-analyses and implications of cross-informant correlations. Psychol Bull 2005; 131:361-82.
• ¹²
  American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). 4^th Ed. Washington DC: American Psychiatric Press; 1994.
• ¹³
  World Health Organization. The ICD-10 classification of mental and Behavioral disorders: clinical descriptions and diagnostic guidelines. Geneva: World Health Organization; 1992.
• 14. Bordin IAS, Mari JJ, Caeiro MF. Validação da versão Brasileira do "Child Behavior Checklist" (CBCL) (Inventário de Comportamentos da Infância e Adolescência): dados preliminares. Revista ABP-APAL 1995; 17:55-66.
• 15. Brasil HH, Bordin IA. Convergent validity of K-SADS-PL by comparison with CBCL in a Portuguese speaking outpatient population. BMC Psychiatry 2010; 10:83.
• 16. Silvares EFM. Atendimento comunitário em clínica-escola de Psicologia. In: Anais do XXIII Congresso Interamericano de Psicologia. San José: Sociedade Interamericana de Psicologia; 1991. p. 408-18.
• 17. Silvares EFM. Social skill training for a cultural deprived population of Brazilian children. In: Annals of the I World Congress of Behavioral and Cognitive Therapies. Copenhagen: Behavioral and Cognitive Therapies Association Press; 1995. p. 212.
• 18. Massola GM, Silvares EFM. Percepção do comportamento infantil por professoras versus sexo e encaminhamento para atendimento psicológico. Psicol Teor Pesqui 1997; 13:303-9.
• 19. Melo MHS, Silvares EFM. Grupo cognitivo-comportamental com famílias de crianças com déficits em habilidades sociais e acadêmicas. Temas Psicol 2003; 11:122-33.
• 20. Arantes MC, Silvares EFM. Uma comparação entre crianças e adolescentes com enurese noturna primária: impacto e problemas de comportamento. Estud Psicol (Campinas) 2007; 24:155-60.
• 21. Tavares-da-Silva MDFD, Farias MA, Silvares EFM, Arantes MC. Adversidade familiar e problemas comportamentais entre adolescentes infratores e não-infratores. Psicol Estud 2008; 13:791-8.
• 22. Sapienza G, Farias MA, Silvares EFM. Competência social e práticas educativas parentais em adolescentes com alto e baixo rendimento acadêmico. Psicol Reflex Crit 2009; 22:208-13.
• 23. Marinho ML, Silvares EFM. Evaluación de la eficacia de un programa de entrenamiento de padres em grupo. Psicol Conduct 2000; 8:299-318.
• 24. Massola GM, Silvares EFM. A percepção do distúrbio de comportamento infantil por agentes sociais versus encaminhamento para atendimento psicoterapêutico. Interam J Psychol 2005; 39:139-50.
• 25. Melo MHS, Silvares EFM. Sociometric effects on rejection of children in grade 2 given a multifocal intervention involving teachers, parents, and children in one Brazilian school. Psychol Rep 2007; 101(3 Pt 2):1162-72.
• 26. Rocha MM, Costa NJ, Silvares EF. Changes on parents' and self-reports of behavioral problems in Brazilian adolescents after behavioral treatment with urine alarm for nocturnal enuresis. Int Braz J Urol 2008; 34:749-57.
• 27. Silvares EFM, Gongora MNA. Psicologia clínica comportamental: a inserção da entrevista com adultos e crianças. São Paulo: Edicon; 1998.
• 28. Silvares EFM. Estudos de caso em psicologia clínica comportamental. 4^a Ed. Campinas: Papirus; 2006.
• 29. Silvares EFM, Baraldi D. Treino de habilidades sociais em grupo com crianças agressivas associado à orientação dos pais: análise empírica de uma proposta de atendimento. In: Del Prette A, Del Prette ZAP, editores. Habilidades sociais, desenvolvimento e aprendizagem: questões conceituais, avaliação e intervenção. Campinas: Alínea; 2003. p. 235-58.
• 30. Silvares EFM, Santos EOL, Meyer SB, Gerencer TT. Um estudo em cinco clínicas-escola brasileiras com a lista de verificação comportamental para crianças (CBCL). In: Silvares EFM, organizador. Atendimento psicológico em clínicas-escola. Campinas: Alínea; 2006. p. 59-72.
• 31. Paula CS, Duarte CS, Bordin IA. Prevalence of mental health problems in children and adolescents from the outskirts of Sao Paulo City: treatment needs and service capacity evaluation. Rev Bras Psiquiatr 2007; 29:11-7.
• 32. Gutt EK, Petresco S, Krelling R, Busatto GF, Bordin IA, Lotufo-Neto F. Gender differences in aggressiveness in children and adolescents at risk for schizophrenia. Rev Bras Psiquiatr 2008; 30:110-7.
• 33. Bordin IA, Duarte CS, Peres CA, Nascimento R, Curto BM, Paula CS. Severe physical punishment: risk of mental health problems for poor urban children in Brazil. Bull World Health Organ 2009; 87:336-44.
• 34. Curto BM, Paula CS, Nascimento R, Murray J, Bordin IA. Environmental factors associated with adolescent antisocial behavior in a poor urban community in Brazil. Soc Psychiatry Psychiatr Epidemiol 2011; 46:1221-31.
• 35. Paula CS, Miranda CT, Bordin IA. Saúde mental na infância e adolescência: revisão dos estudos epidemiológicos brasileiros. In: Lauridsen-Ribeiro E, Tanaka OU, editores. Atenção em saúde mental para crianças e adolescentes no SUS. São Paulo: Editora Hucitec; 2010. p. 75-92.
• 36. Anselmi L, Barros FC, Teodoro ML, Piccinini CA, Menezes AM, Araujo CL, et al. Continuity of behavioral and emotional problems from pre-school years to pre-adolescence in a developing country. J Child Psychol Psychiatry 2008; 49:499-507.
• 37. Fatori D, Curto B, Bordin IAS, Paula CS. Exposição à violência como risco para o surgimento ou a continuidade de comportamento anti-social em adolescentes da região metropolitana de São Paulo. Psicol Teor Prat 2009; 11:179-88.
• 38. Mesquita MLG, Brunoni D, Neto JMP, Kim CA, Melo MHS, Teixeira MCTV. Fenótipo comportamental de crianças e adolescentes com síndrome de Prader-Willi. Rev Paul Pediatr 2010; 28:63-9.
• 39. Teixeira MCTV, Monteiro CRC, Velloso RL, Kim CA, Carreiro LRR. Fenótipo comportamental e cognitivo de crianças e adolescentes com Síndrome de Williams-Beuren. Pró-Fono 2010; 22:215-20.
• 40. Moraes EO, Enumo SRF. Estratégias de enfrentamento da hospitalização em crianças avaliadas por instrumento informatizado. Psico USF 2008; 13:221-31.
• 41. Borsa JC, Nunes MLT. Concordância parental sobre problemas de comportamento infantil através do CBCL. Paidéia 2008; 18:317-30.
• 42. Rocha MM, Silvares EFM. Competências de jovens brasileiros: fator de proteção para problemas comportamentais. In: Actas do VII Simpósio Nacional de Investigação em Psicologia. Braga: Universidade do Minho; 2010. p. 1162-71.
• 43. Del Prette A, Del Prette ZAP. Inventário de habilidades sociais para adolescentes (IHSA-Del-Prette): manual de aplicação, apuração e interpretação. São Paulo: Casa do Psicólogo; 2009.
• 44. Emerich DR, Rocha MM, Silvares EF; Equipe CBCL/6-18 2010. Testagem da estrutura fatorial do Child Behavior Checklist para crianças brasileiras. In: Anais do V Congresso Brasileiro de Avaliação Psicológica [CD-ROM]. Bento Gonçalves: Instituto Brasileiro de Avaliação Psicológica; 2011.
• 45. Rocha MM, Silvares EF. Testagem da estrutura factorial do Youth Self-Report no Brasil. In: Anais do V Congresso Brasileiro de Avaliação Psicológica [CD-ROM]. Bento Gonçalves: Instituto Brasileiro de Avaliação Psicológica; 2011.
• 46. Rocha MM, Araujo LGS, Silvares EFM. Um estudo comparativo entre duas traduções brasileiras do Inventários de Auto-Avaliação para jovens (YSR). Psicol Teor Prat 2008; 10:14-24.
• 47. Flaherty JA, Gaviria FM, Pathak D, Mitchell T, Wintrob R, Richman JA, et al. Developing instruments for cross-cultural psychiatric research. J Nerv Ment Dis 1988; 76:257-63.
• 48. Conklin HM, Reddick WE, Ashford J, Ogg S, Howard SC, Morris EB, et al. Long-term efficacy of methylphenidate in enhancing attention regulation, social skills, and academic abilities of childhood cancer survivors. J Clin Oncol 2010; 28:4465-72.
• 49. Barrera M, Schulte F. A group social skills intervention program for survivors of childhood brain tumors. J Pediatr Psychol 2009; 34:1108-18.
• 50. Bolsoni-Silva AT, Silveira FF, Ribeiro DC. Avaliação dos efeitos de uma intervenção com mães/cuidadoras: contribuições do treinamento em habilidades sociais. Contextos Clínicos 2008; 1:19-27.

Publication Dates

• Publication in this collection
  29 Jan 2013
• Date of issue
  Jan 2013

History

• Received
  05 Dec 2011
• Accepted
  03 Oct 2012
• Reviewed
  13 Aug 2012