Item analysis of the Teacher’s Gifted Rating Screening ScaleAbstract
Objective: The present study aimed to add new evidence of construct validity of the Teacher’s Rating Giftedness Screening Scale. The analysis of its items, in relation to the adjustment indices to the Graduated Response Model, as well as to interpret the sense of the scales formed by the items.
Method: The sample was composed of 666 students (64.7% males). The instrument consisted of 42 items that assess five areas (general intellectual abilities, leadership, creativity, specific academic abilities, and artistic talent), being answered by 224 teachers.
Results: The results indicated that all items had infit/outfit adjustment indexes within the expected value, between 0.5 and 1.5. The factor accuracy, estimated by the graduated response model, varied between 0.74 to 0.85.
Conclusion: The results allowed the elaboration of an interpretation proposal referenced in the item, as well as the identification of the most effective items in discriminating individuals with above average ability level.
Keywords: Behavior rating scale; Educational measurement; Special educational needs; Special education
Resumo
Objetivo: O presente estudo visou ampliar as evidências de validade de construto do instrumento Triagem de Indicadores de Altas Habilidades/Superdotação, por meio da análise dos seus itens, em relação aos índices de ajuste ao Modelo de Resposta Graduada, bem como interpretar o sentido das escalas formadas pelos itens.
Método: A amostra foi composta por 666 estudantes (64,7% meninos). O instrumento, composto por 42 itens que avaliam cinco dimensões (capacidade intelectual geral, liderança, criatividade, habilidades acadêmicas especificas e talento artístico), foi respondido por 224 professores.
Resultados: Os resultados indicaram que todos os itens apresentaram índices de ajuste infit/outfit dentro do valor esperado, entre 0,5 e 1,5. A precisão dos fatores, estimada pelo modelo de resposta graduada, variou entre 0,74 e 0,85.
Conclusão: Os resultados permitiram a elaboração de uma proposta de interpretação referenciada no item, bem como a identificação dos itens que se mostram mais eficazes em discriminar indivíduos com habilidade acima da média.
Palavras-chave: Escala de avaliação comportamental; Avaliação educacional; Necessidades educativas especiais; Educação Especial
Recent studies have shown the importance of understanding giftedness in a multidimensional way (Dai, 2018; Kaufman & Sternberg, 2018). The Brazilian Ministry of Education and Culture defines giftedness as a group of students who demonstrate a high level of potential, either combined or isolated, in the intellectual, academic, leadership, and psychomotor domains (Ministério da Educação e Cultura, 2010). Additionally, they demonstrate high creativity and high engagement with learning, and they carry out tasks that interest them.
Assessment plays an important role in the field of gifted education since the information obtained from these measures can be used to make decisions about the education of students, including whether they meet the diagnostic criteria and, therefore, are eligible for specialized educational services (McBee et al., 2016). In this scenario, standardized performance tests remain a fundamental aspect of identifying gifted students. However, teacher assessments have been increasingly incorporated into the selection criteria to serve as classifications to indicate which students should be tested as part of a more comprehensive identification system (Nakano, 2020). In their jobs, these professionals can observe students in different and varied circumstances (Tirri, 2017), comparing their behavior with that of their peers in the classroom to identify those who stand out (Milic & Simeunovic, 2022).
Considering that a complete and thorough evaluation is generally costly and time-consuming, requiring time and money, both scarce resources in schools, we opted to implement giftedness identification systems in two stages: screening and evaluation (McBee et al., 2016). An initial screening is performed based on the information provided by teachers, who may be the first to identify the students and refer them for evaluation (Alencar et al., 2018). However, this task’s development faced difficulties as teachers were not familiar with the characteristics of gifted students (Martins et al., 2018; Oliveira et al., 2018).
The most conservative estimate, involving only 3% of students, points to the possibility of identifying over 600,000 cases of giftedness among the more than 20 million Brazilians enrolled in basic education in 2019 (Rangni et al., 2021). Data from the 2019 National Institute for Educational Studies and Research “Anísio Teixeira”, School Census, however, show just over 51,000 students have been notified, which represents less than 0.0025% of the enrolled students (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, 2019). The discrepancy between the Census numbers and published literature estimates demonstrates that difficulties are encountered in identifying these individuals (Almeida et al., 2016), their registration in school censuses and, consequently, the provision of adequate care to that population.
With this deficit in mind and aiming to objectify the process of identifying and developing students with indications of giftedness, a screening scale entitled Teacher’s Gifted Rating Screening Scale was proposed. This project aims to mitigate the lack of specific instruments for this population’s assessment, as an auxiliary resource for teachers and other professionals, within a broader process that incorporates a variety of resources, such as direct assessment, observation, interviews, analysis of schoolwork, teacher assessment, and psychological tests, which can be used to suggest giftedness.
For an adequate identification, the literature has recommended a system based on two main phases in the stage preceding the referral of students to programs or interventions. The screening process consists of evaluating a group of potentially talented students or even the entire student population within a rapid and inexpensive process known as nomination (McBee et al., 2016). An important part of this step is signaling and/or screening. It is most commonly carried out by teachers, and it involves responses to a standard checklist of behaviors, usually those that can be applied quickly and collectively. After identification in this first phase, individuals are eligible for the second phase, for confirmatory evaluation, in which additional methods are employed, with a more rigorous and in-depth evaluation of each case, which involves more techniques and instruments and is more costly and time-consuming.
The second phase evaluates whether or not the high potential is present (and in which areas it manifests), and if it is confirmed, educational measures can be established to make use of it (Nakano & Campos, 2019). It is advantageous to use these two phases because they significantly reduce the identification costs.
According to Alencar et al. (2018), most students who are enrolled in gifted programs in Brazil are admitted through their teachers’ indications. A teacher should be regarded as a professional with greater proximity and affinity with students, an important source of consultation to students, which is not considered when exclusively utilizing psychological tests for evaluation (Fabio & Buzzai, 2019). In several countries, during the identification process, teachers have played an important role in naming students with giftedness indicators. Teachers with the appropriate training can serve as the primary resource for identifying these students and referring them for evaluation, as well as implementing appropriate care in cases when the diagnosis is confirmed (Almeida et al., 2017).
Teacher evaluation scales represent a basic component in the selection of participants in gifted education programs, making them one of the most widely used instruments worldwide (Mohamed & Omara, 2020). Typically, these tools are developed based on the hypothesis that certain students exhibit behavioral characteristics that are significantly different from their peers concerning cognitive, physical, socio-emotional, and personal characteristics, which can be recognized by their teachers (Tirri, 2017). It usually involves assessing the presence of behavior patterns that are regarded as typical for students with giftedness, providing important information about the development level of the student (Hertzog et al., 2018).
Using scales for teachers has the additional benefit of allowing the assessment of characteristics that constitute giftedness but are difficult to assess via other measures, such as leadership and motivation (Benson & Kranzler, 2018). Consequently, this type of instrument has been the most often used by psychologists in the United States for screening for giftedness (Renzulli et al., 2009). In a longitudinal approach, it can be used to assess a student’s behavior based on continuous, direct, and systematic observations of the behavior the student presents in situations such as action, production, and performance (Guenther, 2012).
Usually, screening instruments involve the systematic evaluation of behaviors that are regarded as ‘typical’ of gifted students (Hertzog et al., 2018). The result is a reduction in costs and the time normally required for a complete evaluation. However, the scales for teachers are not intended to replace other methods of identification (Pfeiffer & Blei, 2008). Their purpose is to supplement the information provided by other identification techniques and instruments. In addition to teachers, there should be other sources of information that contribute to this process. Examples include portfolio evaluation, interviews, observations, references from peers and parents, and assessment of constructs such as intelligence, creativity, leadership, and motivation.
Several scales for evaluating this objective are available worldwide. We can cite such scales as The Gifted Evaluation Scale (McCarney & Anderson, 1998), The Gifted Rating Scales (Pfeiffer & Jarosewich, 2003), The Gifted and Talented Evaluation Scales (Gilliam et al., 1996), the Scales for Identifying Gifted Students (Ryser & McConnell, 2004), and the Scales for Rating the Behavioral Characteristics of Superior Students (Renzulli et al., 2002). Additional scales include the Cognitive/Learning Abilities Scale (Almeida et al., 2000), The Universal Multiple Abilities Scales Examiner’s Manual (MacCallum & Bracken, 2012), The Universal Talented & Gifted Screener (Bracken & McCallum, 2018), the HOPE Teacher Rating Scale (Gentry et al., 2015), and The Hispanic Bilingual Gifted Screening Instrument (Irby & Lara-Alecio, 1996).
In Brazil, the lack of properly validated screening instruments for this group of students is also notable (Miranda et al., 2013). Currently, there are three scales operated in Brazil: the Gift and Talent Identification Scale (Freitas et al., 2017) is undergoing the publishing phase of its first investigation studies of psychometric qualities (Freitas & Schelini, 2018). The Teacher Talent Identification Scale (Suárez & Wechsler, 2019) is similarly positioned. Another scale, the Characteristics Identification Scale associated with Giftedness, is also being studied (Zaia & Nakano, 2020; Zaia et al., 2018). It is important to point out, however, that it is a self-report scale. It is noteworthy, therefore, that the country has only one teacher scale that is already published and aimed at this purpose and specific audience: the Teacher’s Gifted Rating Screening Scale (TIAH/S) (Nakano, 2021).
As the name implies, the tool intends to aid in signaling the need for a more comprehensive psychological assessment based on the perceptions of teachers and other professionals who work in schools. This information from this first indication can help in the decision about the presence of criteria indicative of giftedness (McClain & Pfeiffer, 2012) and as the first screening for cases that should be referred for a more in-depth evaluation to determine whether such indicators may or may not be confirmed.
Due to the multidimensional nature of this phenomenon, the tool was designed to incorporate AH/S indicators across five areas (general intellectual capacity, specific academic capacities, creativity, leadership, and artistic talent). The TIAH/S was based on the Gifted Rating Scale (Pfeiffer & Jarosewich, 2003), but other instruments, such as The Gifted Evaluation Scale (McCarney & Anderson, 1998), as well as the Gifted and Talented Evaluation Scales (Gilliam et al., 1996), measure the same things.
Several studies have already been conducted to evaluate the psychometric properties of the screening tool: evidence of content validity (Nakano, Campos et al., 2016) based on internal clarity and precision (Nakano & Oliveira, 2019) and based on relationships with external variables of the concurrent type (Nakano, Gozzolli et al., 2016; Nakano et al., 2020). Although the validity of the instrument has been demonstrated in the cited studies, it is worth remembering that the validity of such a source depends on a cumulative process of evidence that supports the interpretations intended with the instrument and should be treated as a continuous search (Ambiel & Carvalho, 2017).
Therefore, the study presented in this article is noteworthy since no previous study had attempted to analyze the properties of the items it used. As the results of the test can be used to make sense of the scale formed by the items, this proposal has been considered a method of examining the construct validity of the test. Therefore, it can be used to support a proposal for interpreting the results (van der Linden & Hambleton, 1996) by identifying items that require a higher level of ability, in this case, indicating signs of giftedness, and can, therefore, identify subjects who should be referred for a more comprehensive evaluation.
Method
Participants
The sample consisted of 666 students, of whom 431 were males (64.7%) aged between 8 and 18 (M = 12.6; SD = 2.21). The students were divided into two groups. Students with giftedness (n = 355) participated in attendance programs (Nucleus of Activities for Gifted Students or Resource Rooms) in the Southern region which are provided free of charge in 20 different cities. Participants in this group ranged in age from 8 to 18 years (M = 13.08; SD = 2.18), with 62.3% of the group being male.
Among the participants identified as gifted, ten different areas were included in the sample, but academic giftedness predominated (45.5%), followed by productive-creative giftedness (18.0%). The sample included students identified as belonging to various intellectual areas (mainly technology, language, codes and technologies, interpersonal relationships, sports, corporal-kinesthetic, astronomy, leadership, etc.), and this information was obtained by teachers who responded.
As for the second group of students (n = 311), there were 16 regular schools in the Southeast (Campinas and Vinhedo, both in the state of São Paulo), Northeast (São Luis, Maranhão state), South (Curitiba, Paraná), and Midwest (Brasilia, Federal District), of which two are private, 13 are public, and one is a military school Participants were aged 9 to 18 years (M = 11.89, SD = 2.05), and 52.6% of them were male.
A total of 224 teachers assessed both groups. Each performed a series of assessments, ranging from one to twelve students. Some teachers are responsible for more than one class or several classes simultaneously, which results in a variation in the number of student assessments.
Instrument
Teacher’s Gifted Rating Screening Scale (Nakano, 2021) - It is composed of 42 statements that address behaviors related to giftedness indicators, divided into five areas (general intellectual capacity, leadership, and creativity, each with eight items, and specific academic ability and artistic abilities, with nine items each). The areas are presented independently, beginning with their definition and then listing the items composing each area in sequential order.
The teacher must evaluate the intensity of each behavior present in the student, choosing one of three alternatives: development below the expected level, development at the same level as peers of the same age, or development above average, marking the column corresponding to their choice with an X. A professional has the option of marking ‘unable to judge’ if she/he does not feel capable of judging the behavior of a particular item. It prevents a teacher from marking an answer that is not right to influence the final grade. The adoption of this Likert format followed the model adopted in the instrument taken as a basis (Gifted Rating Scale).
Defining each of these areas is within the model developed by Pfeiffer et al. (2006), which can be used by the respondent to determine whether each behavior exists, as well as its intensity, when comparing the student with peers of the same age. Table 1 summarizes the composition of each area.
Areas, definitions, and quantity of items covered by Teacher’s Gifted Rating Screening Scale
Raw scores are estimated in each of the five areas before they are transformed into a standardized score and percentile. Studies have been carried out to define the cut-off points of the instrument to determine whether a student has characteristics that are closer to the criteria group (students identified with giftedness), having, as parameters, the sensitivity and specificity indexes (Nakano & Primi, 2020). An individual must earn 22 points in the area of general intellectual capacity to be considered for the awards program, while 17 points are required in the area of leadership. In any of the areas, scores greater than those established as a cut-off point for the tool indicate the presence of giftedness indicators and the need for further investigation.
In addition, it is important to emphasize that the items were designed to encompass behavior that is often present in gifted individuals so that their content can be applied systematically across a range of different age groups. As the scale is based on five behavior categories, it is considered that the items allow the identification of different giftedness indicator profiles without being restricted to any specific category.
Procedures
Initially, the research was approved by the Research Ethics Committee of PUC-Campinas under the opinion number CAAE 21487513.7.0000.5481. Regarding the criteria, the data collection was conducted at an institution located in the state of Santa Catarina. Teachers and professionals employed in the Specialized Educational Service centers from a total of 20 cities were given access to the scale link, which was available online on the Google Docs platform. They were asked to rate the students that they mentored by filling out an evaluation form for each student, as well as indicating in which area they identified giftedness.
Comparatively, the control group’s data collection was carried out with an instrument in pencil and paper in classrooms located in 16 different schools, including two private, 13 public, and a military school located in São Paulo and Parana. Teachers were asked to answer as many scales as possible.
Importantly, the teachers who evaluated the criterion group were involved in the field of giftedness and were familiar with the indicative signs, in addition to regularly participating in training and qualification sessions. Likewise, the teachers who evaluated the control group were not experts in the subject matter and therefore were instructed to select students who, according to their estimation, did not stand out in any way from the other students in the class. When we consider that the main objective of the scale is to be used by teachers from multiple educational systems, this diversity is extremely important.
Data Analysis
Complying with the unidimensionality requirement for analyses based on Item Response Theory models, the statistical modeling was conducted independently for each of the dimensions. Considering the polyatomic nature of the items, the Rasch-Andrich Rating Scale Model was utilized, estimated via Joint Maximum Likelihood in WINSTEPS 3.7 (Linacre, 2015). The difficulty levels of items, adjustment indices (infit and outfit), as well as the item-theta correlation were estimated to assess the items. The precision of the dimensions was assessed, and maps of items and constructs were built to calculate the average amount of ability (theta) required to score at each item. In order to evaluate the relationship between the difficulty of the items and theta levels presented by the participants, the Item-Person Map technique was used (Embretson & Reise, 2000).
To analyze the items in terms of adjustment and the meaning of the scales formed by them, we followed the recommendations of the literature, which suggests values below 0.5 and above 1.5 (especially for infit) as indicative of poor model fit and item-theta correlations above 0.30 as acceptable (Wright & Masters, 1982). In addition, the Differential Functioning of the items (DIF) was estimated based on the participants’ gender.
According to the literature’s recommendation, the presence of DIF was evaluated by a contrast greater than 0.42 or a probability lower than 0.05 (Aguerri et al., 2007). Higher values, either positive or negative, would indicate the presence of an item with a DIF that is favorable to one of the groups. The analysis considered the frequent citations of the underrepresentation of girls in the indications for identifying giftedness, with a predominance of boys in giftedness programs, in the literature (Kruczeveski & Silva, 2018).
Results
The first analysis intended to evaluate the difficulty of the items and the adequacy of their adjustment to the model, thus evaluating the impact of each item individually (Bond & Fox, 2001). These analyses were conducted with adjustment indices, which are a method for comparing what was predicted by the model with what was empirically observed (Smith, 2004). In this case, residuals refer to these differences, and there were two indexes employed: infit (an index to check for discrepancies in items whose difficulties are close to the subject’s ability) and outfit (an index to check for discrepancies in extreme items). An analysis of item-theta correlations was conducted to evaluate each item’s capacity to recover the theta of the respondent.
Considering the requirement of unidimensionality for this type of analysis, the indices evaluated in each of the areas of the scale were determined separately. The results of the first three factors are presented in Table 2. The difficulty index for the Intellectual Ability scale ranged from -0.54 to 0.37. The mean of the items was anchored at zero to determine the metric as well as the possibility of an association between the parameters of the items and the individuals. The infit statistics ranged between -0.57 and 1.27. The outfits ranged from 0.54 to 1.35. All indices were satisfactory. Generally, all items showed a high correlation with theta (greater than 0.82), and the precision was estimated at 0.85, proving to be adequate.
Indices of item fit and precision for the factors of general intellectual ability, academic ability, and creativity
On the second factor, Academic Ability, the item difficulty varied between 0.61 and 0.78, while the infits varied between 0.72 and 1.3. The outfits ranged from 0.69 to 1.40. The indices were all appropriate. All items exhibited high correlations with theta (greater than 0.66), and the precision was estimated at 0.75, considered adequate.
The difficulty indexes for the items comprising the Creativity factor ranged from -0.51 to 0.49, while the outfit index ranged from 0.77 to 1.37. The outfit index ranged between 0.75 and 1.49. In general, all items demonstrated high correlations with theta (greater than 0.69) and a precision of 0.74, which proved to be adequate.
Next, two other factors of the TIAH/S were examined: leadership and artistic talent, the results of which are presented in Table 2. The items that comprise the Leadership factor had difficulty between -0.56 and 0.40, infit between 0.81 and 1.29, and outfit between 0.83 and 1.41. The precision for this factor was 0.93, and the item-theta correlations were higher than 0.75.
Finally, the difficulty indexes for Artistic Talent ranged from -0.67 to 1.50, infits ranged from 0.82 to 1.15, and outfits ranged from 0.76 to 1.28, which is adequate. The precision was 0.81, and the item-theta correlation coefficients were greater than 0.66 (Table 3).
According to the results, the closest residues (infit) are all in good condition; they have found responses at levels higher than expected, notably in two items, one belonging to the Creativity factor (outfit = 1.49) and one belonging to the Leadership factor (outfit = 1.41). Given that part of the sample represented gifted individuals, with average performances usually higher than expected, this result can be interpreted in a more meaningful manner.
Following, the person-item maps were prepared. As a result of their application, the results are interpreted based upon the different levels of the scale, i.e., the thetas (Figure 1).
Regarding intellectual capacity, items 2 and 7 have the greatest potential to identify individuals with higher levels of ability (“Needs little information to solve difficult problems”; “Showing a variety of interests”). As a result, teachers indicate higher alternatives only for students who have a higher general intellectual level than the average student.
As regards specific academic ability, items 3, 9, 8, and 4 have the greatest potential for the achievement of such ends, especially the fourth (“Presents superior performance in mathematics”). In terms of creativity, items 7 and 8 require a higher level of ability to be endorsed (“Usually ask many questions in the classroom”; “Prefers non-routine tasks”). For the teacher to endorse such items, the ability level of the assessed student, in this case, giftedness indicators, will probably stand out as being higher than the average of all other students.
For the Leadership factor, items 6 (“Demonstrates an ability to lead others”) and 7 (“Has a high level of empathy”) are more discriminating; for the Artistic Talent factor, items 6 (“Exhibits ease in non-verbal expressions, such as gestures and gestures”) and 8 (“Follows the lead of a musical instrument”), which have a higher level of difficulty, are more suitable. Such results can be seen in Figure 2.
Other items in the five analyzed factors have a difficulty level below or equal to the average, so students with average abilities receive points in those areas represented by the items and are not required to be highlighted. As a result, such items may not be very effective in identifying individuals with high levels of ability in the areas assessed. This is because the results indicate that even individuals with ability levels below or equal to average are positively assessed by teachers in the other items.
Having analyzed the items, we could determine the items that most discern such individuals, i.e., items whose difficulty level is above one standard deviation from the mean. This case involves items 2 and 7 of the intellectual capacity factor, item 4 for the academic ability factor, items 7 and 8 for the creativity factor, items 4, 6, and 7 for the leadership factor, and items 6 and 8 for the artistic ability factor. On the other hand, it allowed the identification of items that are not effective - that is, the difficulty of being endorsed by the teachers is so low that any student, regardless of their level in the investigated ability, is positively evaluated. This is the case with items 1 and 2 of the Specific Academic Ability area (“Excellent in one or more school subjects” and “Shows good performance on tests and school assignments” respectively), where students with ability levels of more than one standard deviation below the mean score highly. In other words, a teacher may evaluate any student as a student who shows good academic performance without such performance being sufficient to suggest giftedness.
Lastly, we compared the performance of the items according to gender. Among the 42 items on the scale (0.05%), only two items showed DIF. In the creativity dimension, the item “Usually asks a lot of questions in the classroom” showed a DIF in favor of females, showing a lower difficulty value of (+ 0.04) than for male participants (+ 0.077), with a contrast value of -0.72 and a Maentel-Haenszel probability of 0.0001. Additionally, item 8 of the artistic talent dimension (“Dominates some musical instrument”) demonstrated differential functioning, with more difficulty for females (+ 1.81) when compared to males (+ 1.31), with a DIF contrast value of -0.50 and Maentel-Haenszel probability of 0.0014.
Discussion
The Item Response Theory has several distinct advantages over the investigation of the psychometric properties of teacher rating scales, but this type of analysis is rarely employed in gifted education (Ridgley et al., 2019). It was possible to evaluate the parameters of the five dimensions with the help of this analysis model to contemplate the heterogeneity of the group. A more comprehensive understanding of the phenomenon includes many characteristics in addition to intelligence (Kaufman & Sternberg, 2018), which has long been mistakenly regarded as the sole synonym for giftedness (Dai, 2018).
Our analysis allowed us to identify the behaviors that most differentiate gifted students from those who do not have these signs in each of the five dimensions. It was noted, for instance, that the individual with the highest level of intellectual capacity (whose performance is equal to or greater than one standard deviation when compared to peers of the same age) is usually regarded by teachers as the one who has little information required to solve problems and has a wide range of interests. Using the same method, an analysis was carried out in each of the dimensions to identify the behaviors that best distinguish individuals with high ability in each of the domains in which the teacher evaluates the student.
In addition to understanding the adjustment indices of the items, the analysis performed allowed us to determine the level of difficulty of each item, thus identifying those that are most likely to indicate behaviors that are common among gifted individuals. Based on this, we can later compose a reduced version of the instrument, consisting only of items that demonstrated the greatest potential for differentiating between the two groups (identified as gifted and regular students, unidentified). It would include the more commonly observed characteristics in gifted individuals as items.
It listed the following characteristics in this group of students that were identified by teachers as being important: easily solving complex problems, having a wide range of interests, high performance in mathematics, curiosity, preference for non-routine activities, leadership, empathy, and comfort with artistic expression. Such characteristics are also mentioned in the literature as typical of gifted students (Alves, 2020; Turkman, 2020) and in the main theories of giftedness, for example, the Three Rings Model (Renzulli & Reis, 2021), the Triarchic Theory of Intelligence (Sternberg, 2000), and the Differentiated Model of Giftedness and Talent (Gagné, 2010).
However, it is important to highlight that part of these characteristics, such as questioning and difficulties with routine, are commonly seen as undesirable by teachers, involving indiscipline, high critical level, or disruption of the class after quickly finishing activities (Mendonça et al., 2020). Thus, teachers must be made aware of the peculiarities and potential of these students so that they are not harmed in the academic path.
The analysis of the items pointed to a tool whose items present adequate fit indices. The items proved adequate to estimate the variety of abilities present in the study sample, considering inferior to superior performances in each of the evaluated areas. None of the items showed maladjustment, so there was no need to exclude any of them.
As a major contribution to the area, the instrument proposal presented here provides the opportunity to identify giftedness indicators by using a scale, which can be used as a first filter in the screening process of students with higher potential. Given that Brazilian teachers indicate students for evaluation (Alencar et al., 2018) and that subjective beliefs or misguided expectations may affect the process (Matheis et al., 2017), the use of standardized observation tools can reduce the possible influence of these factors. Two major concerns have been identified by Cao et al. (2017): a lack of training of teachers for the proper identification of the various manifestations of the phenomenon; and the assessment of giftedness based on personal understandings that may not reflect scientific definitions. These issues can be addressed with the use of a standardized instrument to observe these behaviors.
In the context of education, one of the key features of the instrument is that the scale is answered by the teacher, who has usually been trained in the area of education and development, which enables all students in the classroom to be evaluated and compared in a variety of situations with their peers of the same age (Rothenbusch et al., 2018). The scale, quick, and easy-to-response tool provides a means to identify the levels of performance that can be high and with a significant reduction in costs compared to the traditional broad evaluation process. This information may complement information obtained through other sources, both at the time of screening and during the subsequent follow-up of students already identified, regarding their progress (Mohamed et al., 2017).
The obtained results support the validity of the instrument, which is more pronounced compared to those previously investigated (Nakano, Campos et al., 2018; Nakano & Oliveira, 2019; Nakano et al., 2016; Nakano, Gozzoli et al., 2018; Nakano et al., 2020). Additionally, the instrument is suited to the purpose for which it was built and can assess signs of AH/S in each of the five dimensions that constitute TIAH/S.
Studies that characterize the psychometric characteristics of the teacher assessment scales are essential because, according to Benson and Kranzler (2018), there are comparatively few studies that investigate their psychometric properties using nonstandardized instruments. It is common for these scales not to show any evidence of validity or accuracy (Pfeiffer, 2015).
In general, the data suggested that the items of the TIAH/S are, for the most part, neutral in relation to differences based on a participant’s gender. In light of the low reporting of female cases, this information is critical to illustrate that the use of the instrument can assist teachers to ensure a more equitable assessment process (Zaia et al., 2019). That is a contrast to the situation that has been reported in the literature (Kruczeveski & Silva, 2018).
It is hoped that the TIAH/S will benefit professionals in relation to the gaps which exist within the subject in Brazil (Alencar et al., 2018; Wechsler et al., 2018), specifically regarding the availability of instruments to identify this phenomenon (McCluskey, 2018).
Conclusion
Talents’ losses due to difficulties in identifying students with giftedness result in a society that is missing out on the contributions that these individuals could make. In addition, this limits the opportunities for these students, diminishing their potential for personal and professional development. Nevertheless, the benefits of a screening phase will only be realized if the evaluation system employed is of high quality and efficiency, suitable for the targeted group, and capable of identifying the specific indicators. Otherwise, the reduction in the cost and time allocated for the evaluation will lead to false negatives, resulting in the loss of students who may otherwise have been identified by other procedures.
As a result of the procedures used in this study, certain limitations were identified. A major factor relates to teachers’ evaluations possibly having been influenced by their conception of giftedness. The qualification of teachers for this identification has been the subject of many debates. It must be noted, however, that the evaluation of the criterion group, which made up more than two thirds of the sample, was conducted by teachers in specialized educational services for identified students so that they are familiar with the topic in great depth. In contrast, the remaining unidentified students (control group) were evaluated by professors who may have little or no knowledge of the subject matter, even though they were asked to assess students based on a series of specific behaviors.
Considering that teachers who have received specific training in gifted education are more likely to identify the signs that indicate the phenomenon than non-trained classroom teachers, the difference in training between the two groups that evaluated the students cannot be ignored. There is a possibility that this difference may have affected the results, so future studies should include the assessment of the teacher of the students already identified in the regular classroom and not only of the teacher of the specialized educational service program.
We recommend that future studies examine the differential functioning of the items concerning other variables, such as age group and region of origin. In addition to the six dimensions discussed in this study, other TIAH/S studies are already planned to expand the screening tool to include additional dimensions. They include socioemotional, psychological, and motivational components. With the expansion of the areas, teachers will be able to evaluate students’ indications for a more detailed psychological evaluation in a broader way and to consider other types of giftedness.
Solange Muglia Wechsler
Editor
References
- Aguerri, M. E., Galibert, M. S., Lozzia, G. S., Abal, F. J. P., & Attorresi, H. F. (2007). Medida del funcionamento diferencial del item en el marco de la teoria de respuesta al item. Interdisciplinaria, 24(1), 95-110.
- Alencar, E. M. L. S., Fleith, D. S., & Carneiro, L. B. (2018). Gifted education in Brazil: historical background, current practices and research trends. In B. Wallace, D. A. Sisk, & J. Senior (Eds.), The SAGE handbook of gifted and talented education (pp. 2-13). SAGE Publications Ltd.
- Almeida, L. S., Lobo, C. C., Almeida, A. I. S., Rocha, R. S., & Piske, F. H. R. (2017). Processos cognitivos e de aprendizagem em crianças sobredotadas: Atenção dos pais e professores. In F. H. R. Piske, C. L. B. Vestena, J. M. Machado, A. O. M. Barby, T. Stoltz, S. Bahia, & S. P. Freitas (Orgs.), Processos afetivos e cognitivos de superdotados e talentosos (pp. 15-39). Editora Prismas.
- Almeida, L. S., Oliveira, E. P., & Melo, A. S. (2000). Bateria de instrumentos para a sinalização de alunos sobredotados e talentosos Universidade do Minho.
-
Almeida, L. S., Araújo, A. M., Sainz-Gómez, M., & Prieto, M. D. (2016). Challenges in the identification of giftedness: issues related to psychological assessment. Anales de Psicología, 32(3), 621-627. https://doi.org/10.6018/analesps.32.3.259311
» https://doi.org/10.6018/analesps.32.3.259311 - Alves, E. S. (2020). Altas habilidades/superdotação: como atender esses alunos? Caderno Intersaberes, 9(18), 214-227.
- Ambiel, R. A. M., & Carvalho, L. F. (2017). Validade e precisão de instrumentos de avaliação psicológica. In M. R. C. Lins & J. C. Borsa (Eds.), Avaliação psicológica: aspectos teóricos e práticos (pp. 115-125). Vozes.
-
Benson, N. F., & Kranzler, J. H. (2018). Another look at the construct validity of the Gifted Rating Scales: preschool/kindergarten and school forms. Journal of Psychoeducational Assessment, 36(8), 782-797. https://doi.org/10.1177/0734282917710377
» https://doi.org/10.1177/0734282917710377 - Bond, T. G., & Fox C. M. (2001). Applying the Rasch model: fundamental measurement in the human sciences Lawrence Erlbaum Associates.
- Bracken, B. A., & McCallum, S. (2018). UTAGS complete kit: universal talented and gifted screener Prufrock Press.
-
Cao, T. H., Jung, J. Y., & Lee, J. (2017). Assessment in gifted education: a review of the literature from 2005 to 2016. Journal of Advances Academics, 28(3), 163-203. http://doi.org/10.1177/1932292X17714572
» https://doi.org/10.1177/1932292X17714572 - Dai, D. Y. (2018). A history of giftedness: a century of quest for identity. In S. Pfeiffer (Ed.), APA Handbook of giftedness and talent (pp. 3-23). American Psychological Association.
- Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists Lawrence Erlbaum.
- Fabio, R. A., & Buzzai, C. (2019). Identifying giftedness: validation of an Italian language giftedness checklist for teachers and parents. Australasian Journal of Gifted Education, 28(1), 36-47.
-
Freitas, M. F. R. L., & Schelini, P. W. (2018). Escala de identificação de dotação e talento: construção de instrumento e validade de conteúdo. Interação em Psicologia, 2(2), 114-122. https://dx.doi.org/10.5380/psi.v22i2.46959
» https://doi.org/10.5380/psi.v22i2.46959 -
Freitas, M. F. R. L., Schelini, P. W., & Pérez, R. (2017). Escala de identificação de dotação e talento: estrutura e consistência internas. Psico-USF, 22(3), 473-484. https://doi.org/10.1590/1413-82712017220308
» https://doi.org/10.1590/1413-82712017220308 -
Gagné, F. (2010). Motivation within the DMGT 2.0 framework. High Ability Studies, 21(2), 81-99. https://doi.org/10.1080/13598139.2010.525341
» https://doi.org/10.1080/13598139.2010.525341 - Gentry, M. L., Pereira, N., Peters, S. J., McIntosh, J. S., & Fugate, C. M. (2015). HOPE Teacher Rating Scale (manual): involving teachers in equitable identification of gifted and talented students in K-12 Prufrock Press.
- Gilliam, J. E., Carpenter, B. O., & Christensen, J. R. (1996). Gifted and Talented Evaluation Scales PRO-ED.
- Guenther, Z. C. (2012). Crianças dotadas e talentosas... não as deixem esperar mais! LTC.
- Hertzog, N. B., Mun, R. U., Duruz, B., & Holliday, A. A. (2018). Identification of strengths and talents in young children. In S. I. Pfeiffer (Ed.), APA Handbook of Giftedness and Talent (pp. 3018-316). American Psychological Association.
-
Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (2019). Censo Escolar: Educação Especial MEC. http://portal.inep.gov.br/web/guest/resultados-e-resumos
» http://portal.inep.gov.br/web/guest/resultados-e-resumos - Irby, B., & Lara-Alecio, R. (1996). Attributes of Hispanic gifted bilingual students as perceived by bilingual educators in Texas. SABE Journal, 11, 120-143.
- Kaufman, S. B., & Sternberg, R. J. (2018). Conceptions of giftedness. In S. Pfeiffer (Ed.), Handbook of giftedness in children: Psycho-Educational theory, research and best practices (pp. 71-91). Springer.
- Kruczeveski, L. R., & Silva, A. O. G. (2018). Altas habilidades/superdotação na educação básica: apontamentos a partir da perspectiva de gênero. Alabastro: Revista eletrônica dos alunos da Escola de Sociologia e Política de São Paulo, 2(11), 37-51.
-
Linacre, J. M. (2015). A user’s guide to winsteps ministep: rasch-model computer programs (Version 5.2.3). Winsteps. https://www.winsteps.com/winman/copyright.htm
» https://www.winsteps.com/winman/copyright.htm -
Martins, B. A., Chacon, M. C. M., & Almeida, L. S. (2018). Estudo comparativo luso-brasileiro sobre a formação inicial de professores em altas habilidades/superdotação com enfoque nos conteúdos curriculares. Revista Brasileira de Educação Especial, 24(3), 309-326. https://doi.org/10.1590/s1413-65382418000300001
» https://doi.org/10.1590/s1413-65382418000300001 -
Matheis, S., Kronborg, L., Schmitt, M., & Preckel, F. (2017). Threat or challenge? Teacher beliefs about gifted students and their relationship to teacher motivation. Gifted and Talented International, 32(2), 134-160. http://doi.org/10.1080/15332276.2018.1537685
» https://doi.org/10.1080/15332276.2018.1537685 -
McBee, M. T., Peters, S. J., & Miller, E. M. (2016). The impact of the nomination stage on gifted program identification: a comprehensive psychometric analysis. Gifted Child Quarterly, 60(4), 258-278. https://doi.org/10.1177/0016986216656256
» https://doi.org/10.1177/0016986216656256 - McCallum, R. S., & Bracken, B. A. (2012) The universal multiple abilities scales examiner’s manual PRO-ED.
- McCarney, S. B., & Anderson, P. D. (1998). The Gifted Evaluation Scale: Technical manual (2nd ed.) Hawthorne Educational Services.
-
McClain, M., & Pfeiffer, S. (2012). Identification of gifted students in the United States today: a look at state definitions, policies and practices. Journal of Applied Psychology, 28(1), 59-88. https://doi.org/10.1080/15 377903.2012.643757
» https://doi.org/10.1080/15 377903.2012.643757 - McCluskey, K. (2018). Gifted education: the future awaits. In B. Wallace, D. A. Sisk, & J. Senior (Eds.), The SAGE handbook of gifted and talented education (pp. 553-565). SAGE Publications Ltd.
-
Mendonça, L. D., Rodrigues, O. M. P. R., & Capellini, V. L. M. F. (2020). Alunos com altas habilidades/superdotação: como se veem e como são vistos por seus pais e professores. Educar em Revista, 36, e71530. https://doi.org/10.1590/0104-4060.71530
» https://doi.org/10.1590/0104-4060.71530 -
Milic, S., & Simeunovic, V. (2022). Concordance between giftedness assessments by teachers, parents, peers and the self-assessment using multiple intelligences. High Ability Studies, 33(1), 1-9. http://doi.org/10.1080/13598139.2020.1832445
» https://doi.org/10.1080/13598139.2020.1832445 -
Ministério da Educação e Cultura (2010). Políticas públicas para as altas habilidades/superdotação MEC. http://www.senado.gov.br/web/comissoes/CE/AP/AP20080626_superdotados_Cl%C3%A1udiaGriboski.pdf
» http://www.senado.gov.br/web/comissoes/CE/AP/AP20080626_superdotados_Cl%C3%A1udiaGriboski.pdf - Miranda, L. M., Araújo, A. M., & Almeida, L. S. (2013). Identification of gifted students by teachers: reliability and validity of the cognitive abilities and learning scale. Revista de Investigación y Divulgación em Psicologia y Logopedia, 3(2), 14-18.
-
Mohamed, A. H. H., & Omara, E. M. N. (2020). Validation of an Arabic version of the Gifted Rating Scale - school form. Journal of the Education of the Gifted, 43(3), 252-269. http://doi.org/10.1177/0162353220933005
» https://doi.org/10.1177/0162353220933005 -
Mohamed, A. H. H., Kazem, A. M., Pfeiffer, S., Alzubaidi, A.-Q., Elwan, R. A., Ambosaidi, A., Al-Washahi, M. & Al-Kharosi, T. (2017). Identification of gifted students in Oman: gender and grade differences on the Gifted Rating Scales- school form. Journal for the Education of the Gifted, 40(3), 289-301. http://doi.org/10.1177/0162353217718277
» https://doi.org/10.1177/0162353217718277 - Nakano, T. C. (2020). Triagem de indicadores de altas habilidades/superdotação para professores: desenvolvimento de instrumental. Revista Diálogos e Perspectivas em Educação Especial, 7(1), 71-86.
- Nakano, T. C. (2021). Triagem de indicadores de altas habilidades/superdotação: versão professor Editora Vetor.
- Nakano, T. C., & Campos, C. R. (2019). Avaliação psicológica das altas habilidades/superdotação: problemas e desafios. In C. R. Campos & T. C. Nakano (Orgs.), Avaliação Psicológica direcionada a populações específicas: técnicas, métodos e estratégias volume II (pp. 99-128). Editora Vetor.
-
Nakano, T. C., & Oliveira, K. S. (2019). Triagem de indicadores de altas habilidades/superdotação: estrutura fatorial. Avaliação Psicológica, 18(4), 448-456. http://dx.doi.org/10.15689/ap.2019.1804.18478.13
» https://doi.org/10.15689/ap.2019.1804.18478.13 - Nakano, T. C., & Primi, R. (2020). Triagem de indicadores de altas habilidades/superdotação: validade de critério. Estudos em Avaliação Educacional, 31(77), 472-495.
- Nakano, T. C., Campos, C. R., & Santos, M. V. (2016). Escala de avaliação das altas habilidades/superdotação - versão professor: validade de conteúdo. Estudos Interdisciplinares em Psicologia, 7(1), 103-123.
-
Nakano, T. C., Gozzoli, M. Z., Alves, R. J. R., & Campos, C. R. (2016). Investigación de la eficácia de uma escala de evaluación de altas habilidades - versión professor. Revista de Estudios y Experiencias en Educación, 15(29), 83-94. https://doi.org.br/10.21703/rexe.20162983945
» https://doi.org/10.21703/rexe.20162983945 - Nakano, T. C., Miliani, A. F. M., Caporossi, D. C., & Gozzoli, M. Z. (2020). Evidências de validade do subteste de criatividade verbal da bateria de avaliação das altas habilidades/superdotação. Revista Iberoamericana de Criatividade e Inovação, 1(1), 13-25.
-
Oliveira, A. R. A., Amorim, R. M., & Pizzi, L. C. V. (2018). Disciplina profissão docente em um curso de Pedagogia: trajetórias, experiências e inovações na formação docente. Revista Eletrônica de Educação, 12(1), 263-278. http://doi.org/10.14244/19827199
» https://doi.org/10.14244/19827199 - Pfeiffer, S. I. (2015). Essentials of gifted assessment John Wiley.
-
Pfeiffer, S. I., & Blei, S. (2008). Gifted identification beyond the IQ test: Rating scales and other assessment procedures. In S. I. Pfeiffer (Ed.), Handbook of giftedness in children: psychoeducational theory, research, and best practices (p. 177-198). Springer Science. https://doi.org/10.1007/978-0-387-74401-8_10
» https://doi.org/10.1007/978-0-387-74401-8_10 - Pfeiffer, S. I., & Jarosewich, T. (2003). Gifted Rating Scales. The Psychological Corporation.
- Pfeiffer, S. I., Kumtepe, A., & Rosado, J. (2006). Gifted identification: Measuring change in a student’s profile of abilities using the Gifted Rating Scales. The School Psychologist, 60(3), 106-111.
- Rangni, R. A., Rossi, C. S., & Koga, F. O. (2021). Estudantes com altas habilidades ou superdotação: desdobramentos dos índices da Sinopse Estatística e dos Microdados na região sudeste do Brasil. Research, Society, and Development, 10(4), e7710413856. http://doi.org/10.33448/rsd-v10i4.13856
-
Renzulli, J. S., & Reis, S. M. (2021). The three-ring conception of giftedness: change in direction from being gifted to the development of gifted behaviors. In R. J. Sternberg & D. Ambrose (Eds.), Conceptions of Giftedness and Talent (pp. 335-356). Palgrave Macmillan. https://doi.org/10.1007/978-3-030-56869-6_5
» https://doi.org/10.1007/978-3-030-56869-6_5 -
Renzulli, J. S., Siegle, D., Reis, S. M., Gavin, M. K., & Reed, R. E. S. (2009). An investigation of the reliability and factor structure of four new scales for rating the behavioral characteristics of superior students. Journal of Advanced Academics, 21(1), 84-108. https://doi.org/10.1177/1932202X0902100105
» https://doi.org/10.1177/1932202X0902100105 - Renzulli, J. S., Smith, L. H., White, A. J., Callahan, C. M., Hartman, R. K., & Westberg, K. L. (2002). Scales for rating the behavioral characteristics of superior students - revised edition Creative Learning Press.
-
Ridgley, L. M., Rubenstein, L. D., & Finch, W. H. (2019). Issues and opportunities when using rating scales to identify creatively gifted students: applying an IRT approach. Gifted and Talented International, 34(1-2), 6-18. http://doi.org/10.1080/15332276.2020.1722041
» https://doi.org/10.1080/15332276.2020.1722041 -
Rothenbusch, S., Voss, T., Golle, J., & Zettler, I. (2018). Linking teacher and parent rating of teacher-nominated gifted elementary school students to each other and to school grades. Gifted Child Quarterly, 62(2), 230-250. http://doi.org/10.1177/0016986217752100
» https://doi.org/10.1177/0016986217752100 - Ryser, G. R., & McConnell, K. (2004). Scales for identifying gifted students Prufrock Press.
- Smith, J. A. (2004). Reflecting on the development of interpretative phenomenological analysis and its contribution to qualitative research in psychology. Qualitative Research in Psychology, 1(1), 39-54.
- Sternberg, R. J. (2000). Psicologia Cognitiva ArtMed.
-
Suárez, J. T., & Wechsler, S. M. (2019). Escala de identificação de talentos pelo professor, ITP: evidências de validade e precisão. Revista de Educação Especial, 32, 1-21. http://dx.doi.org/10.5902/1984686X30814
» https://doi.org//10.5902/1984686X30814 - Tirri, K. (2017). Teacher education is the key to changing the identification and teaching of the gifted. Roeper Review, 39(3), 210-212.
- Turkman, B. (2020). The evolution of the term of giftedness & theories to explain gifted characteristics. Journal of Gifted Education and Creativity, 7(1), 17-24.
- van der Linden, W. J., & Hambleton, R. K. (1996). Handbook of modern item response theory Springer-Verlag.
- Wechsler, S. M., Blumen, S., & Bendelman, K. (2018). Challenges on the identification and development of giftedness in South America. In S. Pfeiffer (Ed.), APA Handbook of Giftedness and Talent (pp. 97-112). American Psychological Association.
- Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement Mesa Press.
-
Zaia, P., & Nakano, T. C. (2020). Escala de Identificação das Altas Habilidades/Superdotação: evidências de validade de critério. Revista Iberoamericana de Diagnóstico y Evaluación e Avaliação Psicológica, 2(55), 31-41. https://doi.org/10.21865/RIDEP55.2.03
» https://doi.org/10.21865/RIDEP55.2.03 -
Zaia, P., Nakano, T. C., & Peixoto, E. M. (2018). Escala de Identificação de Características de Altas Habilidades/Superdotação: análise da estrutura interna. Estudos de Psicologia (Campinas), 35(1), 39-51. https://doi.org/10.1590/1982-02752018000100005
» https://doi.org/10.1590/1982-02752018000100005 - Zaia, P., Nakano, T. C., Miller, J., & Plucker, J. A. (2019). Identifying talented students efficiently and equitably. Sobredotação, 16, 109-124.
-
1
How to cite this article: Nakano, T. C., & Peixoto, E. M. (2023). Item analysis of the Teacher’s Gifted Rating Screening Scale. Estudos de Psicologia (Campinas), 40, e200056. https://doi.org/10.1590/1982-0275202340e200056
Support
Publication Dates
-
Publication in this collection
01 May 2023 -
Date of issue
2023
History
-
Received
15 Apr 2020 -
Reviewed
26 May 2021 -
Accepted
25 Apr 2022
Note: CI: General Intellectual Ability, CR: Creativity, HI: Academic Ability
Note: TA: Artistic Talent, LI: Leadership.