Inventory of Supporting for Socio-Emotional Skills - Forced-Choice Version

Inventário de Suporte para Competências Socioemocionais: Versão Escolha Forçada

Inventario de Apoyo para Habilidades Socioemocionales: Versión Elección Forzada

Abstract

Socioemotional skills are essential for students’ development at school. However, school support has barely generated debates. We aimed to analyze the evidence of the internal structure and convergent-discriminant validity of the scores of the forced-choice version of the Inventory of Supporting for Socio-Emotional Skills (ISSE). In total, 443 middle and high school students (54% of whom were girls) participated in this study. Results indicated that the revised instrument was consistent with its two-dimension theoretical model (support and demand); the forced-choice version showed an appropriate internal structure and no bias by acquiescence. Moreover, based on a multitrait-multimethod matrix, the scores of the Likert forced-choice version showed convergent and discriminant validity. This study contributes to the literature by proposing a multidimensional heteroevalutive instrument to assess support for socioemotional development in the forced-choice format, which reduces response bias.

Keywords:
inventories; psychometrics; bias; factor analysis; teacher characteristics

Resumo

As habilidades socioemocionais são essenciais para o desenvolvimento dos estudantes na escola; no entanto, há pouca discussão sobre o apoio das escolas. Nosso objetivo foi analisar as evidências de estrutura interna e validade convergente-discriminante da versão de escolha forçada do Inventário de Apoio para Habilidades Socioemocionais. No total, participaram 443 estudantes do ensino fundamental e médio (54% do sexo feminino). Os resultados indicaram que o instrumento revisado estava consistente com o modelo teórico de duas dimensões (apoio e demanda); a versão de escolha forçada demonstrou estrutura interna apropriada e não foi enviesada por aquiescência; além disso, com base em uma matriz mutitraço-multimétodo, as pontuações da versão de escolha forçada mostraram validade convergente e discriminante com a versão Likert. Este estudo contribui para a literatura propondo um instrumento heteroavaliativo multidimensional para avaliar o apoio ao desenvolvimento socioemocional no formato de escolha forçada, o que reduz o viés de resposta.

Palavras-chave:
inventário; psicometria; viés; análise fatorial; características do professor

Resumen

Las habilidades socioemocionales son esenciales para el desarrollo de los estudiantes en la escuela; sin embargo, hay poca investigación sobre el apoyo ofrecido por la escuela. Se pretende analizar las evidencias de estructura interna y validez convergente-discriminante de la versión de elección forzada del Inventario de Apoyo para Habilidades Socioemocionales. Un total de 443 estudiantes de la primaria y la secundaria (54% eran niñas) participaron en el estudio. Los resultados indicaron que la herramienta es adecuada al modelo de dos dimensiones (apoyo y demanda); la versión de elección forzada demostró una estructura interna adecuada y libre de aquiescencia; y, además, con base en una matriz multirasgo-multimétodo, los puntajes de la versión de elección forzada mostraron validez convergente con la versión Likert. Este estudio contribuye a la literatura con una herramienta heteroevaluativa multidimensional para examinar el apoyo al desarrollo socioemocional en el formato de elección forzada, lo que reduce el sesgo de la encuesta.

Palavras clave:
inventarios; psicometria; sesgo; análisis factorial; características del professor

Socioemotional skills predict academic success and school retention (Duckworth & Seligman, 2005; Oberle et al., 2014). Socioemotional skills can be developed informally by the unique experiences of each subject and formally by specific programs for socioemotional learning and teacher support. Despite the growing number of studies on socioemotional skills, little is known about teacher support for development. Therefore, the Inventory of Supporting for Socio-Emotional (ISSE) skills was developed to evaluate the support offered by teachers (Valentini et al., 2020). However, the type of items in the original version of the instrument may lead to response bias. In this context, this study sought to test the internal structure of the inventory in the forced-choice version to better control for response bias.

Socioemotional skills are characteristics associated with thoughts, emotions, and behaviors that predict positive outcomes in life, such as performance, health, and well-being (Duckworth & Seligman, 2005; Oberle et al., 2014; Primi et al., 2021; Weissberg et al., 2015). They can be observed as behavior patterns in personal or social situations (Instituto Ayrton Senna [IAS], 2020). Moreover, their importance is evinced in the Brazilian educational guidelines of its National Common Core Curriculum (BNCC) as a fundamental component for human development in dimensions such as self-awareness, self-management, social awareness, relationship skills, and responsible decision-making (Ministério da Educação, 2018).

Studies by the IAS (2020) understand socioemotional skills in a two-level hierarchical dimensional model. The broadest (second-order) level encompasses five dimensions or macro-competences: Engaging with Others, Amity, Self-Management, Emotional Resilience, and Openness (Primi et al., 2021). The more specific (first-order) level consists of facets of each dimension: (1) determination, organization, focus, persistence, and responsibility; (2) social initiative, assertiveness, and enthusiasm; (3) empathy, respect, and trust; (4) stress tolerance, self-confidence, and frustration tolerance; and (5) curiosity to learn and creative imagination (IAS, 2020).

Socioemotional skills based on personality traits, as in the case of the Senna Institute’s model, should be related to the teaching-learning process as they can be developed or reinforced in the school environment. The school plays a fundamental role in life-long learning and well-being as this institution can support the development of students’ socioemotional skills. However, how teachers deal with everyday situations and their role in supporting the development of socioemotional skills remains poorly understood (Primi et al., 2021).

Some models have been published but they have evaluated the general aspects of the school climate, technical aspects, and teaching didactics to the detriment of the socioemotional relationship between teachers and students (Marchezini-Cunha & Tourinho, 2010; Vinha et al., 2016). The model by Vinha et al. (2016) seeks to understand the school climate by variables related to the establishment of rules and conflict resolution, whereas the model of Marchezini-Cunha and Tourinho (2010) classifies teachers’ didactic style in three dimensions: aggressive, which only considers teachers’ rights; passive, which only considers students’ rights; and assertive, which considers teachers’ and students’ rights.

The dimensions proposed by Marchezini-Cunha and Tourinho (2010) can theoretically be encompassed in a general model of parenting styles with two orthogonal axes: demand and support. The demand dimension deals with the requirements to meet goals and objectives and establish desirable behaviors; whereas the support dimension corresponds to the emotional aspects of empathy and flexibility (Darling & Steinberg, 1993; Steinberg et al., 1994). The interaction between these two axes supports the understanding of support for socioemotional development in four styles: authoritative (high demand and high support), authoritarian (high demand and low support), permissive (low demand and high support), and negligent (low demand and low support) (Darling & Steinberg, 1993). Emotional regulation and educational social skills bear an association. Teachers who have a better understanding of their own emotions can offer a more pedagogical approach and promote greater emotional support to their students due to the greater clarity in their ability to approve and value behaviors (Justo & Andretta, 2020).

The support and demand dimensions applied to the school context can help understanding the support schools offer to students. However, only the ISSE (Valentini et al., 2020) is available to evaluate socioemotional support. The ISSE was built based on two dimensions, inspired by the parental styles of demand and support (Darling & Steinberg, 1993; Steinberg et al., 1994).

The ISSE is available in a Likert-type version, in which individuals classify each item on a five-point scale (level of disagreement or agreement with the statements). Respondents should consider a single stimulus at a time, i.e., an absolute judgment of the items in one response (Brown & Maydeu-Olivares, 2018). Although a relatively simple task, these scales may contain response bias (Bensch et al., 2019; Valentini et al., 2020).

Response biasderive from situational demands or from a lack of adequate judgment of test items. One of the most common response bias is social desirability, which is associated with the perceived importance and valuation of the evaluated behavior (for example, an individual with poor social skills who applies for a position that requires interpersonal communication skills will tend to positively endorse items that evaluate social skills) (Bensch et al., 2019; Paulhus, 2002; Paulhus & John, 1998). Another common bias is acquiescence (Acquiescence Response Style), which refers to the systematic uniformity of responses (i.e., a tendency of respondents to agree regardless of the items content). Another common bias is the Halo Effect, which concerns a generalist assessment of items based on a central piece of information. In this case, first impressions increase the weight of the subsequent characteristics of variables.

One of the possibilities of control for response bias is the use of forced-choice scales. In this format, individuals analyze more than one item at a time (block items). Respondents should point out the items that characterize their perception the most and the least and rank items according to their priority (Brown & Maydeu-Olivares, 2018). This study provides an example in Figure 1.

Figure 1
Example of a forced-choice item

The use of forced choice can reduce the level of social desirability if the items in the same block are equally biased. In the case of extreme, median, and acquiescent responses, comparative judgment can eliminate bias. Regarding the halo effect, on the other hand, the obligation of comparative judgment causes respondents to evaluate several characteristics of questionnaire items, tending to reduce the uniform endorsement of all items and increasing discrimination between factors (Brown & Maydeu-Olivares, 2011, 2018).

Thus, controlling for response bias is an increasingly present and necessary strategy to maintain the psychometric quality of the instruments, as well as the estimates of validity and accuracy. Latent models, such as the Thurstonian item response theory (T-IRT), can be used to estimate models with forced-choice items (Brown & Maydeu-Olivares, 2018). In the T-IRT model, subjects compare items in a same block based on a utility parameter. If, within the block, respondents order items so that A > B, then the utility of A will be greater than B (disregarding the associated measurement error). The utility parameters, in turn, are linked to the latent (factor) variable related to the content of questionnaire items (Figure 2). Therefore, the utility parameter is a ghost proxy between the latent factor and the pairwise comparison of items. As the pairwise comparison of the items depends on two utilities, the relationship between utility and binary variable is known a priori and therefore fixed in the model (see Figure 2, values 1 and −1). Brown and Maydeu-Olivares (2011, 2018) offer additional technical explanations of the model.

Figure 2
Generic Model by Brown & Maydeu-Olivares (2011; 2018) for forced-choice items

Note. A vs. B, A vs. C… represent the pairwise comparison between items A, B, and C; ut = utility of the item; f = descriptive factor (of content); ⎣ = factor loading (between factor and utility).

The first version of the ISSE (Valentini et al., 2020) showed promising internal structural validity and acquiescence control but its authors indicated the need for further research using forced-choice models as a possibility to control for response bias. Moreover, the factorial structure of the first version addressed only one dimension. Thus, we aimed to analyze the evidence of internal structure and convergent-discriminant validity of the scores of the forced-choice version of the ISSE; specifically seeking evidence of validity based on its internal structure and of convergent and discriminant validity of the Likert version, estimating acquiescence and correlating it with the factors of the forced-choice version.

Method

Participants

A total of 464 students in the 8th and 9th grades of middle school and of 1st and 2nd grades of high school participated in this study. Data were collected from a non-probabilistic sample at 10 public (80%) and private schools in the state of Rio de Janeiro, Brazil. Data were collected within a group of researchers across various projects from 2016 to 2020. Participants eligible for inclusion were required to be actively attending regular school classes given that the inventory focuses on the relationship between students and teachers. However, 14 cases were excluded from the sample due to uniform responses across all items. Additionally, we removed seven cases that had a significant number of missing items, specifically those missing 20% or more of the inventory. The final sample totaled 443 students, 54% of whom were girls. Students aged from 12 and 23 years participated in this study (M = 15.4 years, SD = 1.58). To calculate the sample size power, we conducted Monte Carlo simulations with 500 datasets, which were designed to replicate the models used in this study.

Instruments

Inventory of Supporting for Socio-Emotional Skills - ISSE-FC (forced-choice version). In this study, this instrument consisted of 60 items, grouped into blocks of three phrases. Therefore, this version of the instrument was composed of 20 triplets. For each block, participants indicated the items that characterize their school the most and the least. Thus, participants listed the order of importance and occurrence of items. An example of how to mark the forced-choice items is shown in Figure 1.

Inventory of Supporting for Socio-emotional Skills (Likert version, Valentini et al., 2020). Inventory composed of 60 items answered on a Likert scale (1 = not characteristic at all, 5 = totally characteristic). The instrument is balanced into positive and negative items to control for acquiescence. When answering the inventory, students should evaluate the support teachers provide to their socioemotional development. The original study showed evidence of its internal structure (unidimensional model showed a CFI=.94, RMSEA = .06 and loadings ranging from .35 to .75).

Procedures

Data collection. All students responded to both versions of the instruments (Likert and forced-choice). The applications took place in person in the classrooms. Applications averaged around 40 minutes. First, an informed assent form was shown to students, who only answered the instruments after agreeing with it.

Data analysis. The data were analyzed in the context of structural equation modeling. The Likert version was analyzed using confirmatory Factor Analysis. This study sought to control for acquiescence by the random intercepts model (Maydeu-Olivares & Coffman, 2006), in which a general method factor is estimated by fixing all factor loadings at 1 for positive and negative items. The method factor is declared as orthogonal to the substantive factors.

For the forced-choice version, this study first recoded the items into binary variables (0 and 1), which are, in turn, associated with the utility parameter. Utility consists of the psychological value respondents attribute to a particular item. In a comparison {i, k}, if subjects prefer item i over k (i.e., if the utility of i is greater than or equal to k), the binary variable was encoded as 1. If the utility of k is greater than that of i, the binary variable was encoded as 0. This research analyzed the data using the Thurstonian factorial model (Brown & Maydeu-Olivares, 2011), a restricted model in which the internal structure is defined a priori. In this second-order model, the observed binary variables are modeled by the utility parameters, which are therefore predicted by latent traits.

The latent factors of both versions (Likert and forced-choice) were then correlated. Moreover, this study also checked the correlation of the acquiescence method factor with the forced-choice factors. The estimator used was the Unweighted Least Squares Mean and Variance. All items were declared as categorical. All analyses were performed on Mplus (version 8.0), considering the following goodness of fit: Comparative Fit Index (CFI) ≥ 0.90, Tucker-Lewis Index (TLI) ≥ 0.90; and a Root Mean Square Error of Approximation (RMSEA) ≤ 0.08 (Schermelleh-Engel et al., 2003).

Ethical Considerations

This study was authorized by the Research Ethics Committee of the Universidade São Francisco (CAAE No. 23740619.8.0000.5514). All participants and guardians volunteered and signed assent/consent forms.

Results

The initial model with all triplets showed a non-positively defined matrix and factor loadings inconsistent with the theoretical model. Restricting the model solved the non-positive matrix: three residual variances to values above 0.01; and four factor loadings of negative items were restricted to values below −0.01. Still, some loadings remained non-significant or non-interpretable. To solve this problem and make the model coherent with the theory, only eight triplets were maintained, totaling 24 items. The reduction was expected as the forced-choice factorial model is sensitive to the composition of the items in triplets. The final model showed an acceptable fit: X ² (gl)= 398.78 (239), p < .01 RMSEA= 0.04 (90% CI [0.03 - 0.05]), CFI = 0.93, TLI = 0.91. An alternative method to evaluate model fit in a forced-choice format involves analyzing its residual matrix. This matrix showed values predominantly close to 0, with a range from 0 to 0.38 and an average of 0.07, indicating that the model strongly represents the data.

The latent modeling for forced choices estimates relationships between observed comparisons of the items, latent variables proxies of utility, and the latent variables of descriptive content (see Figure 2). It is therefore a second-order model in which utilities are first-order factors. Considering that one utility is estimated for each item, the relationship between the utilities and the second-order content factor can be interpreted as a traditional factor loading. To facilitate the interpretation, we standardized its factor loadings (Table 1).

Table 1
Factor loadings of the confirmatory factor analysis in forced choice and Likert formats (with and without control for acquiescence): Support and Demands factors

		Factor loading
		Forced choice		Likert		Likert (controling for acquiescence)
Triplet	Item	Suppor	Deman	Suppor	Deman	Suppor	Deman
1	01. My teachers encourage their students to research what they do not know.	0.71		0.40		0.38
	02. My teachers get the students excited.	0.90		0.58		0.57
	03. My teachers are overly critical.		0.38		−0.27 a		−0.29
2	04. My teachers motivate their students to be creative.	0.64		0.65		0.63
	05. My teachers demand the best from their students.		0.69		0.74		0.69
	06. My teachers get annoyed when their students ask questions.	−0.04ns		−0.36		−0.39
3	13. My teachers demand dedication from their students.		0.54		0.64		0.59
	14. My teachers say students can have a bright future.	0.37		0.55		0.54
	15. My teachers do not mind if students are bullied.	−0.96		−0.68		−0.71
4	22. My teachers demands students to be organized.		0.74		0.61		0.57
	23. My teachers encourage their students to be optimistic.	0.71		0.62		0.60
	24. My teachers criticize innovative students.	−0.6ns		−0.20		−0.23
5	28. My teachers believe in their students’ ability.	0.71		0.74		0.73
	29. My teachers ask their students to be calm when they are agitated.	0.36		0.40		0.38
	30. My teachers make students feel in a bad mood.	−0.48		−0.50		−0.53
6	37. My teachers teach their students to respect differences.	0.71		0.40		0.37
	38. My teachers correct the assigned homework.		0.58		0.46		0.43
	39. My teachers do not give feedback on the exercises their students make.		−0.04ns		0.30		−0.32
7	46. My teachers have no patience with their students.	−0.67		−0.59		−0.61
	47. My teachers humiliate their students.	−1.00		−0.65		−0.67
	48. My teachers keep their students in a good mood.	0.01		0.60		0.58
8	52. My teachers prevent their students from being participatory.		−0.39		0.02ns		−0.01ns
	53. My teachers ridicule their students.	−0.77		−0.71		−0.72
	54. My teachers praise organized students.	0.35		0.55		0.55

• Note. ns = not significant; ^a = this item was tested in an alternative version as support but remains negative in the Likert version.

Some loadings totaled 0. Although insignificant, these loadings and their respective items remained in the model as the exclusion of one of them would imply the exclusion of the entire block. For the Support factor, the items with greater loadings involve encouraging the use of creativity (item 4) and the perception that teachers believe in their students (item 28); as well as negative aspects such as ridiculing students (item 53) and ignoring bullying (item 15). Thus, results indicate that the content of the scale concentrated on the affective and emotional aspects of teacher support. We also highlight that the negative items of the support factor had high loadings. This indicates that the absence of support and socioemotional neglect are important aspects to understand support at schools.

For the Demand factor, the items with the greatest loadings involve requirement/rigorous (item 5), organization (item 22) and dedication (item 13), and homework correction (item 38). We emphasize that the negative items of this dimension had low factor loadings. This may have occurred due to the difficulty to elaborate genuinely inverted items for the lack of demand and, consequently, this study needed to use the word ‘no’ in the composition of some items.

Table 1 also shows the factor loadings of the Likert version of the scale, which were similar to those of its forced-choice version. The control for acquiescence failed to substantially influence most of the factor loadings. However, bias control was important for some items, especially 39 (“My teachers do not give feedback on exercises…”). This item is theoretically negative for the demand factor but its loading was reversed in the Likert version before controlling for acquiescence and again became negative after controlling for acquiescence. Overall, the Likert version seems to distinguish the support and demand content less than the forced-choice version. For example, item 3 (“my teachers are too critical”) is positive for high teacher demand and loaded positively in the forced-choice version but negatively in the Likert version. It is possible that, in the Likert version, the students assessed the extent to which the teacher is “pleasant” (and being “overly critical” is not usually “pleasant”) instead of when criticisms involve demands for improvement. We also sought to evaluate the relationship between the factors of both versions and acquiescence (Table 2).

Table 2
Correlations between the support and demand factors of the forced-choice and Likert instruments with and without controlling for acquiescence

	1.	2.	3.	4.	5.
1. Support (FC)		0.61	0.86	0.63	0.01 ns
2. Demand (FC)	0.62		0.53	0.88	−0.04 ns
3. Support (L)	0.86	0.52		0.8	0 (fixed)
4. Demand (L)	0.61	0.81	0.77		0 (fixed)
5. Acquiescence	na	na	na	na	0.09a

• Note. ns = Non-significant relationship; Below the diagonal are the correlations without control for acquiescence; above the diagonal, the correlations with control for acquiescence; ^a = Variance of Acquiescence; FC = forced-choice questionnaire; L = Likert-format questionnaire; na = not applicable (acquiescence is not estimated in the model described below the diagonal).

The variance for acquiescence in the Likert questionnaire totaled 9%. The control for acquiescence failed to significantly alter the relationship between factors. We also emphasizes the absence of significant correlations between the support and demand factors of the forced-choice scale and the acquiescence method factor. Thus, the acquiescence of the Likert scales and the substantive factors of the forced choice show no shared variance. This further evinces that the forced-choice version is free from acquiescence bias.

Regarding the convergent and discriminant validity of the instrument, it is noteworthy that the monotrait-multimethod relationship was higher than the multitrait-monomethod one. For example, the correlation between support-FC and support-Likert (same trait, different methods) was higher than the correlation between support-FC and demand-FC (different traits, same method). This is important evidence for the discrimination between the support and demand factors. The correlation between the support and demand factors was lower in the forced-choice scales (around 0.60) than in the Likert scales (around 0.80). Therefore, the Likert version seems to show a strong method variance, which controlling for acquiescence failed to completely remove. The support and demand factors of the forced-choice scales, in turn, may have showed a lower relationship due to its lower method variance.

Discussion

In this study we aimed to review the ISSE, test the internal structure of its forced-choice version, and estimate its acquiescence bias. The revised instrument better adhered to the two-dimensional theoretical model, and the forced choice version showed an adequate internal structure free from acquiescence.

The revised instrument adhered to the support and demands model (Darling & Steinberg, 1993; Steinberg et al., 1994). This was an important step toward consolidating the instrument. The structure of the previous version was unifactorial (Valentini et al., 2020), whose scores represented the extent to which students positively or negatively evaluated their teachers. Its one dimensionality restricts the assessment of nuances in the relationship between teachers and students. For example, is a teacher who offers low support (neither praising nor encouraging) similar to another who demands little (requires no dedication or participation)? Do both provide similar support for socioemotional development? The answer seems to be negative, and the results of the revised factorial structure support this conclusion. Therefore, two dimensions can help research to better understand the style of teachers and how a climate of collaboration and dedication is established.

The support dimension showed higher factor loadings. This indicates that support can better differentiate teacher evaluation. The perceived demand may also be more homogeneous among students; i.e., students differentiate less between teachers who demand or not (perhaps because most teachers are demanding). We emphasize that this does not mean that these aspects need intervention. On the other hand, the support (or lack of support) dimension can be investigated regarding socioemotional development and performance at schools. In other words, future studies should investigate whether the greater variability of teacher support is actually related to students’ socioemotional development.

Regarding the internal structure, the forced-choice version seems to better distinguish support and demand content. Although some Likert loadings are higher than those of the forced-choice version (this is expected due to the greater variability of the data in the Likert version), in the Likert version, students seem to respond more based on valence (somewhat dichotomous like or dislike toward teachers). Previous studies have also pointed to this phenomenon in teachers’ self-reports (Elies et al., 2021; Rothenbusch et al., 2018), which may be the result of response bias, especially the halo effect. Item 3, for example, about critical and demanding teachers, negatively loaded in the demands factor of the Likert version (although theoretically positive). When staging their responses, students may think about the displeasure criticism cause them and inversely respond to demand and charge (items 5 and 13, for example). When forced to indicate the most and least characteristic items in the forced-choice version, teachers’ criticism emerges as a demand. Therefore, the forced-choice version seems less susceptible to response bias, making it easier to estimate two distinct factors: support and demand.

Correlation results also indicate the distinction between support and demand factors and the convergent validity between scales. The test of two methodologically distinct versions (Likert and forced-choice) managed to evaluate the extent to which the latent factors correlate, distinguish, and are influenced by a method bias. Confirming the convergent validity, this study found higher correlations between the same dimensions of different versions of the scale (i.e., monotrait-multimethod, r _support = 0.86 and r _demand = 0.81) when compared to correlations between different dimensions in the same version (multitrait-monomethod, r _{Likert-Likert} = 0.77 and r _FC-FC = 0.62). These results indicate that most of the variance in items serves to estimate correlations between similar dimensions (evidence of convergent validity) and that the variance of the method is smaller (evidence of discriminant validity between support and demand). Therefore, our scale has an advantage over the previous instrument (Valentini et al., 2020) by encompassing teachers’ support with evidence of multidimensionality.

Our study also furthers the understanding of the bias of acquiescence in the self-reported instrument of teacher support. Earlier studies have pointed to the relevance of acquiescence control in self-reported instruments (Lechner et al., 2019) but it is hardly discussed in the self-report (Marder et al., 2021). In the Likert version of the instrument, acquiescence showed significant variance (9%). However, this seems to substantially interfere with the internal structure or convergence of factors. Therefore, acquiescence can influence the self-reported scores estimated by the instrument, even if it does not bias the structure. In practice, our suggestion refers to controlling the acquiescence of the estimated scores in the Likert version scores.

If control for acquiescence is important in the Likert version, the forced-choice version seems free from bias. Technically, acquiescence should not influence the responses to the forced-choice items as participants are unable to simultaneously endorse positive and negative items within the same block (Brown & Maydeu-Olivares, 2018). However, idiosyncratic responses are possible in distinct blocks (for example, endorsing a positive item in one block and its negative opposite in another block). Little has been studied about the effect of balancing positive and negative items in forced-choice blocks. Previous studies have found small correlations between acquiescence and the composite factors of forced-choice items (Valentini et al., 2020). This study found an insignificant correlation close to 0. The arrangement of the forced-choice blocks may turn the instrument slightly less susceptible to bias (for example, opposite pairs of items within the same block can differ in terms of the bias opposite pairs produce in distinct blocks); a hypothesis that can be tested in future studies. Still, acquiescence has a very small or non-significant relationship with content factors estimated by forced-choice items, which represents an advantage of this scale design.

Despite the advances this study offers, it is necessary to highlight some of its limitations. The first concerns the design of the blocks of forced-choice items: some were assembled with one-dimensional items, which makes it difficult to estimate latent scores and may explain non-significant factor loadings. The second limitation refers to factor imbalance: the support factor contains more items than the demand factor. The third limitation refers to the lack of design to control for halo bias and social desirability, especially in the forced-choice blocks.

Our study advances the literature by proposing a multidimensional self-reporting instrument to assess the support and demand for socioemotional development in a forced-choice format. The instrument shows evidence of internal and convergent structural validity. The forced-choice version seems to better distinguish between the support and demand factors and reduce the acquiescent response bias. We hope that the instrument can help researchers and technicians to understand and intervene in students’ socioemotional skills.

References

• Bensch, D., Paulhus, D. L., Stankov, L., & Ziegler, M. (2019). Teasing apart overclaiming, overconfidence, and socially desirable responding. Assessment, 26(3), 351-363. https://doi.org/10.1177/1073191117700268
  » https://doi.org/10.1177/1073191117700268
• Brown, A., & Maydeu‐Olivares, A. (2018). Modelling forced‐choice response formats. In P. Irwing, T. Booth, & D. J. Hugues (Eds.), The Wiley Handbook of Psychometric testing (pp. 523-570). Wiley-Blackwell. https://doi.org/10.1002/9781118489772.ch18
  » https://doi.org/10.1002/9781118489772.ch18
• Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71(3), 460-502. https://doi.org/10.1177/0013164410375112
  » https://doi.org/10.1177/0013164410375112
• Darling, N., & Steinberg, L. (1993). Parenting style as context: An integrative model. Psychological Bulletin, 113(3), 487-496. https://doi.org/10.1037/0033-2909.113.3.487
  » https://doi.org/10.1037/0033-2909.113.3.487
• Duckworth, A. L., & Seligman, M. E. P. (2005). Self-discipline outdoes IQ in predicting academic performance of adolescents. Psychological Science, 16(12), 939-944. https://doi.org/10.1111/j.1467-9280.2005.01641.x
  » https://doi.org/10.1111/j.1467-9280.2005.01641.x
• Elies, A., Schabmann, A., & Schimidt, B. M. (2021). Associations between teacher-rated behavioral problems and reading difficulties? Interactions over time and halo effects. Journal of Research in Special Educational Needs, 21(4), 368-380. https://doi.org/10.1111/1471-3802.12536
  » https://doi.org/10.1111/1471-3802.12536
• Instituto Ayrton Senna. (2020). Competências socioemocionais durante o Coronavírus [Socio-emotional skills during the Coronavirus]. https://institutoayrtonsenna.org.br/pt-br/socioemocionais-para-crises.html?utm_source=site&utm_medium=hub-botao-2206#o-que-sao-competencias-socioemocionais
  » https://institutoayrtonsenna.org.br/pt-br/socioemocionais-para-crises.html?utm_source=site&utm_medium=hub-botao-2206#o-que-sao-competencias-socioemocionais
• Justo, A. R., & Andretta, I. (2020). Competências socioemocionais de professores: Avaliação de habilidades sociais educativas e regulação emocional [Teachers’ socio-emotional competencies: Assessment of educational social skills and emotional regulation]. Revista Psicologia da Educação, (50), 104-113. http://pepsic.bvsalud.org/scielo.php?script=sci_abstract&pid=S1414-69752020000100011
  » http://pepsic.bvsalud.org/scielo.php?script=sci_abstract&pid=S1414-69752020000100011
• Lechner, C. M., Partsch, M.V., Danner, D., & Rammstedt, B. (2019). Individual, situational, and cultural correlates of acquiescent responding: Towards a unified conceptual framework. Br J Math Stat Psychol, 72, 426-446. https://doi.org/10.1111/bmsp.12164
  » https://doi.org/10.1111/bmsp.12164
• Marchezini-Cunha, V., & Tourinho, E. Z. (2010). Assertividade e autocontrole: Interpretação analítico-comportamental [Assertiveness and self-control: Behavioral-analytic interpretation]. Psicologia: Teoria e Pesquisa, 26(2), 295-304. https://doi.org/10.1590/S0102-37722010000200011
  » https://doi.org/10.1590/S0102-37722010000200011
• Marder, J., Göllner, R., Wagner, W., & Fauth, B. (2021). Ask me, I (Dis)agree! Acquiescence in student ratings of teaching quality in German vocational schools. Studies in Educational Evaluation, 68, 100937. https://doi.org/10.1016/J.STUEDUC.2020.100937
  » https://doi.org/10.1016/J.STUEDUC.2020.100937
• Maydeu-Olivares, A., & Coffman, D. L. (2006). Random intercept item factor analysis. Psychological Methods, 11(4), 344-362. https://doi.org/10.1037/1082-989X.11.4.344
  » https://doi.org/10.1037/1082-989X.11.4.344
• Ministério da Educação. Secretaria da Educação Básica. (2018). Base Nacional Comum Curricular [Common Core Curriculum]. http://basenacionalcomum.mec.gov.br/wpcontent/uploads/2018/02/bncc-20dez-site.pdf
  » http://basenacionalcomum.mec.gov.br/wpcontent/uploads/2018/02/bncc-20dez-site.pdf
• Oberle, E., Schonert-Reichl, K. A., Hertzman, C., & Zumbo, B. D. (2014). Social-emotional competencies make the grade: Predicting academic success in early adolescence. Journal of Applied Developmental Psychology, 35(3), 138-147. https://doi.org/10.1016/j.appdev.2014.02.004
  » https://doi.org/10.1016/j.appdev.2014.02.004
• Paulhus, D. L. (2002). Socially desirable responding: The evolution of a construct. In H. I. Braun, D. N. Jackson, & D. E. Wiley (Eds.), The role of constructs in psychological and educational measurement (pp. 49-69). Lawrence Erlbaum.
• Paulhus, D. L., & John, O. P. (1998). Egoistic and moralistic biases in self‐perception: The interplay of self‐deceptive styles with basic traits and motives. Journal of Personality, 66(6), 1025-1060. https://doi.org/10.1111/1467-6494.00041
  » https://doi.org/10.1111/1467-6494.00041
• Primi, R., Santos, D., John, O. P., & De Fruyt, F. (2021). SENNA Inventory for the assessment of social and emotional skills in public school students in Brazil: Measuring both identity and self-efficacy. Frontiers in Psychology, 12, e716639. https://doi.org/10.3389/fpsyg.2021.716639
  » https://doi.org/10.3389/fpsyg.2021.716639
• Rothenbusch, S., Voss, T., Golle, J., & Zettler, I. (2018). Linking teacher and parent ratings of teacher-nominated gifted elementary school students to each other and to school grades. Gifted Child Quarterly, 62(2), 230-250. https://doi.org/10.1177/0016986217752100
  » https://doi.org/10.1177/0016986217752100
• Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures. Methods of Psychological Research, 8(2), 23-74.
• Steinberg, L., Lamborn, S. D., Darling, N., Mounts, N. S., & Dornbusch, S. M. (1994). Over-time changes in adjustment and competence among adolescents from authoritative, authoritarian, indulgent, and neglectful families. Child Development, 65(3), 754-770. https://doi.org/10.1111/j.1467-8624.1994.tb00781.x
  » https://doi.org/10.1111/j.1467-8624.1994.tb00781.x
• Valentini, F., Mose, L. B., Ramos, I. S., & Conceição, N. M. (2020). Development of the Inventory of Supporting for Socio-Emotional Skills, evidence of internal structure controlling for acquiescence. Estudos de Psicologia (Campinas), 37, e180161. https://doi.org/10.1590/1982-0275202037e180161
  » https://doi.org/10.1590/1982-0275202037e180161
• Vinha, T. P., Morais, A., Tognetta, L. R. P., Azzi, R. G., Aragão, A. M. F., Marques, C. A. E., Silva, L. M. F., Moro, A., Vivaldi, F. M. C., Ramos, A. M., Oliveira, M. T. A., & Bozza, T. C. L. (2016). O clima escolar e a convivência respeitosa nas instituições educativas [The school climate and respectful coexistence in educational institutions]. Estudos Em Avaliação Educacional, 27(64), 96-127. https://doi.org/10.18222/eae.v27i64.3747
  » https://doi.org/10.18222/eae.v27i64.3747
• Weissberg, R. P., Durlak, J. A., Domitrovich, C. E., & Gullotta, T. P. (2015). Social and emotional learning: Past, present, and future. In J. A. Durlak, C. E. Domitrovich, R. P. Weissberg, & T. P. Gullotta (Eds.), Handbook of social and emotional learning: Research and practice (pp. 3-19). The Guilford Press.

• This research received support from the National Council for Scientific and Technological Development (CNPq, Number 310415/2018-7)

Edited by

Associate editor: Samuel Lincoln Bezerra Lins

Publication Dates

• Publication in this collection
  05 Aug 2024
• Date of issue
  2024

History

• Received
  04 Dec 2023
• Reviewed
  25 Jan 2024
• Accepted
  05 Feb 2024