Validity of the Developmental Index for the Zulliger Test in children and adolescents

Validade do Índice de Desenvolvimento para o Teste de Zulliger em crianças e adolescentes

ABSTRACT

Objective:

Psychological maturity can be comprehended as a multifaceted construct encompassing various psychosocial, emotional, and cognitive attributes. The aim of this study was to investigate evidence of validity of the Psychological Development Index for the Comprehensive System Zulliger Test.

Method:

The data analysis considered the Mann-Whitney test, Cohen's d, Spearman's correlation and the coefficient difference test. Finally, multiple regression analysis was used to verify the relationship between age and Developmental Index with a 253 children and adolescents between 7 and 14 years of age participated.

Results:

The results indicated evidence of concurrent criterion validity to Development Index, pointing out statistically significant differences between age groups.

Conclusion:

Regarding the evidence of convergent validity of the Development Index between the Zulliger Comprehensive System and the Rorschach Performance Assessment System tests, nothing can be proven. This study is limited by the age range of the sample, as well as the transposition of the Rorschach Performance Assessment System Development Index calculation to the Zulliger Comprehensive System.

Keywords:
Development; Child; Reproducibility of results; Psychological tests.

RESUMO

Objetivo:

A maturidade psicológica pode ser compreendida como um construto multifacetado que abrange diversos atributos psicossociais, emocionais e cognitivos. O objetivo deste estudo foi investigar evidências de validade do Índice de Desenvolvimento Psicológico para o Teste Zulliger do Sistema Compreensivo.

Método:

A análise dos dados considerou o teste de Mann-Whitney, d de Cohen, correlação de Spearman e teste de coeficiente de diferença. Por fim, utilizou-se análise de regressão múltipla para verificar a relação entre idade e Índice de Desenvolvimento, com a participação de 253 crianças e adolescentes entre 7 e 14 anos.

Resultados:

Os resultados indicaram evidências de validade de critério concorrente para o Índice de Desenvolvimento, apontando diferenças estatisticamente significativas entre as faixas etárias.

Conclusão:

Quanto às evidências de validade convergente do Índice de Desenvolvimento entre os testes do Sistema Compreensivo Zulliger e do Sistema de Avaliação de Desempenho Rorschach, nada pode ser comprovado. Este estudo limita-se à faixa etária da amostra, bem como à transposição do cálculo do Índice de Desenvolvimento do Sistema de Avaliação de Desempenho de Rorschach para o Sistema Compreensivo Zulliger.

Palavras-chave:
Desenvolvimento da personalidade; Criança; Reprodutibilidade dos resultados; Testes psicológicos

Psychological maturity can be understood from a set of psychosocial, emotional and cognitive characteristics that are expected to become more sophisticated as age increases (Fossas, 2019), and individuals with impairments in the development of these characteristics tend to present psychological immaturity. In this way, the concept of maturity encompasses several aspects that must be understood in their interrelation (Papalia et al., 2019).

Psychological maturity is made of both psychosocial maturity and emotional and cognitive maturity. Psychosocial maturity is the component that involves the development of skills for decision-making, autonomy, formation of personal identity and motivation for establishing interpersonal relationships (Icenogle et al., 2019). In turn, emotional maturity can be perceived as greater control of emotions is identified (Subbarayan & Visvanathan, 2011), being expected that, as age increases, the individual develops more sophisticated psychological resources to deal with the stress of daily life. Concerning cognitive maturity, individuals with better development of this characteristic tend to have higher levels of intelligence and sophistication of thought (Resende et al., 2019).

The importance of a study like this lies in the early detection of levels of psychological development, as it is possible to identify the individual's potential growth, in addition to various information about their capabilities in different aspects of life. Delays in this area are related to adaptive failures and troubles to cope with the challenges of daily life and may be responsible for serious psychological disorders - with impairments in emotional and intellectual aspects, in social life, in the school and family environment - or for physical aggression due to the difficulty of controlling one's own behavior (Morales-Vives et al., 2020; Morales-Vives & Dueñas, 2018). Thus, when identifying signs of psychological immaturity, immediate interventions that are more appropriate for each case can be advised.

To assess psychological maturity, the Development Index (DI) was created, a composite measure created for the Rorschach test, with the aim of measuring psychological development from childhood to adulthood in this instrument (Stanfill et al., 2013). This index allows comparisons between psychological development and chronological age, with the aim of identifying immaturity or precocity present in a protocol. The DI consists of a composite score that assesses everything from the complexity of dealing with life's demands, the ability to understand the consequences of one's actions, to observe events from different angles, and to concern about one's physical and psychological integrity and intellectualization capacity of distressing events (Stanfill et al., 2013). Table 1 presents the variables that constitute the index and their respective meanings.

Table 1
Comparative analysis of the Developmental Index for the Comprehensive System Zulliger Test, according to age, sex and type of school (N = 253)

Variables	Meaning
F%	Level of complexity or psychological effort in coping with life's demands
FQ-% and FQo%	Accuracy of perception and understanding of the environment, influencing judgment and decision-making
m	Perception of states of mental tension and recognition of stressors present in the environment
T and V	Ability to capture subtleties and inconsistencies in the environment and interpersonal relationships
V and FD	Cognitive competence and the ability to withdraw and assess everyday situations from different perspectives
(r)	Information processing from the self
An + Sx	Concerns with adult issues such as physical and mental integrity, sex, gender and procreation
Art	Representation of emotional experiences through intellectual processes

• Note: Based on the studies by Meyer et al. (2017).
• F% - Percentage of Form Determinant
• FQ-% and FQo% - Percentage of Form Determinant with negative Formal Quality and with Ordinary Form Quality
• m - Determinant of inanimate movement
• T and V - Texture and View Determinant
• V and FD - Determinant of View and Form Dimension
• (r) - Reflex Determinant
• An + Sx - Sum of Anatomy and Sexual Content
• Art - Art Content

After the creation of the DI for the Rorschach, two studies investigated the evidence of the validity of this index with different samples. In a study conducted by Giromini et al. (2015), the authors used samples of patients from Milan, Turin and Ohio aged between 5 and 25 years old, with a total of 902 participants, with the aim of seeking evidence of validity for the DI with Italian and American samples. The authors considered the mental productivity in the test (R or number of responses in the test), the level of sophistication and flexibility to face tasks (Complexity Index), the level of psychological maturity (DI) and chronological age. They concluded that, in addition to the significant correlation between DI and age (r between 0.28 and 0.41, p < 0.01 considering the different samples), significant and positive correlations were also obtained between DI and complexity (r ranging from 0.40 to 0.49, p < 0.01), demonstrating that patients with greater psychological maturity ((DI) tended to have higher levels of sophistication and flexibility in coping with day-to-day demands ((Complexity).

The second study, conducted by Resende et al. (2019), also aimed to seek evidence of criterion validity for the DI on the Rorschach, but in a sample composed only of children (N = 231; ranging from 7 to 11 years of age). The authors verified the relationship of DI with age and non-verbal intelligence and, additionally, compared the level of psychological maturity (DI) of children who lived with their families and children who were neglected and lived in institutions for adoption. Regarding age, the results indicated that older children had a significantly higher DI than younger children (t = -2.08 and p = 0.04), considering the group of children with normative performance. Regarding the level of non-verbal intelligence, DI was strongly associated with this construct (r = 0.83 and p < 0.001). In addition, DI scores and non-verbal intelligence scores were significantly lower for children living in institutionalized settings due to family neglect (t = 4.43 and p < 0.001). Multiple regression analysis revealed that 71% of the variability in DI was associated with non-verbal intelligence, while age explained only 2%.

Assuming that the Zulliger test uses inkblots as a task stimulus and it was developed as the image and likeness of the Rorschach test, the following research question arises: can the DI also be used for the Zulliger test? In a brief review of the literature, considering the last 5 years, it was observed that Zulliger validity studies for children pointed out that younger children tended to have lower mental productivity (↓R), were less accurate in the perception of reality, demonstrated less competence to understand the consequences of their actions and were more prone to emotional interference (FQ-, INC, S-%), as well as demonstrating a more fanciful perception of themselves and others [(H)] and being more naïve and dependent on the other (Fd) when compared with older children. On the other hand, older children tended to be more cognitively efficient, deliberately using thought and creativity (D, M, FD), to have more accurate perceptions, and to be more capable of understanding the consequences of their actions (WDA%, XA%) (Cardoso & Oliveira, 2018; Cardoso et al., 2018; Carvalho & Resende, 2018; Resende et al., 2012). Therefore, all these studies indicate that the Zulliger test, like the Rorschach test, can also detect the increase in psychological maturity with increasing chronological age in children from non-clinical groups.

So, to answer the research question, the general objective was to investigate the validity of the DI for the Zulliger Comprehensive System Test (ZSC) in children and adolescents. The specific objectives were: to identify whether the DI is capable of discriminating the different age groups, sex and type of schools in the ZSC Test; verify the convergent validity between the DI in the ZSC with the DI in the Rorschach Performance Assessment System (RPAS); verify the influence of age on the level of psychological maturity.

Method

Participants

All participants from a database who had been submitted to the ZSC and RPAS tests were selected, totaling 253 children and adolescents aged 7 to 14 years old (M = 10.4 and SD = 2.2), of both sexes, from public and private schools. The participants were divided into two groups: G1 composed of children aged 7 to 9 years old (n = 88) and G2 of adolescents aged 10 to 14 years old (n = 165), according to the division of age groups proposed by the World Health Organization (2020).

Instruments

Participant Characterization Form - Collection instrument completed by the guardian, used with the aim of identifying participants concerning sociodemographic characteristics and absence of serious psychopathologies.

Zulliger Test in the Comprehensive System (ZSC) - Individually done (Villemor-Amaral & Primi, 2020) - psychological instrument developed by Hanz Zulliger in the 1940s and used according to the Comprehensive System, which consists of the individual application of three inkblots, assessing the structure and dynamics of the participants' personalities. The answers were coded in the various categories that compose the Comprehensive System (Villemor-Amaral & Primi, 2020), and for this study, only the variables that constitute the DI were used (F%, FQo%, FQ-%, m, FD, T, V, r, An, Sx, Art ).

Rorschach Test - Performance Assessment System (RPAS, Meyer et al., 2017) - psychological instrument developed by Hermann Rorschach in the 1920s and used according to the Performance Assessment System, which consists of the individual application of ten inkblots, assessing the structure and dynamics of the participants' personalities. The answers were coded in the various categories that form the system, and for this study, only the variables that make up the DI were used, as indicated above.

Procedures

Signing the Informed Consent Form (ICF) and completing the Participant Characterization Form consisted of a step prior to the beginning of data collection. The ZSC and RPAS tests were applied in a standardized way according to their respective manuals. The instruments were applied in alternating order to reduce the effect of one on the other and in all test application environments it was possible to guarantee the privacy and confidentiality of the collected information.

All protocols were coded by the second author of this article. In order to establish inter-rater accuracy, 29% of the protocols (n = 73) were randomly selected and coded by an independent judge who did not have access to the original coding and study objectives.

Data Analysis

The variables of all protocols were analyzed using the program R. A Kolmogorov-Simirnov (K-S) normality test was performed with Lilliefors correction, which indicated the absence of normality (p < 0.05) for age and all Development Index variables, in both instruments and, therefore, non-parametric analyzes were used. Initially, a descriptive analysis of the variables was carried out, followed by a comparative analysis of the means of the Development Index for the Zulliger (DI-ZSC) and Rorschach (DI-RPAS) tests according to sex, age and type of school, using the Mann-Whitney test for independent samples. The calculation of the effect size was also performed using Cohen's d (Dancey & Reidy, 2019).

Then, the convergent validity was analyzed using Spearman's correlation coefficient between the number of responses (R) of the instruments. The expected correlation coefficients (r _s) for convergent validity studies are those greater than or equal to 0.40. Analyzes were performed for the total sample and stratified by sex and age. Correlation coefficients between gender and age groups were compared using the coefficient difference significance test.

Finally, multiple linear regression analysis was used to verify the relationship between age and DI, controlling the gender variable, with DI as the dependent variable and sex and age as the independent variables. The results of the regression model were presented as regression coefficient (β) and 95% confidence interval (95% CI), standardized regression coefficient (βp) and statistical significance (p value). The statistical significance of the model was established using the t test. All analyses presenting a p-value of ≤ 0.05 were considered statistically significant.

The research project was approved by the Research Ethics Committee of the Catholic University of Goiás (PUC-GO), under the Certificate of Presentation for Ethical Consideration 0156.0.168.000-11. The participation of children and adolescents was voluntary and ratified by assent in the ICF, previously signed by their legal guardians.

Results

Prior to the analyses, the accuracy between the evaluators was verified using the Intraclass Correlation Coefficient (ICC). For the ZSC test, the ICC ranged from 0.70 to 0.91 and for the RPAS the variation was between 0.85 and 1.00, with the reference values for the coefficients through the ICC classified as moderate between 0.40 to 0.75 and excellent if above 0.75 (Fleiss, 1981), showing moderate to excellent precision for the Zulliger test and excellent for the Rorschach test.

It is noteworthy that, analyzing the total group, the DI for the ZSC test had an average of 15.84 (SD = 3.39, Min = 9.34 and Max = 25.28), while for the RPAS test, the DI obtained a mean value of 16.46 (SD = 1.72, Min = 14 and Max = 21.76). Table 2 shows the results of a study of concurrent criterion validity evidence of the DI in the ZSC through comparative analysis of the index according to age groups, gender and type of school.

Table 2
Comparative analysis of the Developmental Index for the Comprehensive System Zulliger Test, according to age, sex and type of school (N = 253)

Variables	N	DI-ZSC
		M	SD	p*	d
Age
G1 (7-9)	88	14.91	3.22	0.002	0.403
G2 (10-14)	165	16.33	3.48
Sex
Female	129	15.45	3.36	0.106	0.204
Male	124	16.21	3.50
Type of school
Private	126	15.91	3.31	0.683	0.051
Public	127	15.73	3.60

• Note: *Mann-Whitney test for independent samples. d: Cohen's d; N: Number.
• G1: Grupo de participantes com idade entre 7 e 9 anos
• G2: Grupo de participantes com idade entre 10 e 14 anos
• DI-ZCS: Developmental Index - Comprehensive System Zulliger Test

Statistically significant differences were identified for age, with the result for G1 being lower than for G2 (G1: M = 4.91; SD = 3.22 and G2: M = 16.33 and SD = 3.48) with size of small effect (d = 0.403) and p = 0.002. On the other hand, gender and type of school did not present statistically significant differences, demonstrating that DI is not influenced by these variables for this sample.

Table 3 shows the correlation analysis between the variables of both tests, using Spearman's correlation coefficient.

Table 3
Convergent validity analysis, using Spearman's correlation coefficient, between variables (N = 253)

Variables	r s	95% CI	p
DI (RPAS and ZSC)	0.265	0.147; 0.387	< 0.001
F%	0.393	0.284; 0.492	< 0.001
m	0.173	0.051; 0.290	0.006
r	0.235	0.116; 0.348	< 0.001
SumT	-0.004	-0.127; 0.119	0.949
SumV	0.019	-0.104; 0.141	0.764
FD	-0.046	-0.168; 0.077	0.469
FQo%	0.134	0.011; 0.253	0.033
FQ-%	0.160	0.038; 0.277	0.011
An+Xy	0.175	0.053; 0.292	0.005
Art	0.041	-0.082; 0.163	0.519
Sx	-0.036	-0.158; 0.087	0.573

• Note: r _s: Spearman's correlation coefficient; CI: Confidence Interval.
• DI (RPAS and ZSC): Developmental Index (Rorschach Performance Assessment System and Zulliger Comprehensive System)
• F% - Percentage of Form Determinant
• m - Determinant of inanimate movement
• r - Reflex Determinant
• SumT - Sum of Texture Determinants
• Sum - Sum of View Determinants
• FD - Form Dimension
• FQo% - Percentage of Form Determinant with Ordinary Formal Quality
• FQ-% - Percentage of Form Determinant with Negative Formal Quality
• An+Xy - Sum of Anatomy and Sexual Content
• Art - Art Content
• Sx - Sexual Content

It was verified that the correlation between the DI of the ZSC and RPAS tests has a positive value, and is statistically significant, but weak (r _s = 0.265; 95% CI = 0.147 to 0.387; p < 0.001); therefore, the data must be considered insufficient to infer evidence of convergent validity between the instruments for this variable. Considering the variables that form the index, it was observed that 55% (n = 6) also obtained positive, statistically significant, but weak correlations. It was observed that the variable with the highest correlation coefficient was F% (r _s = 0.393; p < 0.001) and the lowest coefficient was FQ-% (r _s = 0.134; p < 0.033).

Table 4 shows the stratified correlation analysis, according to age group (G1 = 7 to 9 years old and G2 = 10 to 14 years old).

Table 4
Convergent validity analysis of the Developmental Index and its variables between the Zulliger Comprehensive System and Rorschach Performance Assessment System, using Spearman's correlation, according to age groups

Variables	G1 (n = 88)			G2 (n = 165)			p*
	r s	95% CI	p	r s	95% CI	p
DI	0.218	0.110; 0.391	0.048	0.287	0.127; 0.409	< 0.001	0.290
F%	0.285	0.081; 0.466	0.006	0.432	0.299; 0.548	< 0.001	0.208
m	0.239	0.032; 0.427	0.025	0.128	-0.025; 0.275	0.101	0.389
r	0.406	0.215; 0.567	< 0.001	0.200	0.047; 0.340	0.011	0.085
SumT	0.004	-0.205; 0.213	0.971	-0.012	-0.148; 0.156	0.876	0.905
SumV	-0.111	-0.313; 0.100	0.304	0.139	-0.014; 0.285	0.075	0.060
FD	-0.181	-0.376; 0.029	0.092	-0.017	-0.169; 0.136	0.832	0.215
FQo%	0.169	-0.041; 0.365	0.115	0.098	-0.055; 0.247	0.212	0.589
FQ-%	0.183	-0.027; 0.377	0.088	0.145	-0.007; 0.291	0.064	0.772
An+Xy	0.069	-0.142; 0.274	0.225	0.236	-0.087; 0.375	0.002	0.200
Art	0.109	-0.042; 0.365	0.314	0.009	-0.143; 0.161	0.911	0.453
Sx	-0.034	-0.241; 0.176	0.754	-0.037	-0.188; 0.166	0.635	0.984

• Note: *Test of statistical significance for two independent correlation coefficients. G1: Children aged 7-10 years old; G2: Adolescents aged 11-14 years old; r _s: Spearman's correlation coefficient; CI: Confidence Interval.
• DI: Developmental Index
• F% - Percentage of Form Determinant
• m - Determinant of inanimate movement
• r - Reflex Determinant
• SumT - Sum of Texture Determinants
• Sum - Sum of View Determinants
• FD - Form Dimension
• FQo% - Percentage of Form Determinant with Ordinary Formal Quality
• FQ-% - Percentage of Form Determinant with Negative Formal Quality
• An+Xy - Sum of Anatomy and Sexual Content
• Art - Art Content
• Sx - Sexual Content

A positive correlation was obtained, statically significant, but weak between the DI of both instruments for G1 (r _s = 0.218; 95% CI = 0.110; 0.391; p = 0.048) and G2 (r _s = 0.287; 95% CI = 0.127, 0.409, p < 0.001). When performing the statistical significance test for independent samples, no difference was identified between the significance of the correlations (p = 0.290), showing that, despite the samples being composed of very different numbers of participants, the correlation coefficients were not different in terms of magnitude.

For G1, weak but, positive and statistically significant correlations were identified for the variables F% (r _s = 0.285 and p = 0.006) and m (r _s = 0.239 and p = 0.025) and moderate, positive and statistically significant correlation for the variable r (r _s = 0.285 and p = 0.006). While in G2 the correlations obtained were weak, positive and statistically significant for the variables F%, r and An+Xy, having the variable F% (r _s = 0.299 and p = 0.001) reach the highest value among these, however, still below the expected (r _s ≥ 0.40), to indicate the existence of evidence of convergent validity.

Table 5 shows the stratified analysis of the correlation, according to the groups, considering the gender variable (male and female).

Table 5
Convergent validity analysis of the Developmental Index and its variables between the Zulliger Comprehensive System and Rorschach Performance Assessment System, using Spearman's correlation, according to gender

Variables	Female (n = 129)			Male (n = 124)			p*
	r s	95% CI	p	r s	95% CI	p
DI	0.278	0.159; 0.398;	0.001	0.224	0.114; 0.352;	0.012	0.325
F%	0.412	0.258; 0.545	< 0.001	0.365	0.202; 0.508	< 0.001	0.656
m	0.167	-0.006; 0.330	0.059	0.192	0.017; 0.356	0.032	0.842
r	0.306	0.141; 0.454	< 0.001	0.191	0.016; 0.355	0.033	0.337
SumT	-0.006	-0.178; 0.167	0.942	0.014	-0.162; 0.189	0.876	0.873
SumV	0.008	-0.165; 0.180	0.924	0.021	-0.155; 0.196	0.731	0.923
FD	-0.019	-0.191; 0.154	0.834	-0.070	-0.243; 0.107	0.438	0.483
FQo%	0.188	-0.016; 0.349	0.033	0.067	-0.110; 0.240	0.462	0.332
FQ-%	0.187	0.015; 0.348	0.034	0.142	-0.035; 0.310	0.117	0.719
An+Xy	0.170	-0.002; 0.333	0.055	0.180	0.004; 0.345	0.046	0.936
Art	-0.036	-0.207; 0.137	0.689	0.148	-0.029; 0.316	0.124	0.147
Sx	-0.039	-0.210; 0.134	0.657	-0.025	-0.200; 0.151	0.781	0.912

• Note: *Test of statistical significance for two independent correlation coefficients. r _s: Spearman's correlation coefficient; CI: Confidence Interval.
• DI: Developmental
• F% - Percentage of Form Determinant
• m - Determinant of inanimate movement
• r - Reflex Determinant
• SumT - Sum of Texture Determinants
• Sum - Sum of View Determinants
• FD - Form Dimension
• FQo% - Percentage of Form Determinant with Ordinary Formal Quality
• FQ-% - Percentage of Form Determinant with Negative Formal Quality
• An+Xy - Sum of Anatomy and Sexual Content
• Art - Art Content
• Sx - Sexual Content

This analysis pointed to a positive and significant correlation between the DI between ZSC and RPAS in girls (r _s = 0.278; 95% CI = 0.159; 0.398; p = 0.001) and boys (r _s = 0.224; 95% CI = 0.114; 0.352; p= 0.012), as well as no statistical difference was identified between the correlation coefficients of the groups (p = 0.325). For the group of girls, moderate to weak, positive and statistically significant correlations were identified for the variables F%, r, FQo% and FQ-%, but only F% (r _s = 0.412 and p < 0.001) reached the parameter value for convergent validity. Considering the group of boys, no variable reached the minimum acceptable value (r _s = 0.4).

Despite the DI and some variables indicating a correlation for the independent groups, it is noteworthy that no variable showed a difference between the correlation coefficients, demonstrating that psychological maturity, as well as the variables that compose the index, do not seem to be influenced with regard to the sex. Finally, Table 6 presents the linear regression analysis.

Table 6
Multiple linear regression analysis of the association between age and Developmental Index (N = 253)

Variables	β	95% CI	βp	SD	p
DI-RPAS
Age	0.12	0.02; 0.22	0.15	0.051	0.017
Sex	0.30	-0.14; 0.73	0.08	0.223	0.186
Intercept
F (p): 3.82 (p = 0.023)
R2 = 0.051
DI-ZSC
Age	0.42	0.24; 0.60	0.28	0.089	< 0.001
Sex	0.77	-0.04; 1.57	0.11	0.401	0.063
Intercept
F (p): 12.43 (p < 0.001)
R2 = 0.091

• Note: β: Regression coefficient; β_p: Standardized regression coefficient; CI: Confidence Interval; F: Test F; R ²: Determination Coefficient.
• DI-RPAS: Developmental Index (Rorschach Performance Assessment System)
• DI-ZSC: Developmental Index (Zulliger Comprehensive System)

The relationship between age and DI measured by the two instruments was verified, including the gender variable in the model for adjustment. The analyzes showed positive associations between age and the index in the ZSC (βp = 0.28; p < 0.001) and in the RPAS (βp = 0.15; p = 0.017). As a result, 9.1% of the DI can be explained as a function of age, while for the RPAS test, the result indicates that 5.1% of the DI result can be explained by this same variable, suggesting that little of the DI variable can be explained in terms of age for this sample.

Discussion

The present study aimed to investigate DI validity for the ZSC in children and adolescents. The data pointed to evidence of concurrent criterion validity of the DI in the ZSC through the statistically significant differences of the index between children and adolescents, showing that children have less psychological maturity when compared to adolescents, demonstrating that the DI is influenced in terms of age, as well as the sensitivity of the instrument to capture the personality nuances associated with development (Resende et al., 2019).

A limiting factor in this sample is the restricted age range of 7 to 14 years old, even more so when it is divided into 7-9 and 10-14 years old. This is a classic range constraint problem that can mitigate or reduce the differences between age groups considering the DI. This age range is much smaller than that used by Stanfill et al. (2013) and Giromini et al. (2015), who compared children, adolescents and adults aged between 8 to 45 and 5 to 25 years old, respectively. However, even when the group of participants in this study was divided into children and adolescents, reducing the amplitude of age groups, the differences were significant, which corroborates the validity criterion of the DI in Zulliger. It is expected that there will be larger differences and larger effect sizes, with studies involving a broader age range.

On the other hand, the absence of differences between genders and types of schools corroborates the expectations of previous studies with the index applied to the Rorschach (Nascimento et al., 2017), showing that children and adolescents do not essentially differ in the vast majority of test variables, although some differences can be found in approximately 5% of the variables studied, not essentially interfering with psychological maturity in juvenile samples by sex or types of schools.

In addition to evidence of criterion validity, the analyzes sought evidence of convergent validity between the DI-ZSC and DI-RPAS, but the data did not indicate this type of evidence validity. That is, through the correlation between the DI in the ZSC and in the RPAS, it was observed that the DI-ZSC sensitively captures the characteristics associated with psychological development, since the DI-RPAS already has studies that indicate its validity for evaluating different levels of psychological development (Giromini et al., 2015; Resende et al., 2019).

Five DI variables (45%; insert variables here) did not correlate across the tests in this sample, perhaps suggesting an important limitation of DI when applied to Zulliger. One hypothesis for the lack of correlation between these variables in the tests is their low prevalence in children and adolescents (Resende et al., 2019). Variables of low prevalence are those with an average of less than one response per protocol. This type of variable favors low correlations, as they can be more easily neglected when coding a test.

Additional analyzes of the prevalence of DI variables showed that 63% (n = 7) of them had low prevalence in both instruments (T, V, FD, r, An+Xy, Art, Sx), so the interval restriction of these variables may have reduced the power of these analyzes. It is possible that the differences are more apparent in samples that include a larger age range. Possibly, the developmental aspects of the variables may be relevant in older people. On the other hand, the DI variables with the highest prevalence in the children's tests (F%, FQo% and FQ-%) were significantly related between both tests.

Furthermore, it may be that these variables are not so directly correlated between the tests too, since there are not always correlations between the performance of a respondent between the two tests (Gonçalves et al., 2019; Villemor-Amaral & Cardoso, 2012; Villemor-Amaral et al., 2016).

It is also understood that one of the major limitations of this study is the transposition of the DI developed for the Rorschach test (SC or RPAS) to the ZSC, disregarding that the weights of the index variables were calculated based on a series of empirical procedures that justify index syntax in Rorschach SC or RPAS. Stanfill et al. (2013) used various methodologies based on regression analysis to select the best combination of variables that, as a group, optimized age prediction, which was not done for the development of the index for the Zulliger. In addition to this limitation, the low amplitude of the age range may also have limited studies on the validity of this index, which was constructed and validated based on large samples composed of a wide extent of age ranges.

Regardless of all this, it is noteworthy that the Conselho Federal de Psicologia (2018) advises that correlations ≥0.40 are sufficient for evidence of convergent validity between different instruments that assess the same construct. In this sense, the DI, which is observed in the whole group or subdivided by age group or gender, did not show evidence of convergent validity between the two tests. Furthermore, only 9.1% of psychological development could be explained by age in Zulliger, while in Rorschach only 5% of DI was associated with age for this sample. It is worth noting that studies such as that by Villemor-Amaral et al. (2016) point out that the Zulliger test is an instrument that enables a more agile evaluation, which would not necessarily indicate the same depth resulting from evaluations with the Rorschach tests.

Conclusion

This study provides support for evidence of concurrent DI criterion validity, as the results point to statistically significant differences between age groups. Regarding the evidence of convergent validity of the DI between the ZSC and RPAS tests, nothing can be proven, both when considering age groups and gender as independent variables. Therefore, the current results do not suggest that the DI in the ZSC can be used as a procedure to measure psychological maturity since the associations found, although positive and significant, are weak to support such a relationship with the DI-RPAS.

It is also understood that one of the major limitations of this study is the transposition of the DI developed for the Rorschach test (SC or RPAS) to the ZSC, disregarding that the weights of the index variables were calculated based on a series of empirical procedures that justify the index syntax in Rorschach SC or RPAS (Stanfill et al., 2013). In addition to this limitation, the low amplitude of the age range may also have limited studies on the validity of this index, which contains variables of low prevalence among children, which interferes with correlation studies.

It is suggested that future studies with the DI applied to the ZSC should consider samples of a higher age range, as well as seek to verify the validity of the index by considering samples with different psychopathologies. Thus, reinforcing findings that DI is compromised in situations which there are psychological damages. Again, it is worth noting that the Zulliger and Rorschach tests are not equivalent and linear between their stimuli (Villemor-Amaral et al., 2016), which may also be a justification for the discrepancy in findings between these instruments.

References

• Cardoso, L. M., & Oliveira, J. C. (2018). Meninos e meninas: Influências culturais no método de Zulliger. Revista Avaliação Psicológica, 17(1), 101-110. https://doi.org/10.15689/ap.2017.1701.11.13361
  » https://doi.org/10.15689/ap.2017.1701.11.13361
• Cardoso, L. M., Gomes, G. V. A., & Vieira, T. S. (2018). Validity evidence of the Zulliger-SC Test to children’s assessment. Psico-USF, 23(3), 451-460. https://doi.org/10.1590/1413-82712018230305
  » https://doi.org/10.1590/1413-82712018230305
• Carvalho, A. C. M., & Resende, A. C. (2018). Desempenho de crianças e adolescentes não pacientes no Zulliger SC. Revista Avaliação Psicológica, 17(1), 142-154. https://doi.org/10.15689/ap.2017.1701.15.13795
  » https://doi.org/10.15689/ap.2017.1701.15.13795
• Conselho Federal de Psicologia. (2018). Resolução 009 de 2018, 29 CFP.
• Dancey, C. P., & Reidy, J. (2019). Estatística sem matemática para psicologia (7th ed.). Penso.
• Fleiss, J. L. (1981). Métodos estatísticos para taxas e proporções (2o edição). New York: John Wiley.
• Fossas, A. (2019). Psychological maturity predicts different forms of happiness. Journal of Happiness Studies, 20(6), 1933-1952. https://doi.org/10.1007/s10902-018-0033-9
  » https://doi.org/10.1007/s10902-018-0033-9
• Giromini, L., Viglione, D. J., Brusadelli, E., Lang, M., Reese, J. B., & Zennaro, A. (2015). Cross-cultural validation of the Rorschach developmental index. Journal of Personality Assessment, 97(4), 348-353. https://doi.org/10.1080/00223891.2014.960927
  » https://doi.org/10.1080/00223891.2014.960927
• Gonçalves, A. P., Zuanazzi, A. C., & Villemor-Amaral, A. E. (2019). Aplicação R-Otimizada no Zulliger: Evidências de validade com pacientes depressivos. Avaliação Psicológica, 18(2), 111-120. https://doi.org/10.15689/ap.2019.1802.16215.01
  » https://doi.org/10.15689/ap.2019.1802.16215.01
• Icenogle, G., Steinberg, L., Duell, N., Chein, J., Chang, L., Chaudhary, N., Di Giunta, L., Dodge, K. A., Fanti, K. A., Lansford, J. E., Oburu, P., Pastorelli, C., Skinner, A. T., Sorbring, E., Tapanya, S., Uribe Tirado, L. M., Alampay, L. P., Al-Hassan, S. M., Takash, H. M. S., & Bacchini, D. (2019). Adolescents' cognitive capacity reaches adult levels prior to their psychosocial maturity: Evidence for a "maturity gap" in a multinational, cross-sectional sample. Law and Human Behavior, 43(1), 69-85. https://doi.org/10.1037/lhb0000315
  » https://doi.org/10.1037/lhb0000315
• Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E., & Erdberg, P. (2017). R-PAS Sistema de Avaliação Por Performance no Rorschach (1st ed.). Hogrefe.
• Morales-Vives, F., & Dueñas, J. M. (2018). Predicting suicidal ideation in adolescent boys and girls: The role of psychological maturity, personality traits, depression and life satisfaction. The Spanish Journal of Psychology, 21, E10. https://doi.org/10.1017/sjp.2018.12
  » https://doi.org/10.1017/sjp.2018.12
• Morales-Vives, F., Dueñas, J.-M., & Ferrando, P. J. (2020). The role of psychological maturity and the big five personality traits in the victimization through indirect aggression. Psicothema, 34(2), 192-199. https://doi.org/10.7334/psicothema2020.491
  » https://doi.org/10.7334/psicothema2020.491
• Nascimento, R. S. G. F., Resende, A. C., & Ribeiro, R. S. (2017). Crianças, adolescentes e o método de Rorschach Pearson.
• Papalia, D. E., Feldman, R. D., & Martorell, G. (2019). O mundo da criança: Da infância à adolescência (13th ed.). Artmed.
• Resende, A. C., Carvalho, T. C. R., & Martins, W. (2012). Desempenho Médio de Crianças e Adolescentes no Método de Rorschach Sistema Compreensivo. Avaliação Psicológica, 11(3), 375 -394. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712012000300006&lng=pt&tlng=pt
  » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712012000300006&lng=pt&tlng=pt
• Resende, A. C., Viglione, D. J., Martins, L. D., & Yazigi, L. (2019). Criterion validity of the Rorschach developmental index with children. Journal of Personality Assessment, 101(2), 191-198. https://doi.org/10.1080/00223891.2017.1368021
  » https://doi.org/10.1080/00223891.2017.1368021
• Stanfill, M. L., Viglione, D. J., & Resende, A. C. (2013). Measuring psychological development with the rorschach. Journal of Personality Assessment, 95(2), 174-186. https://doi.org/10.1080/00223891.2012.740538
  » https://doi.org/10.1080/00223891.2012.740538
• Subbarayan, K., & Visvanathan, G. (2011). A study on emotional maturity of college students. Resent Research in Science and Technology, 3(1), 153-155. https://updatepublishing.com/journal/index.php/rrst/article/view/558
  » https://updatepublishing.com/journal/index.php/rrst/article/view/558
• Villemor-Amaral, A. E. De, Pianowski, G., & Carvalho, L. D. F. (2016). Issues about color, human movement, and number of responses in the Zulliger test. Rorschachiana, 37(2), 95-113. https://doi.org/10.1027/1192-5604/a000068
  » https://doi.org/10.1027/1192-5604/a000068
• Villemor-Amaral, A. E., & Cardoso, L. M. (2012). Validade convergente do tipo de vivência (EB) no Teste de Zulliger/SC. Psico, 43(1), 109-115. https://revistaseletronicas.pucrs.br/ojs/index.php/revistapsico/article/view/11105
  » https://revistaseletronicas.pucrs.br/ojs/index.php/revistapsico/article/view/11105
• Villemor-Amaral, A. E., & Primi, R. (2020). ZSC Teste de Zulliger no sistema compreensivo: Forma individual (1st ed.). Hogrefe.
• World Health Organization. (2020). Adolescent health WHO. https://www.who.int/health-topics/adolescent-health#tab=tab_1
  » https://www.who.int/health-topics/adolescent-health#tab=tab_1

• 1
  Article based on the dissertation by L. P. PEREIRA, entitled “Propriedades Psicométricas, Maturidade Psicológica e Engajamento Cognitivo e Afetivo do Teste de Zulliger”. Pontifícia Universidade Católica de Goiás, 2021.

• Support:

  Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) - Edital: 01/2019 and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes) - Edital: 01/2020.

Edited by

Editor:

Tatiana de Cássia Nakano

Publication Dates

• Publication in this collection
  26 Aug 2024
• Date of issue
  2024

History

• Received
  27 Apr 2021
• Reviewed
  16 Nov 2022
• Accepted
  26 July 2023