Validad Incremental entre la Prueba de Wartegg y Rorschach (R-PAS)

paideia Paidéia (Ribeirão Preto) Paidéia (Ribeirão Preto) 0103-863X 1982-4327 Universidade de São Paulo, Faculdade de Filosofia Ciências e Letras de Ribeirão Preto, Programa de Pós-Graduação em Psicologia Resumen Validad incremental dice respecto a lo cuanto una medida puede añadir a la previsión de un criterio, más de lo que puede ser previsto por otras fuentes de datos, o sea, de cual manera un instrumento puede complementar y auxiliar en la comprensión de informaciones obtenidas por otro. El objetivo del estudio fue verificar la evidencia de validez incremental entre la Prueba de Wartegg y lo Rorschach (R-PAS). Participado 40 sujetos con edades entre 21 y 70 años, divididos en dos grupos, un compuesto por pacientes con diagnóstico de esquizofrenia y otro, por sujetos sin histórico de enfermedad psiquiátrica. Todos respondieron a lo Roraschach y a la Prueba de Wartegg. Los resultados indicaron capacidad predictiva entre los instrumentos de 75% para la variable Calidad formal, 98% para Movimiento y 100% para Contenido. Se sugieren nuevos estudios acerca de las evidencias de validad, con amuestras mayores y también análisis de otras variables no exploradas en el presente estudio. Psychological evaluation is defined as a process that aims to understand instances of the psychological functioning of individuals, in order to make possible understandings, thus operating scientifically based interventions (Cunha, 2000; Noronha & Alchieri, 2004). In this process, psychological tests stand out for their great usefulness in identifying nuances of psychological functioning, but they are not the only source of information available, other sources such as interviews, anamneses or even the synthesis of various instruments is necessary (Campos, 2013; Vieira, 2017). Although there are objective parameters for the use of the tests, it is important to understand the social, clinical and theoretical context that underlies each decision making, considered as the main objective of the evaluation process (Villemor-Amaral & Pasqualini-Casado, 2006). To be used, the tests must present minimum acceptable parameters such as evidence of validity, reliability and standardization so that they can support decision making (Urbina, 2007). However, although these parameters give reliability to the data found, they do not guarantee that the process will be developed in a satisfactory manner, and may even be limiting to a comprehensive understanding when faced in a dichotomous manner (Campos, 2013; Cunha, 2000). In this sense, some authors (Brackett & Mayer, 2003; Capitão & Cardoso, 2012; Franco & Villemor-Amaral, 2012; Haynes & Lench, 2003; Merino Soto, 2014; Silveira, Oliveira, & Bandeira, 2018) have used the proposal of incremental validity in the sense of complementing the data during a psychological evaluation process in different contexts. Broadly speaking, incremental validity refers to how much a measure can add to the prediction of a criterion, above what can be predicted by other data sources, that is, how one instrument can complement and assist in understanding information obtained by another (Hunsley & Meyer, 2003). Andrei, Siegling, Aloe, Baldaro and Petrides (2016) and Haynes and Lench (2003) point out that there are some different objectives and models in the verification of incremental validity, being linked to the validity of the instrument’s predictive power, content and criteria, the effectiveness of treatments, monitoring of changes in individuals, force of discrimination of sensitivity and specificity in addition to the ecological validity linked to different contexts. However, the authors synthesize that, even with particularities, they all seek to verify the predictive efficacy of a particular phenomenon. The operationalization in the search for incremental validity is done by a regression analysis in order to verify the predictive value of the instruments. The prediction is observed as the data sources are responsible for the variation of the observed criterion. Thus, they can contribute to the improvement in the prediction of important criteria such as diagnosis, treatment design or even future performance (Hunsley & Meyer, 2003; Ng & Feldman, 2015). Authors like Brackett and Mayer (2003), Capitão and Cardoso (2012) and Ng (2015), employed incremental validity seeking to understand how the complementarity of data can assist in the process of evaluation, understanding and consequent treatment of an individual. These authors conclude that low correlations do not indicate a problem about evidence of validity. Even if low, they can have a great impact on the interpretations performed in clinical practice, even more when reinforced by other techniques and sources of information. These indicators work as points of convergence between the instruments, which in turn, access information in a differentiated way, thus involving different forms of mediation, as Schmitt, Hofmann, Gschwendner, Gerstenberg and Zinkernagel comment on (2015). Regarding the two techniques used in this study, about the Rorschach test, it is possible to verify a large number of studies, which gives the technique the accumulation of favorable evidence as well as the constant development of new interpretations, which can be verified with the recent systematization performed by Meyer, Viglione, Mihura, Erard and Erdberg (2011), proposing the Rorschach Perfomance Assessment System (R-PAS), with the objective of improving the use of some variables used as well as optimizing the application (Mihura & Meyer, 2018; Pianowski, Meyer, Villemor-Amaral, Zuanazzi,l& Nascimento, 2019). The Wartegg Test, on the other hand, has not been the object of many studies, disfavoring its practical applicability. Considering publications from 2015 onwards, only five works were found in international literature. Rizzo, Della Villa and Crisi (2015) conducted a case study with a 17-year-old boy with anxious symptoms and possible delusions. The Watergg Test was used in the search of clinical/qualitative indicators being possible to identify predominantly psychological discomfort and depressive indicators, according to the authors. Crisi and Dentale (2016), conducted a study with the objective of verifying three classification categories of the Wartegg Test, namely evocative character (EC), formal quality (FQ) and affective quality (AQ). For this, 564 subjects participated, 290 male, with a mean age of 24.6 years (SD = 3.54) divided into three groups being anxious, psychotic and control group. The authors used the difference between means to verify significant differences between the groups. They also performed a precision analysis between judges, and 30 protocols of the study were evaluated by two judges finding good concordance rates (EC = 0.74; FQ = 0.92; AQ = 0.71). In the same year, Engelman et al. (2016), conducted a case study of a 49-year-old woman with suspected attention deficit and hyperactivity (ADHD), with the objective of the clinical understanding of the patient from MMPI-2 data, the Rorschach (SC) and the Wartegg Tests. The authors observed that the Wartegg Test showed indicators of sadness and depression, emphasizing that it does not present specific indicators for ADHD. Later, Vari et al. (2017) conducted a study to verify the correlation between skin diseases and depression, anxiety and suicidal ideation. For this they used the Wartegg Test, the Millon Clinical Multiaxial Inventory-III, in addition to a questionnaire on personality indicators. A total of 94 subjects participated in the study, 52 of whom were diagnosed with psoriasis and 42 were from the control group. The ages varied between 18 and 70 years (M = 42.05; SD = 11.21). The results for Wartegg indicated a difference between the groups for the variables affective quality (AQ) and global rejection (GR), and the authors concluded that the indicators favorable to the use of the tool were. Finally, Yuana, Harjunowibowo, Karyanta and Budiyanto (2018) sought to verify the effectiveness of a computerized system, using similarity of cosine, for interpretation of content indicators for the Wartegg Test. In this study, they used five protocols of the test. The results indicated that the interpretations made from the software analyses, proved to be consistent. In summary, in the last five years, five studies have been published with the Wartegg Test, being two case studies, one in which the test was used for information gathering, without, however, being the focus of the study and two in which its psychometric parameters were studied. This scenario motivated studies of the system proposed by Pessotto (2018a), based on the Rorschach test (R-PAS), and was later studied by Pessotto and Primi (2018c, 2018b). It is believed that studies involving these two instruments can help in the understanding of psychological instances through their incremental validity and clinical applicability. Therefore, the objective of this study was to verify evidence of incremental validity between the Wartegg and the Rorschach (R-PAS) tests based on indicators of formal quality, movement and content of the two tests. Method Participants Forty subjects participated in this study and they were divided into two groups. Group 1 consisted of 20 patients diagnosed with schizophrenia from two institutions, a psychiatric hospital located in the interior of São Paulo and a public institution with care for this population in the capital. For group 2, the subjects were paired, with no history of psychiatric diagnoses, considering age (SD = 5), ethnicity, marital status, gender and schooling. For all participants, age ranged from 21 to 70 (M = 40; SD = 12.8), 90% being male. About ethnicity, 70% declared themselves white, 15% black and 15% from other ethnic groups. Regarding schooling, 35% had incomplete elementary schooling, 25% complete elementary schooling, 20% incomplete high schooling, 5% complete high schooling, 5% incomplete high schooling, 5% complete high schooling and 5% post-graduation. The procedures were performed by the first author of this study, in two institutions in the interior of São Paulo, which offer psychiatric care, being the subjects of the clinical group approached directly in these institutions. In the control group, the subjects were contacted via internet, due to the statistical pairing and approached in their residences. Instruments Wartegg Test. Graphic self-expression test, which aims to identify aspects of personality. The method uses a stimulus sheet composed of eight squares of 4 cm X 4 cm. Each one contains a printed stimulus, and the subject is asked to continue the drawing, from these printed stimuli. The application lasts approximately 15 minutes. In this study the Pessotto system (2018a) was used. For this study, the variables formal quality (FQu and FQ-), movement (M, FM, m, a and p) and content (H, (H), Hd, A, Ad, (Ad), Na, Art, Ay, Cg, Fi and NC) were used, as they already present favorable evidence in previous studies (Pessotto & Primi, 2018a, 2018b). The formal quality indicates whether or not the design performed is adequate to the initial stimulus of the frame. Movement is used when the drawing indicates action of any nature and finally, the content refers to the elements present in the subject’s drawing. A deepening of the codes and their indicators presented in this system, can be verified in Pessotto and Primi (2018b, 2018c). Rorschach Perfomance Assessment System (R-PAS). A technique by Meyer et al. (2011) consists in sequentially evaluating ten boards containing paint stains, asking him to answer the question “what could this be? In this system (R-PAS) the subject must provide two to three answers per board. This phase is called association. It is followed by the clarification phase in which each answer is taken up, seeking to identify two questions, namely, what made the subject see a certain object and where the stain is located. The R-PAS presents concordance rates between judges ranging from 0.86 to 0.89 in a set of interpretations and also presents concordance rates ranging from 0.77 to 0.96 in response coding. In addition, it offers several indexes of evidence of validity, such as the ability to perceive emotions correlated to the proper perception of reality ([FQ]r = 0.30) and generate emotional states to facilitate the work, correlated to lower levels of stressors ([m]r = 0.25). In this study formal quality variables (FQo, FQu, FQ- and FQn) were used, which indicates the suitability of the figure seen to the contours of the stain, movement (M, FM, m, a and p), characterizing figures seen with some type of action and finally, contents (H, (H), Hd, A, (A), Ad, (Ad), Na, Art, Ay, Bl, Cg, Ex, Fi and NC), indicating the elements present in the response of the subject, such as human and animal figures, for example. Procedures Data collection. The applications were made individually in places made available by the collection fields. All subjects responded to the Wartegg and the Rorschach Tests with an average total duration of 40 minutes. Data analysis. For the analyses, Pearson’s correlation was initiated, followed by Student’s t test in order to identify the differences between the variables for the two groups, followed by Cohen’s d test in order to verify the size of the effect of the expressive differences for each one of them, in the common variables between the techniques. Finally, hierarchical logistic regression was used to verify the incremental validity between the instruments. The Statistical Package for Social Sciences (SPSS) in version 21 was used for the analysis. Ethical Considerations The project was approved by the Research Ethics Committee of the Universidade São Francisco (Opinion No. 408,154, CAAE 20552313.6.0000.5514). After this approval, the subjects were invited to participate in the study by signing the Free and Informed Consent Term (FICT). According to resolution 196/96 of the National Health Council, the participant himself may sign the FICT, except in situations where it presents reduced capacity for self-determination. In these cases, a legal representative has been requested to sign the FICT. Results For the analyses carried out in order to achieve the objectives of this study, a set of variables was selected, namely formal quality, movement and contents, as described in the method. For the formal quality variables (FQ) besides using them separately, the Rorschach test FQo and FQu codes were added, because in the Wartegg Test there is still no frequency data referring to this variable, presenting only FQu. Pearson’s correlation between the variables was then employed in order to observe similarities between them. The results can be seen in Table 1. Table 1 Pearson’s correlation between the Rorschach and Wartegg Tests Wartegg H (H) Hd (Hd) A Ad (Ad) An Art Ay Rorschach H 0.60** -0.11 0.12 0.01 0.02 0.08 -0.08 -0.08 -0.08 0.08 (H) -0.04 -0.05 -0.09 0.13 0.36* 0.04 -0.15 -0.02 -0.15 0.11 Hd 0.2 -0.15 0.23 -0.09 -0.07 0.04 -0.01 -0.11 -0.01 0.1 (Hd) -0.04 -0.1 0.08 -0.18 -0.2 0.01 -0.1 0.21 -0.1 0.01 A 0.04 0.09 0.06 0.07 0.50** -0.12 0.01 -0.27 0.01 -0.01 (A) -0.07 0.19 0.09 0.12 0.40* -0.03 -0.11 -0.11 -0.11 0.29 Ad -0.11 -0.13 0.14 -0.2 -0.16 -0.07 0.44** 0.25 0.44** 0.04 (Ad) -0.09 -0.08 -0.03 0.33* -0.16 0.22 -0.05 -0.05 -0.05 -0.1 An -0.05 -0.12 0.06 -0.01 -0.01 -0.08 -0.12 0.26 -0.12 -0.13 Art 0 -0.13 -0.15 0.09 -0.03 -0.17 -0.1 -0.1 -0.1 0.06 Ay -0.02 -0.06 -0.09 0.06 -0.12 0 -0.06 -0.06 -0.06 -0.11 Bl -0.08 -0.06 -0.18 -0.12 -0.14 0.28 -0.05 -0.05 -0.05 -0.08 Cg 0.06 -0.12 0.16 -0.09 0.39* -0.03 0.03 0.18 0.03 0.23 Ex -0.04 -0.04 -0.1 -0.07 -0.08 -0.05 -0.03 -0.03 -0.03 -0.05 Fi 0.18 -0.11 -0.02 0.08 -0.16 0.3 -0.08 -0.08 -0.08 0 NC -0.07 0.05 -0.31 0.03 0.12 0.24 -0.14 -0.14 -0.14 -0.07 FQu 0.12 0 0.22 0.04 0.26 -0.05 -0.01 0.25 -0.01 -0.01 FQ- 0.07 0.01 -0.08 -0.2 0.19 0.07 -0.17 -0.17 -0.17 0.28 FQn -0.08 -0.03 -0.23 -0.03 -0.13 -0.08 -0.06 -0.06 -0.06 -0.04 FQo 0.02 -0.07 0.21 0.19 -0.04 0.09 0.07 -0.16 0.07 -0.12 FQo/u -0.07 -0.08 0.26 0.23 0.05 0.02 0.01 0.14 0.01 -0.11 M 0.45** -0.14 0.27 -0.16 0.57** 0.02 -0.2 0.01 -0.2 0.19 FM 0.21 -0.1 0.03 0 0.49** 0 0.04 -0.21 0.04 0.23 m 0.05 0.05 -0.16 0.15 0.23 0.14 0.27 -0.12 0.27 -0.21 a 0.32* -0.06 -0.01 -0.01 0.59** 0.15 -0.12 -0.16 -0.12 0.22 p 0.16 -0.19 0.40* -0.14 0.18 -0.31* 0.24 0.01 0.24 0.02 Wartegg Cg Fi NC FQu FQ- M FM m a p Rorschach H 0.07 -0.11 0 0.08 -0.05 0.32* 0.23 0.02 0.31 -0.04 (H) -0.22 -0.13 0.15 -0.09 0.1 0.13 0.22 0 0.07 0.13 Hd 0.28 -0.16 -0.02 -0.08 0.06 -0.02 -0.08 0.13 0.1 -0.09 (Hd) 0.08 0.08 -0.02 0.13 -0.18 -0.09 -0.05 0.25 0.13 -0.1 A -0.05 0.12 -0.18 -0.03 0.04 0.33* 0.27 -0.01 0.14 0.27 (A) 0.04 -0.16 -0.14 -0.19 0.22 -0.03 0.06 -0.2 -0.07 -0.09 Ad 0.15 -0.06 -0.02 0.21 -0.22 -0.18 0.02 0.2 0.12 -0.2 (Ad) -0.08 -0.08 -0.01 -0.01 0.02 0.05 -0.11 -0.03 -0.07 0.1 An -0.18 0.04 0.14 0.26 -0.23 -0.01 -0.06 -0.02 -0.11 0.13 Art 0 0 0.08 0.25 -0.3 -0.14 0 0.1 0.02 -0.08 Ay -0.09 0.05 0.16 -0.1 0.13 -0.01 -0.13 0.01 -0.07 0.04 Bl -0.07 -0.07 0.25 -0.34* 0.35* -0.17 -0.1 0.02 -0.04 -0.16 Cg 0.05 -0.06 -0.01 0.02 0.01 0.06 0.07 -0.05 0.01 0.04 Ex -0.04 -0.04 0.17 0.11 -0.11 -0.1 -0.05 -0.08 -0.07 -0.09 Fi -0.12 -0.12 0.01 -0.24 0.22 0.16 -0.04 -0.06 0.03 0.05 NC -0.1 -0.01 0.09 -0.44** 0.45** -0.02 -0.15 -0.09 -0.13 0.03 FQu 0.22 0.13 -0.25 0.26 -0.24 0.28 0.05 0.18 0.08 0.40* FQ- -0.04 -0.19 0.18 -0.38* 0.41** -0.28 0.09 -0.18 -0.06 -0.36* FQn -0.09 -0.05 0.29 -0.1 0.12 -0.17 -0.1 -0.17 -0.17 -0.13 FQo -0.07 0.15 -0.15 0.25 -0.27 0.46** 0.03 0.16 0.23 0.32* Fqo/u -0.05 0.27 -0.23 0.3 -0.31 0.47** 0 0.12 0.15 0.40** M 0.12 -0.09 -0.05 0.03 0 0.34* 0.39* -0.02 0.31* 0.02 FM -0.08 -0.03 0.03 0.08 -0.05 0.45** 0.59** 0.04 0.41** 0.14 m -0.17 0.11 -0.16 -0.3 0.3 0.2 -0.04 0.09 0.11 0.12 a -0.03 -0.17 -0.02 -0.08 0.11 0.34* 0.50** -0.03 -0.3 0.11 p 0.02 0.33* -0.08 0.26 -0.25 0.49** 0.2 0.12 0.41** 0.07 Note. ** Significant correlation at 0.01 level; * Significant correlation at 0.05 level. In Table 1 it is possible to observe a correlation of strong magnitude (0.60), 21 correlations ranging from 0.40 to 0.59 and 17 from 0.31 to 0.39. The greatest correlation found was verified between the H contents of the two techniques, signaling similarity in their variance. It is also possible to observe correlations between the content codes of both techniques comprising (Hd), A, Ad, (Ad) and Art from Wartegg and (H), A, Ad, (Ad), Cg from the Rorschach test varying between 0.33 and 0.50. Another set of correlations were observed between the movement indicators ranging from 0.31 to 0.59 with this value of greater magnitude being verified between the FM codes of the two techniques. Other correlations can still be observed, but in general, indicate covariances between the indicators of the two tests. Then the Student’s t test was applied in order to verify the differences between the variables for the two groups, as well as Cohen’s d, in order to verify the size of the effect of the differences between the groups. These indexes can be seen in Table 2. Table 2 Descriptive statistics, Student’s t test and Cohen’s d test for groups with schizophrenia (Eqz) and normative (Nor) for Rorschach and Wartegg Tests Rorschach Wartegg Var. Group M SD N t p d Var. Group M SD N t p d H EQ 1.4 1.27 20 -1.102 0.277 0.35 H EQ 0.05 0.22 20 -0.831 0.411 0.26 N 1.85 1.31 20 N 0.15 0.49 20 (H) EQ 1.05 1.32 20 -0.623 0.537 0.20 (H) EQ 0.45 1.57 20 1.127 0.267 -0.36 N 1.3 1.22 20 N 0.05 0.22 20 Hd EQ 1.2 1.61 20 0.4 0.692 -0.13 Hd EQ 0.45 0.76 20 -0.873 0.388 0.28 N 1 1.56 20 N 0.7 1.03 20 (Hd) EQ 0.4 0.60 20 0.9 0.374 -0.28 (Hd) EQ 0.1 0.45 20 -1.515 0.138 0.48 N 0.25 0.44 20 N 0.35 0.59 20 A EQ 6.95 3.71 20 -1.501 0.142 0.47 A EQ 0.2 0.52 20 -0.326 0.746 0.10 N 8.55 3.00 20 N 0.25 0.44 20 (A) EQ 0.4 0.60 20 0 1 0.00 (A) EQ 0 0.00 20 - - - N 0.4 0.60 20 N 0 0.00 20 Ad EQ 1.4 1.79 20 -0.094 0.926 0.03 Ad EQ 0.15 0.37 20 1 0.324 -0.58 N 1.45 1.57 20 N 0 0.00 20 (Ad) EQ 0.1 0.31 20 0 1 0.00 (Ad) EQ 0.05 0.22 20 1 0.324 -0.32 N 0.1 0.31 20 N 0 0.00 20 An EQ 1.45 1.93 20 -0.445 0.659 0.14 An EQ 0.05 0.22 20 1 0.324 -0.32 N 1.75 2.31 20 N 0 0.00 20 Art EQ 0.45 1.00 20 -0.369 0.714 0.12 Art EQ 0.05 0.22 20 1 0.324 -0.32 N 0.55 0.69 20 N 0 0.00 20 Ay EQ 0.45 1.10 20 0.953 0.347 -0.30 Ay EQ 0 0.00 20 -1.831 0.075 0.58 N 0.2 0.41 20 N 0.15 0.37 20 Bl EQ 0.15 0.37 20 1.831 0.075 -0.58 Bl EQ 0 0.00 20 - - - N 0 0.00 20 N 0 0.00 20 Cg EQ 0.65 1.27 20 -0.716 0.478 0.23 Cg EQ 0.05 0.22 20 0 1 0.00 N 0.9 0.91 20 N 0.05 0.22 20 Ex EQ 0 0.00 20 -1 0.324 0.32 Ex EQ 0 0.00 20 - - - N 0.05 0.22 20 N 0 0.00 20 Fi EQ 0.3 0.57 20 -0.238 0.813 0.08 Fi EQ 0.05 0.22 20 0 1 0.00 N 0.35 0.75 20 N 0.05 0.22 20 Sx EQ 0 0.00 20 - - - Sx EQ 0 0.00 20 - - - N 0 0.00 20 N 0 0.00 20 NC EQ 5.85 4.68 20 1.15 0.257 -0.36 NC EQ 6.1 2.00 20 -0.673 0.505 0.21 N 4.5 2.37 20 N 6.45 1.19 20 FQu EQ 5.2 2.35 20 -2.37 0.023 0.75 FQu EQ 4.15 3.53 20 -3.974 0.001 1.20 N 7 2.45 20 N 7.45 1.64 20 FQ- EQ 4.2 2.33 20 2.251 0.03 -0.71 FQ- EQ 3.65 3.63 20 3.56 0.001 -1.13 N 2.6 2.16 20 N 0.5 1.57 20 M EQ 2.35 1.93 20 -1.41 0.167 0.45 M EQ 0.1 0.45 20 -3.126 0.003 0.99 N 3.35 2.52 20 N 0.7 0.73 20 FM EQ 2.35 2.01 20 -2.631 0.012 0.83 FM EQ 0 0.00 20 -2.179 0.036 0.69 N 4.3 2.64 20 N 0.2 0.41 20 m EQ 0.65 0.88 20 0.372 0.712 -0.12 m EQ 0.35 0.67 20 0.515 0.609 -0.16 N 0.55 0.83 20 N 0.25 0.55 20 a EQ 3.8 3.94 20 -1.835 0.074 0.58 a EQ 0.45 0.89 20 -0.15 0.881 0.05 N 6.1 3.99 20 N 0.5 1.19 20 p EQ 1.55 1.61 20 -1.558 0.128 0.49 p EQ 0 0.00 20 -4.333 0 1.37 N 2.25 1.21 20 N 0.65 0.67 20 The results of Table 2 are favorable to the objectives of this study. It is possible to observe that the variables related to formal quality (FQu and FQ-) in both techniques presented significant differences between the groups. In addition, the averages for the M and FM of the Wartegg and the Rorschach Tests, were also verified as different among the groups. Finally, linear regression analysis was used to verify the incremental validity between the two techniques. For this analysis we used the same variables indicated above, but grouped in broad indicators, namely formal quality, movement and contents. The analysis was performed separately for each of the variables. The results can be seen in Table 3. Table 3 Ranking table and predicted percentage for regression analysis Formal quality Predicted group % correct Observed EQ Nor group EQ 15 5 75 Nor 5 15 75 % global 75% Movement Predicted grupo % correct Observed EQ Nor group EQ 20 0 100 Nor 1 19 95 % global 98% Content Predicted group % correct Observed EQ Nor group EQ 20 0 100 Nor 0 20 100 % global 100% In Table 3 it is possible to verify good predictive indices for the 3 variables. For the formal quality it is observed that the values for true positives and negatives were equal, reaching 75% of the cases (R2 Cox & Snell = 0.416 and R2 Nagelkerke = 0.554). For movement, there are 20 true positives and 19 true negatives, predicting 98% of the cases (R2 Cox & Snell = 0.725 and R2 Nagelkerke = 0.967), Finally, for content it is possible to observe that all true positives and negatives were found, predicting 100% of the cases (R2 Cox & Snell = 0.750 and R2 Nagelkerke = 1). Discussion Initially, it is possible to verify that the correlations observed in Table 1 corroborate with the objective of this study going against the initial hypotheses about their interpretations. The correlation between the H codes in the two techniques indicates good similarity between them, that is, as the person demonstrates that he is able to perceive and perceive the other in a realistic and multifaceted way in the Rorschach test, tends to demonstrate the same in Wartegg (r = 60). Moore, Viglione, Rosenfarb, Patterson and Mausbach (2013) signal that this is an indicator of mental representations linked to interpersonal relations having observed correlation between the Rorschach (R-PAS) test and Social Skills Performance Assessment (SSPA). Still considering indicators linked to the focus of attention of the individual [(Hd), A, Ad, (Ad) and Art of Wartegg and (H), A, Ad, (Ad), Cg of Rorschach], the correlations observed proved positive for the scope of this study. In this sense, there is similarity in the perception of details or even more simplified perception, not encompassing the whole, between the response process of the two tests. Exner and Sendin (1999), Mihura, Meyer, Dumitrascu and Bombel (2013) and Moore et al. (2013) point out that beyond the relationship, the focus of interest of the subject, can signal interest in the other, indicating questions about personal relationships. These results show good integration between the techniques, informing that to some degree, these indicators seem to access similar latent abilities, verified by the covariances observed. The correlations observed between the movement indicators of the two techniques signal some level of participation of the perception/perception mechanisms linked to primary needs, especially for the variable FM (Mihura et al., 2013). Therefore, it is possible to understand that, in general, responses containing movement are linked to internal inferences, since effectively, the stains do not present movement. Thus, the effort to foresee an action or emotion in the Rorschach test, is in line with the same indicator in the Wartegg Test. The same can be observed in formal quality variables. Several authors (Marques, Chaves, & Yazigi, 2012; Pianowski & Villemor-Amaral, 2010; Yazigi et al., 2016) also observed a relationship between formal quality and perceptual elements, usually with patients diagnosed with schizophrenia, this being a good indicator for the disorder. In this sense, observing the correlations between the formal quality codes, it is possible to notice that as the subject tends to distort his perception in the Rorschach test, providing poor responses to the contours of the stain, he also tends to complete the figures of the Wartegg Test without much reference to the initial stimulus of the frame, that is, with little integration in his/her perception between the initial stimulus and the final product. Other correlations can still be observed, but in general, they signal covariances between the indicators of the two tests. It is important to consider the differences between the nature of the stimuli of the two techniques, making it necessary other studies for further inferences about the intepretations for the Wartegg Test. However, this analysis points to the existence of covariance, without however presenting high correlations, indicating that part of the latent ability accessed is shared in some way, but there is still part of the variance that is not accessed. In this sense, the Student’s t test was applied in order to verify the differences and Cohen’s d to find the size of the effect of the differences between the groups evaluated by both techniques. These indexes can be seen in Table 2. The differences found in the variables linked to formal quality (FQu and FQ-) in both techniques were already expected for the Rorschach test (Pianowski, Meyer, & Villemor-Amaral, 2016; Pianowski & Villemor-Amaral, 2010; Vieira & Villemor-Amaral, 2015) and focuses on assigning the Wartegg Test the same property (Pacico, Hutz, Schneider, & Bandeira, 2015; Urbina, 2007), This presents even greater values for the Cohen d indicating greater impact on the variable. Alves, Quaglia, Bachett and Oliveira (2014) in a study investigating the depth perception in subjects with schizophrenia noted marked perceptual distortion, especially in subjects medicated over a period of less than four months, indicating distortion of the form observed. In another study, Amaro, Areco and Nascimento (2017) also found significant differences in a sample of 50 elderly people, corroborating the sensitivity of the variable for the assessment of perceptual elements. There was also a significant difference for movement codes (M and FM), which corroborates the findings of some authors (Exner & Sendin, 1999; Pianowski & Villemor-Amaral, 2010) in studies that verified the relevance of these variables in the study of schizophrenia, especially for human movement, predominantly associated with QF-, indicating rupture with reality and possible psychotic thinking, which may be related to this variable having been observed as statistically different in the t-test. Mihura et al. (2013) in a study on the indicators of the variables in the Rorschach test (R-PAS) it is emphasized that M is associated with mental abilities, including planning and empathy, which would justify the higher values of the normative group, since patients with schizophrenia may present deficits in social skills (Murta, 2005). These analyses allow us to verify that the two techniques share aspects in which they are able to verify, through their variables, that is, to infer types of psychological functioning from the criteria used in the responses of the subject, as proposed by Urbina (2007). Even sharing these possibilities, it is important to understand that part of the variance of latent abilities are not measured by one technique or another, due to the different formats of the items, mode of data collection and, consequently, access to the latent trace (Schmitt et al., 2015). In this sense, as proposed by Captain and Cardoso (2012) the use of different techniques may favor the completion of information. In view of this consideration, the observed results of linear regression indicate the importance of using techniques together in the use of psychological evaluation as indicated in the incremental validity (Capitão & Cardoso, 2012; Hunsley & Meyer, 2003). The levels of prediction evidenced by both techniques have proven effective in predicting schizophrenia. In the case of formal quality, together they predict correctly, 75% of the cases, this indicates that, designing first what the Rorschach test can predict, Wartegg is added for the rest of the data. Likewise, for movement and content, the hit percentage is high, reaching 100% in the second case. Thus, the incremental validity evidenced from the two instruments may prove useful for the clinical evaluation context, as some researchers have also observed for other instruments (Brackett & Mayer, 2003; Capitão & Cardoso, 2012; Franco & Villemor-Amaral, 2012). In this sense, it is possible to observe through the analyses that the instruments, although they show convergence as to the latent traits evaluated, their joint use can help in the intervening process adding great preventive power to the evaluation process. Therefore, it is possible to conclude that the use of the Wartegg Test can increase evidence of validity to the Rorschach test by enriching clinical decisions in psychological evaluation processes, signaling important indicators about the perceptual distortion of the assessed subjects. It is understood that the present study is an initial for the Pessotto system (2018a) for the Wartegg Test, conceiving the need to improve it, as well as the exploration of other variables. Another issue to be considered as a limitation may be the reduced number of subjects. Even this fact being observed in other studies with self-expression techniques (Scortegagna & Villemor-Amaral, 2013; Yazigi et al., 2013), it is important to enlarge the sample, as well as the evaluation in other contexts. Referências Alves, A., Quaglia, M. A. C., Bachett, L. S., & Oliveira, M. S. (2014). Percepção monocular da profundidade ou relevo na ilusão da máscara côncava na esquizofrenia [Monocular depth perception or relief in the hollow-face illusion in the schizophrenia]. Estudos de Psicologia (Natal), 19(1), 40-47. doi:10.1590/S1413-294X2014000100006 Alves A. Quaglia M. A. C. Bachett L. S. Oliveira M. S. 2014 Percepção monocular da profundidade ou relevo na ilusão da máscara côncava na esquizofrenia Monocular depth perception or relief in the hollow-face illusion in the schizophrenia Estudos de Psicologia Natal 19 1 40 47 10.1590/S1413-294X2014000100006 Amaro, T. A. C., Areco, K. N., & Nascimento, R. S. G. (2017). Avaliação dos aspectos da personalidade em pessoas idosas na cidade de São Paulo por meio do Rorschach Performance System (R-PAS) [Evaluation of personality aspects in elderly people in Sao Paulo city using Rorschach Performance System (R-PAS)]. Revista Kairós Gerontologia, 20(2), 221-228. doi:10.23925/2176-901X.2017v20i2p211-228 Amaro T. A. C. Areco K. N. Nascimento R. S. G. 2017 Avaliação dos aspectos da personalidade em pessoas idosas na cidade de São Paulo por meio do Rorschach Performance System (R-PAS) Evaluation of personality aspects in elderly people in Sao Paulo city using Rorschach Performance System (R-PAS) Revista Kairós Gerontologia 20 2 221 228 10.23925/2176-901X.2017v20i2p211-228 Andrei, F., Siegling, A. B., Aloe, A. M., Baldaro, B., & Petrides, K. V. (2016). The incremental validity of the Trait Emotional Intelligence Questionnaire (TEIQue): A systematic review and meta-analysis. Journal of Personality Assessment, 98(3), 261-276. doi:10.1080/00223891.2015.1084630 Andrei F. Siegling A. B. Aloe A. M. Baldaro B. Petrides K. V. 2016 The incremental validity of the Trait Emotional Intelligence Questionnaire (TEIQue): A systematic review and meta-analysis Journal of Personality Assessment 98 3 261 276 10.1080/00223891.2015.1084630 Brackett, M. A., & Mayer, J. D. (2003). Convergent, discriminant, and incremental validity of competing measures of emotional intelligence. Personality and Social Psychology Bulletin, 29(9), 1147-1158. doi:10.1177/0146167203254596 Brackett M. A. Mayer J. D. 2003 Convergent, discriminant, and incremental validity of competing measures of emotional intelligence Personality and Social Psychology Bulletin 29 9 1147 1158 10.1177/0146167203254596 Campos, R. C. (2013). Além dos números há uma pessoa: Sobre a utilização clínica de testes [Behind the numbers there is a person: About the clinical use of tests]. Avaliação Psicológica, 12(3), 291-298. Retrieved from http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712013000300003 Campos R. C. 2013 Além dos números há uma pessoa: Sobre a utilização clínica de testes Behind the numbers there is a person: About the clinical use of tests Avaliação Psicológica 12 3 291 298 http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712013000300003 Capitão, C. G., & Cardoso, L. M. (2012). Evidence of incremental validity between pfister test and human figure drawing. International Journal of Psychology and Behavioral Sciences, 2(4), 120-129. doi:10.5923/j.ijpbs.20120204.07 Capitão C. G. Cardoso L. M. 2012 Evidence of incremental validity between pfister test and human figure drawing International Journal of Psychology and Behavioral Sciences 2 4 120 129 10.5923/j.ijpbs.20120204.07 Crisi, A., & Dentale, F. (2016). The Wartegg Drawing Completion Test: Inter-rater agreement and criterion validity of three new scoring categories. International Journal of Psychology and Psychological Therapy, 16(1), 83-90. Crisi A. Dentale F. 2016 The Wartegg Drawing Completion Test: Inter-rater agreement and criterion validity of three new scoring categories International Journal of Psychology and Psychological Therapy 16 1 83 90 Cunha, J. A. (2000). Psicodiagnóstico-V [Psychodiagnosis-V]. Porto Alegre, RS: Artmed. Cunha J. A. 2000 Psicodiagnóstico-V Psychodiagnosis-V Porto Alegre, RS Artmed Engelman, D. H., Allyn, J. B., Crisi, A., Finn, S. E., Fischer, C. T., & Nakamura, N. (2016). “Why am I so stuck?”: A collaborative/therapeutic assessment case discussion. Journal of Personality Assessment, 98(4), 360-373. doi:10.1080/00223891.2015.1119154 Engelman D. H. Allyn J. B. Crisi A. Finn S. E. Fischer C. T. Nakamura N. 2016 “Why am I so stuck?”: A collaborative/therapeutic assessment case discussion Journal of Personality Assessment 98 4 360 373 10.1080/00223891.2015.1119154 Exner, J. E. Jr., & Sendín, C. (1999). Manual de interpretação do Rorschach: Para o sistema compreensivo [Rorschach interpretation manual: For the comprehensive system] (L. Y. Massuh, Trans.). São Paulo, SP: Casa do Psicólogo. Exner J. E. Jr. Sendín C. 1999 Manual de interpretação do Rorschach: Para o sistema compreensivo Rorschach interpretation manual: For the comprehensive system Massuh L. Y. São Paulo, SP Casa do Psicólogo Franco, R. R. C., & Villemor-Amaral, A. E. (2012). Validade incremental do Zulliger e do Pfister no contexto da toxicomania [Incremental validity of the Zulliger and Pfister in the contexto of drug addiction]. Psico-USF, 17(1), 73-83. doi:10.1590/S1413-82712012000100009 Franco R. R. C. Villemor-Amaral A. E. 2012 Validade incremental do Zulliger e do Pfister no contexto da toxicomania Incremental validity of the Zulliger and Pfister in the contexto of drug addiction Psico-USF 17 1 73 83 10.1590/S1413-82712012000100009 Haynes, S. N., & Lench, H. C. (2003). Incremental validity of new clinical assessment measures. Psychological Assessment, 15(4), 456-466. doi:10.1037/1040-3590.15.4.456 Haynes S. N. Lench H. C. 2003 Incremental validity of new clinical assessment measures Psychological Assessment 15 4 456 466 10.1037/1040-3590.15.4.456 Hunsley, J., & Meyer, G. J. (2003). The incremental validity of psychological testing and assessment: Conceptual, methodological, and statistical issues. Psychological Assessment, 15(4), 446-455. doi:10.1037/1040-3590.15.4.446 Hunsley J. Meyer G. J. 2003 The incremental validity of psychological testing and assessment: Conceptual, methodological, and statistical issues Psychological Assessment 15 4 446 455 10.1037/1040-3590.15.4.446 Marques, T. C., Chaves, A. C., & Yazigi, L. (2012). Estudo parcial da validação do Atlas do Rorschach Sistema Compreensivo em amostra de pacientes psiquiátricos de São Paulo. [Partial validation study of the Rorschach Comprehensive System atlas on a sample of psychiatric patients of São Paulo]. Psico-USF, 17(3), 417-416. doi:10.1590/S1413-82712012000300008 Marques T. C. Chaves A. C. Yazigi L. 2012 Estudo parcial da validação do Atlas do Rorschach Sistema Compreensivo em amostra de pacientes psiquiátricos de São Paulo Partial validation study of the Rorschach Comprehensive System atlas on a sample of psychiatric patients of São Paulo Psico-USF 17 3 417 416 10.1590/S1413-82712012000300008 Merino Soto, C. A. (2014). Validez incremental del Test Gestáltico de Bender Modicado, en niños que inician el primer grado [Incremental validity for the Modified Bender Gestalt Test in children commencing first grade]. Avances en Psicología Latinoamericana, 32(2), 275-286. doi:10.12804/apl32.2.2014.07 Merino Soto C. A. 2014 Validez incremental del Test Gestáltico de Bender Modicado, en niños que inician el primer grado Incremental validity for the Modified Bender Gestalt Test in children commencing first grade Avances en Psicología Latinoamericana 32 2 275 286 10.12804/apl32.2.2014.07 Meyer, G. J., Viglione, D. J., Mihura, J. L., Erard, R. E., & Erdberg, P. (2011). Rorschach Performance Assessment System. Toledo, OH: Rorschach Performance Assessment System/LLC. Meyer G. J. Viglione D. J. Mihura J. L. Erard R. E. Erdberg P. 2011 Rorschach Performance Assessment System Toledo, OH Rorschach Performance Assessment System/LLC Mihura, J. L., & Meyer, G. J. (2018). Using the Rorschach Performance Assessment System (R-PAS). New York, NY: Guilford. Mihura J. L. Meyer G. J. 2018 Using the Rorschach Performance Assessment System (R-PAS) New York, NY Guilford Mihura, J. L., Meyer, G. J., Dumitrascu, N., & Bombel, G. (2013). The validity of individual Rorschach variables: Systematic reviews and meta-analyses of the comprehensive system. Psychological Bulletin, 139(3), 548-605. doi:10.1037/a0029406 Mihura J. L. Meyer G. J. Dumitrascu N. Bombel G. 2013 The validity of individual Rorschach variables: Systematic reviews and meta-analyses of the comprehensive system Psychological Bulletin 139 3 548 605 10.1037/a0029406 Moore, R. C., Viglione, D. J., Rosenfarb, I. S., Patterson, T. L., & Mausbach, B. T. (2013). Rorschach measures of cognition relate to everyday and social functioning in schizophrenia. Psychological Assessment, 25(1), 253-263. doi:10.1037/a0030546 Moore R. C. Viglione D. J. Rosenfarb I. S. Patterson T. L. Mausbach B. T. 2013 Rorschach measures of cognition relate to everyday and social functioning in schizophrenia Psychological Assessment 25 1 253 263 10.1037/a0030546 Murta, S. G. (2005). Aplicações do treinamento em habilidades sociais: Análise da produção nacional [Applications of training in social skills: Analysis of national production]. Psicologia: Reflexão e Crítica, 18(2), 283-291. doi:10.1590/S0102-79722005000200017 Murta S. G. 2005 Aplicações do treinamento em habilidades sociais: Análise da produção nacional Applications of training in social skills: Analysis of national production Psicologia: Reflexão e Crítica 18 2 283 291 10.1590/S0102-79722005000200017 Noronha, A. P. P., & Alchieri, J. C. (2004). Knowledge in psychological assessment. Estudos de Psicologia (Campinas), 21(1), 43-52. doi:10.1590/S0103-166X2004000100004 Noronha A. P. P. Alchieri J. C. 2004 Knowledge in psychological assessment Estudos de Psicologia Campinas 21 1 43 52 10.1590/S0103-166X2004000100004 Ng, T. W. H. (2015). The incremental validity of organizational commitment, organizational trust, and organizational identification. Journal of Vocational Behavior, 88, 154-163. doi:10.1016/j.jvb.2015.03.003 Ng T. W. H. 2015 The incremental validity of organizational commitment, organizational trust, and organizational identification Journal of Vocational Behavior 88 154 163 10.1016/j.jvb.2015.03.003 Ng, T. W. H., & Feldman, D. C. (2015). Ethical leadership: Meta-analytic evidence of criterion-related and incremental validity. Journal of Applied Psychology, 100(3), 948-965. doi:10.1037/a0038246 Ng T. W. H. Feldman D. C. 2015 Ethical leadership: Meta-analytic evidence of criterion-related and incremental validity Journal of Applied Psychology 100 3 948 965 10.1037/a0038246 Pacico, J. C., Hutz, C. S., Schneider, A. M. A., & Bandeira, D. R. (2015). Validade [Validity]. In C. S. Hutz, D. R. Bandeira, & C. M. Trentini (Orgs.), Psicometria [Psychometry] (pp. 71-84). Porto Alegre, RS: Artmed . Pacico J. C. Hutz C. S. Schneider A. M. A. Bandeira D. R. 2015 Validade Validity Hutz C. S. Bandeira D. R. Trentini C. M. Psicometria Psychometry 71 84 Porto Alegre, RS Artmed Pessotto, F., & Primi, R. (2018a). A study of Coding Systems for the Wartegg Test and their relations with the Rorschach (R- PAS). Trends in Psychology, 26(4), 1847-1860. doi:10.9788/TP2018.4-06pt Pessotto F. Primi R. 2018 A study of Coding Systems for the Wartegg Test and their relations with the Rorschach (R- PAS) Trends in Psychology 26 4 1847 1860 10.9788/TP2018.4-06pt Pessotto, F., & Primi, R. (2018b). Evidências de validade de critério para o teste de Wartegg [Evidence of criterion validity for the Wartegg Test]. Avaliação Psicológica, 17(3), 279-291. doi:10.15689/ap.2018.1703.13941.01 Pessotto F. Primi R. 2018 Evidências de validade de critério para o teste de Wartegg Evidence of criterion validity for the Wartegg Test Avaliação Psicológica 17 3 279 291 10.15689/ap.2018.1703.13941.01 Pessotto, F., & Primi, R. (2018c). Evidências de validade convergente para o Teste de Wartegg [Convergent validity evidences for Wartegg Test]. Psico, 49(1), 73-80. doi:10.15448/1980-8623.2018.1.26935 Pessotto F. Primi R. 2018 Evidências de validade convergente para o Teste de Wartegg Convergent validity evidences for Wartegg Test Psico 49 1 73 80 10.15448/1980-8623.2018.1.26935 Pianowski, G., Meyer, G. J., & Villemor-Amaral, A. E. (2016). The impact of R-optimized administration modeling procedures on Brazilian normative reference values for Rorschach Scores. Journal of Personality Assessment, 98(4), 408-418. doi:10.1080/00223891.2016.1148701 Pianowski G. Meyer G. J. Villemor-Amaral A. E. 2016 The impact of R-optimized administration modeling procedures on Brazilian normative reference values for Rorschach Scores Journal of Personality Assessment 98 4 408 418 10.1080/00223891.2016.1148701 Pianowski, G., Meyer, G. J., Villemor-Amaral, A. E., Zuanazzi, A. C., & Nascimento, R. S. G. F. (2019). Does the Rorschach Performance Assessment System (R-PAS) differ from the Comprehensive System (CS) on variables relevant to interpretation? Journal of Personality Assessment, 1-16. Advance online publication. doi:10.1080/00223891.2019.1677678 Pianowski G. Meyer G. J. Villemor-Amaral A. E. Zuanazzi A. C. Nascimento R. S. G. F. 2019 Does the Rorschach Performance Assessment System (R-PAS) differ from the Comprehensive System (CS) on variables relevant to interpretation? Journal of Personality Assessment 1 16 Advance online publication 10.1080/00223891.2019.1677678 Pianowski, G., & Villemor-Amaral, A. E. (2010). Localização e qualidade formal do Rorschach-SC no Brasil: Validade com não-pacientes [Location and formal quality of the Rorschach-SC in Brazil: Validity with non-patient sample]. Psico-USF, 15(3), 333-343. doi:10.1590/S1413-82712010000300007 Pianowski G. Villemor-Amaral A. E. 2010 Localização e qualidade formal do Rorschach-SC no Brasil: Validade com não-pacientes Location and formal quality of the Rorschach-SC in Brazil: Validity with non-patient sample Psico-USF 15 3 333 343 10.1590/S1413-82712010000300007 Rizzo, A., Della Villa, L., & Crisi, A. (2015). Can the problematic Internet use evolve in a pre-psychotic state? A single case study with the Wartegg. Computers in Human Behavior, 51(A), 532-538. doi:10.1016/j.chb.2015.04.063 Rizzo A. Della Villa L. Crisi A. 2015 Can the problematic Internet use evolve in a pre-psychotic state? A single case study with the Wartegg Computers in Human Behavior 51 A 532 538 10.1016/j.chb.2015.04.063 Schmitt, M., Hofmann, W., Gschwendner, T., Gerstenberg, F., & Zinkernagel, A. (2015). A model of moderated convergence between direct, indirect, and behavioral measures of personality traits. In T. M. Ortner & F. J. R. van de Vijver (Eds.), Behavior-based assessment in psychology: Going beyond self-report in the personality, affective, motivation, and social domains: Vol. 1. Psychological assessment - Science and practice (pp. 29-44). Göttingen, Germany: Hogrefe. Schmitt M. Hofmann W. Gschwendner T. Gerstenberg F. Zinkernagel A. 2015 A model of moderated convergence between direct, indirect, and behavioral measures of personality traits Ortner T. M. van de Vijver R. Behavior-based assessment in psychology: Going beyond self-report in the personality, affective, motivation, and social domains: Vol. 1. Psychological assessment - Science and practice 29 44 Göttingen, Germany Hogrefe Scortegagna, S. A., & Villemor-Amaral, A. E. (2013). Rorschach e pedofilia: A fidedignidade no Teste-Reteste [Rorschach and pedophilia: A reliability at Test-Retest]. Psico, 44(4), 508-517. Retrieved from https://revistaseletronicas.pucrs.br/ojs/index.php/revistapsico/article/view/11651 Scortegagna S. A. Villemor-Amaral A. E. 2013 Rorschach e pedofilia: A fidedignidade no Teste-Reteste Rorschach and pedophilia: A reliability at Test-Retest Psico 44 4 508 517 https://revistaseletronicas.pucrs.br/ojs/index.php/revistapsico/article/view/11651 Silveira, L. B., Oliveira, S. E. S., & Bandeira, D. R. (2018). Validity evidences of the Inventory of Personality Organization - Brasil (IPO-Br): Its relation with the Five-Factor Model of Personality. Trends in Psychology , 26(4), 1875-1889. doi:10.9788/TP2018.4-07pt Silveira L. B. Oliveira S. E. S. Bandeira D. R. 2018 Validity evidences of the Inventory of Personality Organization - Brasil (IPO-Br): Its relation with the Five-Factor Model of Personality Trends in Psychology 26 4 1875 1889 10.9788/TP2018.4-07pt Urbina, S. (2007). Fundamentos da testagem psicológica [Fundamentals of psychological testing] (C. Dornelles, Trans.). Porto Alegre, RS: Artmed . Urbina S. 2007 Fundamentos da testagem psicológica Fundamentals of psychological testing Dornelles C. Porto Alegre, RS Artmed Vari, C., Velotti, P., Crisi, A., Carlesimo, S., Richetta, A. G., & Zavattini, G. C. (2017). Investigating personality and psychopathology in patients with psoriasis. Rorschachiana, 38(2), 87-107. doi:10.1027/1192-5604/a000092 Vari C. Velotti P. Crisi A. Carlesimo S. Richetta A. G. Zavattini G. C. 2017 Investigating personality and psychopathology in patients with psoriasis Rorschachiana 38 2 87 107 10.1027/1192-5604/a000092 Vieira, P. G. (2017). Avaliação terapêutica: Uma nova abordagem para o psicodiagnóstico [Therapeutic assessment: A new approach to psychodiagnosis]. In P. Landin, R. Almeida, & A. Roma (Eds.), Avaliação psicológica: A atualidade da prática profissional (Vol. 1, pp. 21-31). São Paulo, SP: Leader. Vieira P. G. 2017 Avaliação terapêutica: Uma nova abordagem para o psicodiagnóstico Therapeutic assessment: A new approach to psychodiagnosis Landin P. Almeida R. Roma A. Avaliação psicológica: A atualidade da prática profissional 1 21 31 São Paulo, SP Leader Vieira, P. G., & Villemor-Amaral, A. E. (2015). Evidências de validade do Rorschach Performance Assessment System no diagnóstico da esquizofrenia [Evidence of validity of Rorschach Performance Assessment System in the diagnosis of schizophrenia]. Avaliação Psicológica, 14(1), 53-62. Retrieved from http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712015000100007 Vieira P. G. Villemor-Amaral A. E. 2015 Evidências de validade do Rorschach Performance Assessment System no diagnóstico da esquizofrenia Evidence of validity of Rorschach Performance Assessment System in the diagnosis of schizophrenia Avaliação Psicológica 14 1 53 62 http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712015000100007 Villemor-Amaral, A. E., & Pasqualini-Casado, L. (2006). A cientificidade das técnicas projetivas em debate [The scientific status of projective techniques in discussion]. Psico-USF, 11(2), 185-193. doi:10.1590/S1413-82712006000200007 Villemor-Amaral A. E. Pasqualini-Casado L. 2006 A cientificidade das técnicas projetivas em debate The scientific status of projective techniques in discussion Psico-USF 11 2 185 193 10.1590/S1413-82712006000200007 Yazigi, L., Areco, K. C. N., Fiore, M. L. M., Barros, R., Lerman, T. G., Abela, R. K., … Marques, T. C. (2013). Avaliação de processo psicoterápico por meio do Rorschach Performance Assessment System [Evaluation of the psychotherapeutic process through the Rorschach Performance Assessment System]. Fragmentos de Cultura, 23(4), 515-526. doi:10.18224/frag.v23i4.2983 Yazigi L. Areco K. C. N. Fiore M. L. M. Barros R. Lerman T. G. Abela R. K. Marques T. C. 2013 Avaliação de processo psicoterápico por meio do Rorschach Performance Assessment System Evaluation of the psychotherapeutic process through the Rorschach Performance Assessment System Fragmentos de Cultura 23 4 515 526 10.18224/frag.v23i4.2983 Yazigi, L., Semer, N. L., Fiore, M. L. M., Abela, R. K., Lerman, T. G., & Marques, T. C. (2016). Form quality in Rorschach Comprehensive System and R-PAS: Sample of psychiatric cases. Paidéia (Ribeirão Preto), 26(63), 53-61. doi:10.1590/1982-43272663201607 Yazigi L. Semer N. L. Fiore M. L. M. Abela R. K. Lerman T. G. Marques T. C. 2016 Form quality in Rorschach Comprehensive System and R-PAS: Sample of psychiatric cases Paidéia Ribeirão Preto 26 63 53 61 10.1590/1982-43272663201607 Yuana, R. A., Harjunowibowo, D., Karyanta, N. A., & Budiyanto, C. W. (2018). Data similarity filtering of Wartegg Personality Test result using cosine-similarity. International Journal of Recent Contributions from Engineering, Science & IT (IJES), 6(3), 19-28. Retrieved from https://online-journals.org/index.php/i-jes/article/view/9413/5257 Yuana R. A. Harjunowibowo D. Karyanta N. A. Budiyanto C. W. 2018 Data similarity filtering of Wartegg Personality Test result using cosine-similarity International Journal of Recent Contributions from Engineering, Science & IT (IJES) 6 3 19 28 https://online-journals.org/index.php/i-jes/article/view/9413/5257 5 Article derived from the PhD thesis of the first author under the supervision of the second, defended in 2015, in the Post graduate Program in Psychology with emphasis on Psychological Assessment from the Universidade São Francisco.