Adaptation and psychometric properties of the Anxiety Symptoms Questionnaire for Brazil

Adaptação e propriedades psicométricas do Questionário de Sintomas de Ansiedade para o Brasil

Abstract

Objective

The present study aimed to adapt the Anxiety Symptoms Questionnaire and analyze its psychometric properties for the Brazilian population.

Method

The adaptation was conducted in six stages: translation, synthesis, back translation, expert analysis, evaluation by the target population, and a pilot study. A total of 441 adults (M = 35.36; SD = 11.08 years) were interviewed, with 301 classified as healthy, 105 as having anxiety, and 35 as having depression.

Results

The Anxiety Symptoms Questionnaire results demonstrated evidence of content validity, exhibiting high internal consistency (Cronbach’s alpha = 0.975 and McDonald’s omega = 0.976). Moreover, the questionnaire showed evidence of criterion validity by effectively distinguishing between groups with and without anxiety (F(2.438) = 22.647; p < 0.01), as well as convergent validity with the structured clinical interview for DSM-5 (ρ = 0.70, p < 0.01), and correlations with anxiety (ρ = 0.62, p < 0.01), depression (ρ = 0.64, p < 0.01), and stress (ρ = 0.70, p < 0.01) scores from another instrument.

Conclusion

The Anxiety Symptoms Questionnaire is valid and reliable for use by healthcare professionals in the Brazilian population.

Keywords
Anxiety; Evaluation; Psychopathology; Test reliability; Test validity

Resumo

Objetivo

O presente estudo teve como objetivo adaptar o Questionário de Sintomas de Ansiedade e analisar suas propriedades psicométricas para a população brasileira.

Método

A adaptação foi realizada em seis etapas: tradução, síntese, retrotradução, análise de especialistas, avaliação da população alvo e estudo piloto, foram entrevistados 441 adultos (M = 35,36; DP = 11,08 anos): 301 saudáveis, 105 com ansiedade e 35 com depressão.

Resultados

Os resultados do Questionário de Sintomas de Ansiedade demonstraram evidências de validade de conteúdo, alta consistência interna (alfa de Cronbach = 0,975 e ômega de McDonald = 0,976), evidências de validade de critério ao diferenciar grupos com e sem ansiedade (F(2,438) = 22,647; p = < 0,01 ), validade convergente com a entrevista estruturada do DSM-5 (ρ = 0,70 e p < 0,01) e com o escore de ansiedade (ρ = 0,62 e p < 0,01), depressão (ρ = 0,64 e p < 0,01) e estresse (ρ = 0,70 e p < 0,01) de outro instrumento.

Conclusão

O Questionário de Sintomas de Ansiedade é válido e confiável para ser aplicado por profissionais da saúde na população brasileira.

Palavras-chave
Ansiedade; Avaliação; Psicopatologia; Fidedignidade do teste; Validade do teste

Anxiety is an adaptive and universal human reaction to stressful situations. However, as outlined in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders − DSM-5 (American Psychiatric Association [APA], 2014), anxiety disorders are distinguished from adaptive fears, characterized by an emotional response to a real or perceived imminent threat, and anxiety, marked by the anticipation of future threats. The key differentiator lies in the excessive or prolonged duration of these disorders, relative to the individual’s phase of human development (APA, 2014; Craske & Stein, 2016). Such symptoms significantly interfere with individuals’ lives, leading to difficulties in interpersonal relationships, daily activities, and other aspects of life (APA, 2014; DeSousa et al., 2013; Vanzeler, 2020).

According to the latest survey conducted by the World Health Organization, approximately 301 million people worldwide suffered from an anxiety disorder (World Health Organization [WHO], 2022). Brazil leads as the country with the highest prevalence, with this diagnosis present in 9.3% of the population (WHO, 2017). Furthermore, according to a systematic review and meta-analysis, rates almost four times higher are found in the university population (Demenech et al., 2021).

In 2020, with the declared pandemic due to the emergence of the coronavirus (COVID-19, Coronavirus Disease 2019), a new reality was imposed on people’s lives – that of social distancing and isolation. In this condition, it was expected that the population’s mental health would be impacted. A study conducted by Goulart et al. (2021) with 1996 Brazilians, between May and July 2020, indicated that 81.9% of the sample showed symptoms of moderate or severe anxiety. Further, there was a strong positive correlation between a long period of social distancing and symptoms of anxiety and depression. Barros et al. (2020) interviewed 45,161 Brazilians between April and May 2020, and 52.6% revealed that they often felt anxious or nervous. In this study, young adults (69.5%) demonstrated the highest prevalence of these symptoms compared to older adults (31.7%).

The most common anxiety disorders include Agoraphobia, Panic Disorder, Separation Anxiety Disorder, Social Phobia, and Generalized Anxiety Disorder (APA, 2014). The diagnoses of these disorders are linked to the frequency, intensity, and duration of the symptoms, degree of distress, and impairments in adaptive functioning (APA, 2014; Baker et al., 2019; Craske & Stein, 2016). The diagnostic assessment is complex and nuanced, given that symptoms such as fear, anxiety, excessive worry, irritability, difficulty concentrating, muscle tension, sleep disturbance, and restlessness occur in a variety of settings (Baker et al., 2019). When left untreated, anxiety disorders tend to become chronic, making it essential for healthcare professionals to have appropriate and valid instruments on hand to contribute to the assessment and propose the best treatments. Moreover, when the diagnosis is made early, there is a higher likelihood of the patient having good prognoses (Craske & Stein, 2016; DeSousa et al., 2013; Vanzeler, 2020).

In Brazil, according to the review study by Obelar (2016), the most commonly used instruments for assessing anxiety symptoms are the Beck Anxiety Inventory (BAI), the Hospital Anxiety and Depression Scale (HADS), the State-Trait Anxiety Inventory (STAI), the Social Phobia Inventory (SPIN), and the Hamilton Rating Scale. Additionally, Moreno et al. (2016) provided evidence of the reliability of the Generalized Anxiety Disorder 7-item scale (GAD-7). Of the mentioned instruments, only the STAI (not favorable) and BAI (favorable) are registered in the Psychological Testing Evaluation System (SATEPSI) of the Conselho Federal de Psicologia (CFP, Brazilian Federal Psychology Council). The BAI was recently included in the SATEPSI list of favorable instruments, but it does not consider the intensity and frequency of anxiety symptoms. Therefore, there are no instruments that specifically evaluate intensity and frequency of anxiety symptoms in adults with adequate psychometric properties in Brazil.

The literature review highlighted a lack of instruments with adequate evidence of validity and reliability to assess anxiety symptoms available for healthcare professionals to use in the adult population, making it necessary to adapt or build them. In this context, the Anxiety Symptoms Questionnaire (ASQ), developed by a team from the Massachusetts General Hospital, was identified (Baker et al., 2019). The ASQ is a brief and practical self-report scale that evaluates the severity of anxiety symptoms by measuring their intensity and frequency in the last week. It was designed to be a comprehensive instrument, covering physical, emotional, cognitive, and behavioral anxiety symptoms, including nervousness, worry, irritability, difficulty relaxing, insomnia, lack of energy, concentration difficulties, somatic symptoms, and impairments due to anxiety (Baker et al., 2019).

The ASQ is ideal for healthcare contexts such as hospitals, clinics, outpatient facilities, and other places where it is necessary to assess potential anxiety symptoms in patients. The questionnaire demonstrated high reliability in samples of American adults, both with and without a diagnosed anxiety disorder. It yielded Cronbach’s alpha coefficients of 0.94 and 0.96 for the overall scale, 0.89 and 0.93 for symptom intensity, and 0.90 and 0.93 for symptom frequency, respectively (Baker et al., 2019).

Before implementing a foreign instrument in Brazil, it is essential to adapt and scrutinize its psychometric properties for the specific characteristics of the population to whom it will be administered (Pacico, 2015). Therefore, when adapting assessment instruments, it is important to follow several steps: translation, synthesis of the translated version, back translation into the original language, expert analysis, application to a group belonging to the target population, and pilot study (Beaton et al., 2000; Borsa et al., 2012). Finally, an analysis of the instrument’s validity and reliability is performed to ensure that it provides accurate and consistent results, enabling the production of normative data for the intended population.

Given the foregoing, this study aimed to culturally adapt and assess the psychometric properties of the ASQ for the Brazilian context. The specific objectives were: (a) to adapt the questionnaire in predetermined stages; (b) to analyze content-based validity evidence through expert judges’ analysis; (c) to verify reliability evidence through Cronbach’s alpha and McDonald’s omega; (d) to analyze convergent validity evidence by correlating ASQ results with other instruments; (e) to investigate criterion-based validity evidence by comparing ASQ scores in groups with and without anxiety diagnoses; and (f) to verify evidence of validity based on the internal structure of the ASQ through exploratory factor analysis. The hypothesis was that the ASQ would present adequate validity and reliability evidence for the Brazilian population, consistent with the results found in the original version of the instrument (Baker et al., 2019). Considering the current context and the growing population experiencing anxiety symptoms, instruments that can assist in conducting a thorough assessment are needed to contribute to diagnoses and plan possible interventions for anxiety disorders.

Method

Approval for the adaptation of the ASQ for the Brazilian population was obtained from Dr. Amanda Baker, the author of the instrument, through email correspondence. The study was approved by the Research Ethics Committee (CAAE number 46285221.8.0000.5344). Upon agreement from the authors and REC approval, the adaptation was conducted in six stages, following Borsa et al.’s (2012) recommendations:

1st Stage − Initial Translation: The ASQ was initially translated from English to Portuguese by two independent Brazilian expert judges proficient in both languages; this resulted in two translated versions (T1 and T2). One expert was a healthcare professional, and the other was from the exact sciences.

2nd Stage − Translation Synthesis: The researchers analyzed translations T1 and T2 to create a version of the instrument suitable for the Brazilian population. While there were no divergences among the bilingual experts, certain terms were adjusted to improve item comprehension.

3rd Stage − Expert Evaluation (content validity evidence): The synthesized translation was distributed via Google Forms to three judges, specialists in Psychological Assessment with clinical and research experience. They were queried with yes or no questions regarding semantic equivalence (assessing if words had the same meaning), idiomatic equivalence (analyzing if difficult-to-translate items from the original instrument were adapted by an equivalent expression), experiential equivalence (observing if a specific item is applicable in the new culture), conceptual equivalence (evaluating if a term or expression measures the same aspect), and anxiety construct. The researchers synthesized the expert judges’ analyses, modifying only 12 sentences (36.4%) based on their suggestions.

4th Stage − Assessment by target population: A group representing the target population assessed the clarity and comprehensibility of the instrument items through in-person interviews and the utilization of Google Forms. Seven adults, ranging in age from 26 to 71, were conveniently selected and interviewed. Participants were queried about any uncertainties or challenges encountered while responding to the instrument. Subsequently, three modifications were implemented in response to the interviews, involving clarifications in response instructions and providing a clearer explanation for item 15.

5th Stage − Back translation: Two independent Brazilian judges, fluent in English (specialized in Portuguese-English translations), reverse-translated the instrument into the language of origin to ensure the conceptual meaning of terms while respecting cultural specificities. The researchers synthesized these versions, and no significant discrepancies were found. The resulting version was then presented to the instrument’s author to confirm conceptual equivalence. The author suggested modifications to three items, with two being accepted to maintain conceptual equivalence with the original version of the ASQ.

6th Stage − Pilot study: Thirty adults, with a mean age of 42.7 years (SD = 9.18), holding higher education (76.7%), participated in the study by completing the ASQ online. Among them, 20 were women (66.7%). During this stage, the instrument demonstrated robust item reliability (Cronbach’s alpha = 0.966 and McDonald’s omega = 0.978). There were no reports of difficulties in understanding the questionnaire. Consequently, this version of the instrument was retained for further analysis of psychometric properties.

Participants

For the analysis of the remaining ASQ’s psychometric properties, 467 adults, accessed by convenience, responded to the questionnaire online. Inclusion criteria considered only literate individuals of Brazilian nationality who were over the age of 18. Twenty-six participants who reported psychiatric diagnoses unrelated to the anxiety construct (eating disorders, dissociative disorders, attention deficit disorder, among others) were excluded. The remaining 441 participants were divided into three groups based on self-reporting: those without a history of psychiatric diagnosis (healthy group), those with a diagnosis of anxiety or anxiety and depression (anxiety/anxiety and depression group), and those with depression (depression group). Participants with isolated anxiety symptoms or combined symptoms of anxiety and depression were grouped together, as there were no statistically significant differences in the scale results of these participants. Therefore, they were classified as individuals with comorbid symptoms. Participants in the anxiety group had different diagnoses of this disorder, and 60% were using some form of medication. The sample’s sociodemographic data are presented in Table 1.

Table 1
Sociodemographic data of the total sample and of the participants divided into groups: Without a diagnosis (healthy), anxiety or anxiety and depression, and depression

Characteristics		Total sample N = 441	Healthy n = 301	Anxiety and Anxiety/depression n = 105	Depression n = 35
Mean Age (SD)		35.36 (11.08)	36.09 (11.56)	32.94 (9.52)	36.29 (10.42)
		%
Gender
	Male	31.5	36.2	20.0	25.7
	Female	68.3	63.5	80.0	74.3
	Other	00.2	0.3	0	0
Marital status
	Single	42.6	40.9	46.7	45.7
	Married/Common-law marriage	48.6	50.2	43.8	48.6
	Separated/Divorced	07.5	07.6	07.6	05.7
	Widowed	2	0.3	0	0
	Other	01.1	1	01.9	0
Education
	Incomplete Primary Education	00.2	0.3	0	02.9
	Complete Primary Education	00.5	0.3	0	0
	Incomplete Secondary Education	01.1	0.7	02.9	0
	Complete Secondary Education	07.7	9.0	06.7	0
	Incomplete Higher Education	32.7	31.6	38.1	25.7
	Complete Higher Education	57.8	58.1	52.4	71.4
Family Income (minimum wages)
	Up to one minimum wage	01.6	0.7	02.9	05.7
	From one to two minimum wages	11.8	12.6	12.4	02.9
	From three to five minimum wages	36.3	39.5	26.7	37.1
	Above five minimum wages	50.3	47.2	58.1	54.3

Data Collection Instruments and Procedures

Data collection occurred online from September to November 2021 to ensure participant safety during the pandemic. This approach enhances information reach, data collection efficiency, and cost-effectiveness (Faleiros et al., 2016). Participation invitations were sent based on convenience, utilizing the researchers’ email and social media network.

The questionnaire was developed using the online tool Google Forms. Participants were provided access to the research objectives and the Informed Consent Form (ICF), which included a confirmation and acceptance field. All data were exclusively stored and accessed by the researchers to ensure the confidentiality of participant information. Subsequently, participants completed a sociodemographic data questionnaire developed by the researchers, gathering personal information such as age, gender, education, family income, and health-related questions. Following this, participants responded to the Anxiety Symptoms Questionnaire (ASQ) (Baker et al., 2019), comprised of 17 items rated on a 10-point Likert scale. The questionnaire assesses the intensity (discomfort level) and frequency (recurrence) of symptoms over the past week. The frequency and intensity are scored on a scale ranging from 0 to 170, with a total scale score of up to 340 points (available in the supplementary material).

Participants also responded to the Structured Clinical Interview for DSM-5 (SCID-5 - CV clinical version) (APA, 2014), which assesses the presence of physical symptoms and impairments in the individual’s functioning related to anxiety symptoms, containing 14 self-report questions with yes or no responses (First et al., 2017). Finally, they completed the Depression, Anxiety and Stress Scale − DASS-21. This is a self-report scale that assesses these symptoms, with 21 statements ranging from 0 (did not apply to me at all) to 3 points (applied to me very much or most of the time). The Portuguese version was adapted and validated by Vignola and Tucci (2014), with a Cronbach’s alpha of 0.92 for depression, 0.90 for stress, and 0.86 for anxiety, indicating good internal consistency for each subscale. At the end, participants received a link to a booklet on managing anxiety during the pandemic (Silva et al., 2020) as a token of appreciation for participating in the study.

Results

Content validity evidence was obtained through the evaluation of expert judges, with a criterion of at least 80% agreement among them. Of the instrument’s 33 items, there was 90.9% agreement in semantic equivalence and 93.9% in idiomatic, experiential, and conceptual equivalences. The judges agreed that all the items measured the anxiety construct (100% agreement).

Cronbach’s alpha (α) and McDonald’s omega (ω) were analyzed to assess the instrument’s reliability. For an index to be deemed reliable, it should surpass 0.70 (Terwee et al., 2007). In the analysis of internal consistency of the ASQ (N = 441), a high reliability index was found for the total scale (α = 0.975 and ω = 0.976), as well as in the intensity and frequency scores (α = 0.950 and ω = 0.952) of anxiety symptoms. In the sample divided into subgroups, a high reliability index of the total score was also found in the healthy (α = 0.974 and ω = 0.975), anxiety/anxiety and depression (α = 0.971 and ω = 0.971), and depression (α = 0.966 and ω = 0.965) groups.

Convergent validity evidence was analyzed between the results of the ASQ, SCID-5, and DASS-21 using Pearson’s correlation coefficient. Regarding the strength of correlations, values from 0 to 0.19 were considered very weak, from 0.20 to 0.39 weak, from 0.40 to 0.69 moderate, from 0.70 to 0.89 strong, and from 0.90 to 1.0 very strong (Dancey & Reidy, 2013). In the analysis of convergent validity evidence, the ASQ exhibited a significant positive correlation, ranging from moderate to strong, with the DASS-21 scores and SCID-5 items in both the total sample and subgroups, as illustrated in Table 2. Stronger correlations were observed in the healthy group for the total ASQ scale, as well as the intensity and frequency subscales, compared to the clinical groups. The weakest correlations were identified in the anxiety and anxiety/depression group, specifically between the total ASQ scale and the frequency subscale of ASQ with the DASS-21 subscales (Table 2).

Table 2
Pearson’s Correlation between Anxiety Symptoms Questionnaire, The Structured Clinical Interview for DSM-5, and Depression, Anxiety and Stress Scale -21

Groups		SCID-5	DASS-21 Depression	DASS-21 Stress	DASS-21 Anxiety
Total sample (N = 441)
	ASQ Intensity	0.676**	0.588**	0.665**	0.574**
	ASQ Frequency	0.681**	0.590**	0.670**	0.575**
	ASQ Total	0.684**	0.594**	0.673**	0.579**
Healthy (n = 301)
	ASQ Intensity	0.696**	0.636**	0.700**	0.619**
	ASQ Frequency	0.694**	0.629**	0.700**	0.608**
	ASQ Total	0.701**	0.638**	0.700**	0.619**
Anxiety and Anxiety/Depression (n = 105)
	ASQ Intensity	0.405**	0.388**	0.448**	0.332**
	ASQ Frequency	0.434**	0.398**	0.459**	0.350**
	ASQ Total	0.424**	0.397**	0.458**	0.344**
Depression (n = 35)
	ASQ Intensity	0.719**	0.423**	0.554**	0.465**
	ASQ Frequency	0.750**	0.487**	0.605**	0.540**
	ASQ Total	0.742**	0.459**	0.585**	0.507**

• Note:
• ^**
  Significant correlation at the 0.01 level.
• ASQ: Anxiety Symptoms Questionnaire; DASS-21: Depression, Anxiety and Stress Scale; SCID-5: The Structured Clinical Interview for DSM-5 - Clinical Version.

To obtain criterion validity evidence among groups of individuals with or without any psychopathological diagnosis (anxiety, anxiety/depression, and depression), a one-way analysis of variance (one-way ANOVA) was conducted, with Hochberg’s GT2 post hoc test (Nascimento et al., 2019). In addition to statistical significance, following Cohen (1988), it was checked whether differences between groups had small (partial eta-squared - η_p² < 0.01), medium (η_p² between 0.02 and 0.06), or large (η_p² > 0.14) effect sizes. Statistically significant differences with medium effect sizes were found in the sample (healthy, anxiety/depression, and depression groups) for the total score (F(2.438) = 22.647; p < 0.001; η_p² = 0.094), intensity (F(2.438) = 23.280; p < 0.001; η_p² = 0.096), and frequency (F(2.438) = 21.225; p < 0.001; η_p² = 0.088) of anxiety symptoms reported in the instrument. The healthy group had the lowest scores in total (M = 109.01; SD = 70.73), intensity (M = 56.09; SD = 35.54), and frequency of the ASQ (M = 52.93; SD = 35.79). The anxiety group had higher scores compared to the other groups in total (M = 163.12; SD = 72.96), intensity (M = 83.83; SD = 37.30), and frequency of the ASQ (M = 79.30; SD = 36.36). The depression group had scores similar to the healthy group in total (M = 126.23; SD = 67.01), intensity (M = 65.20; SD = 34.74), and frequency of the ASQ (M = 61.03; SD = 33.01). Group comparisons are outlined in Table 3. The Hochberg GT2 post hoc test demonstrated significant differences between the anxiety and anxiety/depression groups compared to the healthy and depression groups in the ASQ scores. There were no statistically significant differences in ASQ scores between the healthy and depression groups.

Table 3
Hochberg’s General Type-2 Post hoc Test in the Comparisons of Anxiety Symptoms Questionnaire Scores

ASQ Total
Groups	Groups	Mean difference	p	95% CI
				Limit inferior	Limit superior
Healthy	Anxiety and Anxiety/depression	-54.111*	<0.001	-73.39	-34.83
	Depression	-17.215	00.438	-47.60	13.17
Anxiety and Anxiety/depression	Depression	0036.895*	00.240	03.69	70.10
ASQ Intensity
Groups	Groups	F	p	95% CI
				Limit inferior	Limit superior
Healthy	Anxiety and Anxiety/depression	0-27.742*	<0.001	-37.50	0-17.99
	Depression	-9.114	00.398	-24.48	0 6.26
Anxiety and Anxiety/depression	Depression	0018.629*	00.240	001.83	35.43
ASQ Frequency
Groups	Groups	F	p	95% CI
				Limit inferior	Limit superior
Healthy	Anxiety and Anxiety/depression	0-26.368*	<0.001	-36.07	-16.66
	Depression	-8.102	00.497	-23.39	0 7.19
Anxiety and Anxiety/depression	Depression	0 18.267*	00.270	001.56	34.98

• Note:
• ^*
  Significant difference at the 0.05 level.
• ASQ: Anxiety Symptoms Questionnaire; CI: Confidence Interval.

Exploratory factor analysis was conducted to assess the ASQ’s structure in the Brazilian sample, utilizing a polychoric matrix and the Robust Diagonally Weighted Least Squares (RDWLS) extraction method (Asparouhov & Muthen, 2010). The number of factors to be retained was determined by using the Parallel Analysis method with random permutation of observed data (Timmerman & Lorenzo-Seva, 2011), employing Robust Promin rotation (Lorenzo-Seva & Ferrando, 2019). Bartlett’s test of sphericity (4925.6; df = 561; p < 0.001) and the Kaiser-Meyer-Olkin (KMO) index (KMO = 0.91) suggested interpretability of the item correlation matrix. The parallel analysis indicated the extraction of a single factor, confirming the unidimensionality of the ASQ, consistent with the original study involving the American sample. As depicted graphically in Figure 1, the percentage of explained variance in the actual data surpasses the percentage of explained variance in the random data for only one factor.

Figure 1
Parallel analysis results, indicating the retention of one factor

The factor loadings of the items, the Composite Reliability index, as well as the replicability estimate of the factor scores (H-index) (Ferrando & Lorenzo-Seva, 2018) are presented in Table 4. The items exhibited appropriate factor loadings, with the lowest value being 0.623 (“item dormancy, tingling, flushing”, intensity) and the highest being 0.862 (“feeling restless”, intensity). The composite reliability of the factors was also adequate (above 0.70), as well as the measure of replicability of the factorial structure (H-index > 0.80) (Ferrando & Lorenzo-Seva, 2018). It is noteworthy that the factorial structure showed appropriate fit indices (χ² = 1823.348; df = 527; p < 0.001; RMSEA = 0.075; CFI = 0.981; TLI = 0.98). Finally, it is highlighted that the indicators Unidimensional Congruence (UniCo = 0.988), Explained Common Variance (ECV = 0.921), and Mean of Item Residual Absolute Loadings (MIREAL = 0.169) (Ferrando & Lorenzo-Seva, 2018) supported the unidimensionality of the scale.

Table 4
Factor loadings of the Anxiety Symptoms Questionnaire items

Items	Factor loading
Feeling restless, keyed up, or on edge (Intensity)	0.862
Feeling restless, keyed up, or on edge (Frequency)	0.851
Trouble relaxing (Intensity)	0.827
Trouble relaxing (Frequency)	0.825
Anxiety (Frequency)	0.817
Worrying (Frequency)	0.810
Worrying (Intensity)	0.807
Trouble functioning at home, work (Intensity)	0.804
Trouble functioning at home, work (Frequency)	0.802
Fatigue or lack of energy (Intensity)	0.797
Anxiety (Intensity)	0.793
Fatigue or lack of energy (Frequency)	0.790
Problems with concentration or attention (Frequency)	0.789
Anticipating or fearing something bad might happen (Frequency)	0.787
Problems with concentration or attention (Intensity)	0.785
Anticipating or fearing something bad might happen (Intensity)	0.782
Nervousness (Frequency)	0.781
Nervousness (Intensity)	0.761
Irritability (Intensity)	0.757
Irritability (Frequency)	0.751
Shortness of breath, chest tightness or pain, pounding/skipping/racing heartbeat (Intensity)	0.731
Dizziness, lightheadedness, headaches, trembling or shakiness (Intensity)	0.717
Muscle tension or tightness (Frequency)	0.717
Dizziness, lightheadedness, headaches, trembling or shakiness (Frequency)	0.717
Shortness of breath, chest tightness or pain, pounding/skipping/racing heartbeat (Frequency)	0.716
Muscle tension or tightness (Intensity)	0.710
Trouble remembering things (Intensity)	0.709
Trouble remembering things (Frequency)	0.706
Stomach upset, nausea, constipation, diarrhea, or irritable bowels (Intensity)	0.693
Stomach upset, nausea, constipation, diarrhea, or irritable bowels (Frequency)	0.677
Trouble falling or staying asleep (Frequency)	0.636
Numbness, tingling, excessive sweating, or flushing (Frequency)	0.628
Trouble falling or staying asleep (Intensity)	0.626
Numbness, tingling, excessive sweating, or flushing (Intensity)	0.623
Composite reliability	0.978
H-Latent	0.980
H-Observed	0.932

Discussion

In adapting the ASQ to assess anxiety symptoms in the Brazilian context, this study revealed appropriate psychometric properties. The results, derived from a predominantly young adult sample, suggest that the ASQ provides evidence of validity based on content, external criterion-relatedness, internal consistency, and convergent validity. These findings collectively support the instrument’s suitability for evaluating anxiety symptoms in Brazilian adults. Nonetheless, further investigation with a more diverse sample, encompassing different age groups and educational levels, is warranted.

As highlighted in the ASQ adaptation procedures, the series of steps undertaken aim to render the instrument understandable and culturally suitable for the target population. Furthermore, translation, back translation, expert judge analysis, and the pilot study were essential for ensuring that the process was conducted appropriately, as suggested by Beaton et al. (2000) and Borsa et al. (2012). The importance of conducting test adaptations in stages is to ensure that the adapted instrument maintains reproducibility and validity while minimizing biases from the original instrument’s culture (Borges et al., 2010; Pacico, 2015). Instrument adaptation studies in Brazil follow international recommendations (Erazo-Chavez et al., 2021; Henklain et al., 2020; Rama, 2021), contributing to establishing a methodological standard and obtaining more consistent results for our context, as demonstrated with the ASQ.

Regarding content validity evidence, during the expert judges’ analysis stage, it was verified that the items of the ASQ measure the anxiety construct. The judges’ role was crucial in this stage, as, in addition to confirming content validity evidence, they provided suggestions to render the instrument more suitable for measuring the construct. Content validity is essential and central to the process of aligning the instrument’s content with the construct (American Educational Research Association [AERA] et al., 2014; Alexandre & Coluci, 2011), as it can interfere with parameter evaluation if the content is inappropriate, even if other validity indices are satisfactory (Haynes et al., 1995).

The results of the ASQ’s reliability analysis indicated that the instrument demonstrates excellent internal consistency, with values above 0.90. Similar indices were found in the original study of the instrument (Baker et al., 2019), supporting the hypothesis that similar psychometric properties would be found in both countries. Reliability indices indicate that all items in the scale measure the same construct, (Souza et al., 2017) in this case, anxiety symptoms, demonstrating adequate evidence of item reliability for the ASQ in the Brazilian context.

The results of the ASQ analyses also indicated that it shows evidence of convergent validity, correlating moderately with SCID-5 scores (DSM-5 clinical interview) and DASS-21 scores (a scale with validity evidence to assess stress, depression, and anxiety symptoms in adults in Brazil) in the total sample. The anxiety, depression, and stress constructs share common symptoms, which often overlap (Patias et al., 2016) or have a propensity to be comorbid (Craske & Stein, 2016; Demenech et al., 2021), which may explain the stronger correlations in the group of adults without a reported psychopathological diagnosis.

The correlations between the ASQ and the DASS-21 showed moderate to strong positive indices in the depression and without a diagnosis (healthy) groups. The ASQ differs from the DASS-21 by assessing the intensity and frequency of symptoms, which may account for the more moderate correlations. The anxiety and anxiety/depression group showed weak positive correlations in responses across all instruments. This group comprises a heterogeneous sample, with individuals experiencing different types and degrees of symptoms, whether or not they are undergoing treatment or using medications, which may explain the lower correlations. Moreover, the data were self-reported, lacking clinical verification of the diagnosis, and participants did not disclose whether they were undergoing psychotherapy, representing a study limitation. Therefore, for future research, it is suggested to use samples with clinically confirmed diagnostic groups and control for the treatment variable to obtain more homogeneous groups, thus reducing the potential for biases in the results.

Correlations between the ASQ and the SCID-5 showed moderate to strong indices (0.67 to 0.75) in the total scale and in the intensity and frequency subscales, except for the anxiety and anxiety/depression group. The SCID-5 is based on DSM-5 criteria, which considers the intensity and frequency of anxiety symptoms for its diagnostic parameters, aligning with the ASQ assessment, potentially explaining the strong correlations. The relationship between results from tests measuring the same construct provides crucial validity evidence (AERA et al., 2014); in other words, high correlations with external variables are indicative of convergent validity. Therefore, based on correlation analyses, considering sample and instrument peculiarities, it is understood that the ASQ demonstrates evidence of convergent validity. Furthermore, the ASQ is an interdisciplinary instrument suitable for use in various health contexts, as its assessment encompasses aspects aligned with DSM-5 (Baker et al., 2019).

In the comparisons between groups, no significant differences were observed between the depression group and the group without a diagnosis (healthy) in terms of intensity, frequency, and the total ASQ score. This contrasts with the original study, which demonstrated effective discrimination between groups with anxiety and depression in participants without comorbidities (Baker et al., 2019). This finding suggests that the ASQ is not conducive to contributing to the diagnosis of depression in Brazilian adults. Depression encompasses symptoms such as a depressed mood, diminished interest or pleasure in almost all activities, and feelings of worthlessness or excessive guilt (APA, 2014), aspects not measured by the ASQ. Furthermore, the ASQ was not constructed for this purpose; its intended use is solely to contribute to the assessment of anxiety disorders.

Conversely, the comparison between the anxiety and anxiety/depression groups and the control group revealed statistically significant differences. This indicates that the ASQ can effectively distinguish individuals with and without anxiety symptoms, thereby reinforcing its criterion-related validity evidence. Criterion validity is associated with the instrument’s potential to predict the performance of an external variable (AERA et al., 2014). Evidence of criterion validity is crucial in clinical practice as it underscores the instrument’s appropriateness for measuring the target construct in a representative sample (Alexandre et al., 2013; Rodrigues et al., 2019).

The study to construct the ASQ envisioned four factors in the exploratory factor analysis, as the instrument encompasses the physical, emotional, cognitive, and behavioral symptoms of anxiety. However, these symptoms are interconnected, as an individual with thoughts of anxiety demonstrates this simultaneously in their behavior and emotions. The exploratory factor analysis confirmed the unidimensionality of the instrument, emphasizing its evidence of validity based on internal structure, as also found in the original scale (Baker et al., 2019). Therefore, the ASQ presents an appropriate theoretical framework in assessing anxiety disorders, represented by its set of items. The validity of an instrument cannot be assessed from a single source of evidence. It is established based on various sources of validity evidence to ensure that a particular instrument evaluates what it purports to. Thus, the more validity evidence obtained for an instrument, the more reliable the interpretation of its results (Pacico et al., 2015). The ASQ presented validity evidence based on content, criterion, convergent, and internal structure, reinforcing that it is a suitable instrument for use in adults in the Brazilian context.

Anxiety is a symptom to which all individuals are susceptible to feeling at some point or in certain situations. What distinguishes it as adaptive or pathological is the intensity and frequency of symptoms and the impact it has on daily activities (APA, 2014). The high prevalence of Brazilians experiencing anxiety symptoms (WHO, 2017), further exacerbated by the COVID-19 pandemic (Barros et al., 2020; Goularte et al., 2021) particularly in the young adult population (Barros et al., 2020; Demenech et al., 2021), has negatively influenced the emotional health of this population (Barros et al., 2020; Schuch et al., 2020). From this perspective, considering the importance of having accurate instruments to measure anxiety symptoms, the ASQ becomes a suitable option due to its ability to comprehensively assess symptoms, potentially aiding healthcare professionals in making more precise diagnoses.

Conclusion

This study presented the adaptation process and psychometric properties of an instrument for assessing anxiety symptoms in adults, a novelty in the Brazilian context. The ASQ demonstrated consistent content validity evidence through expert judges’ evaluation, convergent validity through correlations with other instruments, criterion validity through group comparisons (healthy, anxiety and anxiety/depression, and depression groups), and internal structure validity, indicating its unidimensionality. Furthermore, the results showed excellent internal consistency indices through Cronbach’s alpha and McDonald’s omega. Therefore, the ASQ is a suitable instrument for assessing anxiety symptoms in Brazilian adults. It distinguishes itself from other scales by the comprehensive assessment of symptoms, considering both intensity and frequency, two variables crucial for diagnosing anxiety disorders, reducing the likelihood of false positives.

The limitations of this study include the use of convenience sampling as the data collection method. Specifically, this type of sample may not fully represent the Brazilian population, especially given that the majority of participants were young adults, female, and had completed higher education. Additionally, the data collection period during the COVID-19 pandemic may have influenced the results due to the psychological impacts caused by the restrictions during this period, as well as the online data collection method. Another consideration is the heterogeneity of the anxiety and anxiety/depression group, which includes various types and degrees of symptoms, obtained through self-report without clinical validation of the diagnosis. Information about whether or not participants were undergoing psychological treatment was also not considered. For future studies, clinical groups will be defined based on anxiety scale cutoff points, in addition to participant self-reports. Variables such as the time since diagnosis, medication use, and psychiatric treatment will be considered to homogenize the sample.

For future studies, it is also suggested that the validity evidence of the scale be supplement with regression analysis to examine covariates of ASQ scores, as in the original study. Additionally, there is a recommendation to expand the use of the instrument to other populations with diverse sociodemographic characteristics. At this juncture, the applicability of the ASQ is being tested in adolescents, as this phase of human development is also critical for anxiety disorders. In future studies, the psychometric properties of the pencil-and-paper version of the ASQ will be compared, considering that the instrument was initially tested online (remotely).

Finally, based on the results, it is believed that the ASQ is a reliable instrument suitable for application in the Brazilian population across various healthcare contexts. However, caution is warranted in interpreting results for samples not tested in the present study. Given the escalating number of individuals experiencing anxiety symptoms, particularly in this pandemic period, the ASQ proves to be a suitable tool for remote assessments, applicable online, with effective potential to aid in investigating the diagnosis of this disorder.

Acknowledgments

We would like to thank the expert judges and volunteers who contributed to this research.

• How to cite ithis article: Ujihara, M. M. H., & Rodrigues, J. C. (2024). Adaptation and psychometric properties of the Anxiety Symptoms Questionnaire for Brazil. Estudos de Psicologia (Campinas), 41, e220035. https://doi.org/10.1590/1982-0275202441e220035

References

• Alexandre, N. M. C., & Coluci, M. Z. O. (2011). Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciência & Saúde Coletiva, 16(7), 3061-3068. https://doi.org/10.1590/S1413-81232011000800006
  » https://doi.org/10.1590/S1413-81232011000800006
• Alexandre, N. M. C., Gallasch, C. H., Lima, M. H. M., & Rodrigues, R. C. M. (2013). A confiabilidade no desenvolvimento e avaliação de instrumentos de medida na área da saúde. Revista Eletrônica de Enfermagem, 15(3), 800-807. https://doi.org/10.5216/REE.V15I3.20776
  » https://doi.org/10.5216/REE.V15I3.20776
• American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing
• American Psychiatric Association. (2014). Manual diagnóstico e estatístico de transtornos mentais: DSM-5 (5th ed.). Artmed.
• Asparouhov, T., & Muthen, B. (2010). Simple second order chi-square correction Unpublished manuscript. Available from: https://www.statmodel.com/download/WLSMV_new_chi21.pdf
  » https://www.statmodel.com/download/WLSMV_new_chi21.pdf
• Baker, A., Simon, N., Keshaviah, A., Farabaugh, A., Deckersbach, T., Worthington, J. J., Hoge, E., Fava, M., & Pollack, M. P. (2019). Anxiety Symptoms Questionnaire (ASQ): Development and validation. General Psychiatry, 32(6), e100144. https://doi.org/10.1136/gpsych-2019-100144
  » https://doi.org/10.1136/gpsych-2019-100144
• Barros, M. B. A., Lima, M. G., Malta, D.C., Szwarcwald, C. L., Azevedo, R. C. S., Romero, D., Souza-Júnior, P. R. B., Azevedo, L. O., Machado, I. E., Damacena, G. M., Gomes, C. S., Werneck, A. O., Silva, D. R. P., Pina, M. F., & Gracie, R. (2020). Relato de tristeza/depressão, nervosismo/ansiedade e problemas de sono na população adulta brasileira durante a pandemia de COVID-19. Epidemiologia e Serviços de Saúde, 29(4), e2020427. https://doi.org/10.1590/S1679-49742020000400018
  » https://doi.org/10.1590/S1679-49742020000400018
• Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. https://doi.org/10.1097/00007632-200012150-00014
  » https://doi.org/10.1097/00007632-200012150-00014
• Borges, V., Balbinotti, M., & Teodoro, M. (2010). Tradução e validação de conteúdo: Uma proposta para a adaptação de instrumentos. In L. Pasquali (Org.), Instrumentação psicológica: fundamentos e prática (pp. 506-520). Artmed.
• Borsa, J. C., Damasio, B. F., & Bandeira, D. R. (2012). Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia, 22(53), 423-432. https://dx.doi.org/10.1590/S0103-863X2012000300014
  » https://doi.org/10.1590/S0103-863X2012000300014
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
• Craske, M. G., & Stein, M. B. (2016). Anxiety. Lancet, 388(10063), 3048-3059. https://doi.org/10.1016/S0140-6736(16)30381-6
  » https://doi.org/10.1016/S0140-6736(16)30381-6
• Dancey, C. P., & Reidy, J. (2013). Estatística sem matemática para psicologia (5th ed.) Artmed.
• Demenech, L. M., Oliveira, A. T., Neiva-Silva, L., & Dumith, S. C. (2021). Prevalence of anxiety, depression and suicidal behaviors among Brazilian undergraduate students: A systematic review and meta-analysis. Journal of Affective Disorders, 282, 147-159. https://doi.org/10.1016/j.jad.2020.12.108
  » https://doi.org/10.1016/j.jad.2020.12.108
• DeSousa, D. A., Moreno, A. L., Gauer, G., Manfro, G. G., & Koller, S. H. (2013). Revisão sistemática de instrumentos para avaliação de ansiedade na população brasileira. Avaliação Psicológica, 12(3), 397-410 http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712013000300015&lng=pt&tlng=pt
  » http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712013000300015&lng=pt&tlng=pt
• Erazo-Chavez, L. J., La-Rotta, E. I. G., & Onocko-Campos, R. T. (2021). Adaptação transcultural do Recovery Self-Assessment RSA-R família/Brasil: evidências de validade baseada no conteúdo. Ciência & Saúde Coletiva, 26, 3693–3704. https://doi.org/10.1590/1413-81232021269.2.32692019
  » https://doi.org/10.1590/1413-81232021269.2.32692019
• Faleiros, F., Käppler, C., Ramos Pontes, F. A., Souza da Costa Silva, S., Santos Nogueira de Goes, F., & Dias Cucick, C. (2016). Uso de questionário online e divulgação virtual como estratégia de coleta de dados em estudos científicos. Texto & Contexto Enfermagem, 25(4), 1-6. https://doi.org/10.1590/0104-07072016003880014
  » https://doi.org/10.1590/0104-07072016003880014
• Ferrando, P. J., & Lorenzo-Seva U. (2018). Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educational and Psychological Measurement, 78, 762-780. https://doi.org/10.1177/0013164417719308
  » https://doi.org/10.1177/0013164417719308
• First, M. B.W., Janet B. W. K., Rhonda S., & Spitzer, R. L. (2017). Entrevista clínica estruturada do DSM-5: SCID- 5 –CV: versão clínica Artmed.
• Goularte, J. F., Serafim, S. D., Colombo, R., Hogg, B., Caldieraro, M. A., & Rosa, A. R. (2021). COVID-19 and mental health in Brazil: Psychiatric symptoms in the general population. Journal of Psychiatric Research, 132, 32-37. https://doi.org/10.1016/j.jpsychires.2020.09.021
  » https://doi.org/10.1016/j.jpsychires.2020.09.021
• Haynes, S. N., Richard, D. C. S., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238-247. https://doi.org/10.1037/1040-3590.7.3.238
  » https://doi.org/10.1037/1040-3590.7.3.238
• Henklain, M., Carmo, J., Haydu, V., Muniz, M., Buskist, W., & Keeley, J. W. (2020). Teacher Behavior Checklist: Psychometric evidence in teacher evaluation by Brazilian university students. Paidéia, 30, e3025. https://doi.org/10.1590/1982-4327e3025
  » https://doi.org/10.1590/1982-4327e3025
• Lorenzo-Seva, U., & Ferrando, P. J. (2019). Robust Promin: A method for diagonally weighted factor rotation. Revista Peruana de Psicología, 25, 99-106. https://doi.org/10.24265/liberabit.2019.v25n1.08
  » https://doi.org/10.24265/liberabit.2019.v25n1.08
• Moreno, A., De Sousa, D., Souza, A., Manfro, G., Salum, G., Koller, S., Osório, F., & Crippa, J. (2016). Factor structure, reliability, and item parameters of the Brazilian-Portuguese version of the GAD-7 Questionnaire. Temas em Psicologia, 24, 367-376. https://doi.org/10.9788/TP2016.1-25
  » https://doi.org/10.9788/TP2016.1-25
• Nascimento, D. C., Rocha, C., & Prestes, J. (2019). Procedimentos post hoc: orientação para praticantes de estatística em ciências da saúde. Arquivos de Ciências do Esporte, 6, 45-49. https://doi.org/10.17648/aces.v6n2.2783
  » https://doi.org/10.17648/aces.v6n2.2783
• Obelar, R. M. (2016). Avaliação psicológica nos transtornos de ansiedade: Estudos brasileiros [Monografia de especialização em Avaliação Psicológica, Universidade Federal do Rio Grande do Sul]. Lume - Repositório Digital. https://lume.ufrgs.br/handle/10183/147064
  » https://lume.ufrgs.br/handle/10183/147064
• Pacico J. C. (2015). Como é feito um teste? Produção de itens. In C. S. Hutz, D. R. Bandeira, & C. M. Trentini (Eds.), Psicometria (pp. 54-69). Artmed.
• Pacico, J. C., Hutz, C. S., Schneider, A. M. A., & Bandeira, D. (2015). Validade. In C. S. Hutz, D. R. Bandeira, & C. M. Trentini (Eds.), Psicometria (pp. 70-83). Artmed.
• Patias, N., Machado, W., Bandeira, D., & Dell’Aglio, D. (2016). Depression Anxiety and Stress Scale (DASS-21) - Short Form: Adaptação e validação para adolescentes Brasileiros. Psico-USF, 21, 459-469. https://doi.org/10.1590/1413-82712016210302
  » https://doi.org/10.1590/1413-82712016210302
• Rama, C. G., Bernardes, F. B., Lefton-Greif, M. A., Levy, D. S., & Bosa, V. L. (2021). Translation, cultural adaptation, reliability, and validity evidence of the Feeding/Swallowing Impact Survey (FS-IS) to Brazilian Portuguese. Dysphagia, 37(5), 1226-1237. https://doi.org/10.1007/s00455-021-10383-4
  » https://doi.org/10.1007/s00455-021-10383-4
• Rodrigues, J. D., Becker, N., Beckenkamp, C. L., Miná, C. S., de Salles, J. F., & Bandeira, D. R. (2019). Psychometric properties of cognitive screening for patients with cerebrovascular diseases: A systematic review. Dementia & Neuropsychologia, 13(1), 31-43. https://doi.org/10.1590/1980-57642018dn13-010004
  » https://doi.org/10.1590/1980-57642018dn13-010004
• Schuch, F. B., Bulzing, R. A., Meyer, J., Vancampfort, D., Firth, J., Stubbs, B., Grabovac, I., Willeit, P., Tavares, V., Calegaro, V. C., Deenik, J., López-Sánchez, G. F., Veronese, N., Caperchione, C. M., Sadarangani, K. P., Abufaraj, M., Tully, M. A., & Smith, L. (2020). Associations of moderate to vigorous physical activity and sedentary behavior with depressive and anxiety symptoms in self-isolating people during the COVID-19 pandemic: A cross-sectional survey in Brazil. Psychiatry Research, 292, e113339. https://doi.org/10.1016/j.psychres.2020.113339
  » https://doi.org/10.1016/j.psychres.2020.113339
• Silva, J. S., Oliveira, A. L. C. B., Fernandes, M. A., & Paula, J. M. S. F. (2020). Ansiedade em tempos de Covid-19: Como lidar? Universidade Federal do Piauí. https://ufpi.br/arquivos_download/arquivos/Cartilha_-Como_lidar_com_a_ansiedade_em_tempos_de_COVID-19-compactado20200518172617.pdf
  » https://ufpi.br/arquivos_download/arquivos/Cartilha_-Como_lidar_com_a_ansiedade_em_tempos_de_COVID-19-compactado20200518172617.pdf
• Souza, A. C, Alexandre, N. M. C., & Guirardello, E. de B. (2017). Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade. Epidemiologia e Serviços de Saúde, 26(3), 649-659. https://dx.doi.org/10.5123/s1679-49742017000300022
  » https://doi.org/10.5123/s1679-49742017000300022
• Terwee, C., Bot, S., Boer, M., van der Windt, D., Knol, D., Dekker, J., Bouter, L., & De Vet, H. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60, 34-42. https://doi.org/10.1016/j.jclinepi.2006.03.012
  » https://doi.org/10.1016/j.jclinepi.2006.03.012
• Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16, 209-220. https://dx.doi.org/doi:10.1037/a0023353
  » https://doi.org/10.1037/a0023353
• Vanzeler, M. L. A. (2020). Transtornos de ansiedade e avaliação psicológica: Instrumentos utilizados no Brasil. Revista Científica Multidisciplinar Núcleo do Conhecimento, 10(13), 100-120. https://doi.org/10.32749/nucleodoconhecimento.com.br/psicologia/instrumentos-utilizados
  » https://doi.org/10.32749/nucleodoconhecimento.com.br/psicologia/instrumentos-utilizados
• Vignola, R. C., & Tucci, A. M. (2014). Adaptation and validation of the depression, anxiety, and stress scale (DASS) to Brazilian Portuguese. Journal of Affective Disorders, 155, 104-109. https://doi.org/10.1016/j.jad.2013.10.031
  » https://doi.org/10.1016/j.jad.2013.10.031
• World Health Organization. (2017). Depression and other common mental disorders: Global health estimates https://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf
  » https://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf
• World Health Organization. (2022). World mental health report: Transforming mental health for all https://iris.who.int/bitstream/handle/10665/356119/9789240049338-eng.pdf?sequence=1
  » https://iris.who.int/bitstream/handle/10665/356119/9789240049338-eng.pdf?sequence=1

Editor

Tatiana de Cássia Nakano

Publication Dates

• Publication in this collection
  23 Sept 2024
• Date of issue
  2024

History

• Received
  05 Apr 2022
• Reviewed
  12 June 2023
• Accepted
  14 Dec 2023