Validity evidence of the Depression, Anxiety, and Stress Scale in Brazilian nursing workers

Abstract

Objectives  To investigate the validity evidence of the Depression, Anxiety, and Stress Scale (DASS-21) and to analyze the prevalence of depression, anxiety, and stress symptoms in a sample of Brazilian nursing workers.

Methods  This was a cross-sectional study with non-probability convenience sampling. A total of 4053 Brazilian nursing workers participated in the study. The psychometric properties of DASS-21 were assessed by analysis of the validity based on the internal structure (factorial, convergent, and discriminant construct validity, invariance of the factorial model, and reliability); relationships with external measures (positive and negative convergent validity of the depression, anxiety, and stress factors with the emotional impairment, job satisfaction, and individual and occupational variables); and response process of the items (analysis of the differential item functioning in distinct subgroups). Data reliability was certified by estimating the ordinal alpha coefficient and the composite reliability. The prevalence of depression, anxiety, and stress symptoms was estimated based on recommendations from the original authors of DASS-21.

Results  During validation of the internal structure, the confirmatory factor analysis showed that the fit of the sample to the original DASS-21 model was adequate and the strong measurement invariance of the model was confirmed in different groups. Analysis based on external measures showed that the depression, anxiety, and stress factors had positive and moderate-strong correlations with the emotional impairment domain and negative and moderate correlations with the job satisfaction construct. Analysis of the pattern of response to items confirmed the uniformity and stability of the instrument for the sample. Regarding symptoms, the high prevalence of extremely severe levels of depression, anxiety, and stress was highlighted among participants.

Conclusion  DASS-21 presented adequate validity and reliability to assess depression, anxiety, and stress symptoms among Brazilian nursing workers.

Occupational health; Depression; Anxiety; Stress, psychological; Psychometrics; Psychiatric Status Rating Scales; Nurse practitioners; Surveys and questionnaires; Brazil

Resumo

Objetivo  Investigar as evidências de validade da Depression, Anxiety and Stress Scale (DASS-21) para análise da prevalência de sintomas de depressão, ansiedade e estresse em uma amostra de trabalhadores de enfermagem brasileiros.

Métodos  Estudo transversal, com amostragem não-probabilística. Participaram do estudo 4053 trabalhadores de enfermagem brasileiros. As propriedades psicométricas da DASS-21 foram avaliadas por meio de análise da validade baseada na estrutura interna (validade de construto fatorial, convergente e discriminante; invariância do modelo fatorial; confiabilidade); validade baseada nas relações com medidas externas (validade convergente positiva e negativa dos fatores depressão, ansiedade e estresse com os construtos prejuízo emocional, satisfação no trabalho e variáveis individuais e ocupacionais); validade baseada no padrão de resposta aos itens do instrumento (análise da função diferencial dos itens em subgrupos distintos). A confiabilidade dos dados foi atestada por meio da estimação do coeficiente alfa ordinal e da confiabilidade composta. A prevalência de sintomas de depressão, ansiedade e estresse foi estimada a partir das recomendações dos autores originais da DASS-21.

Resultados  Durante a validação da estrutura interna, a análise fatorial confirmatória revelou que o modelo original da DASS-21 apresentou ajustamento adequado para a amostra e foi atestada a invariância de medida forte do modelo entre grupos distintos. A análise baseada em medidas externas mostrou que os fatores depressão, ansiedade e estresse apresentaram correlações positivas e moderadas-fortes com o domínio prejuízo emocional e correlações negativas e moderadas com o construto satisfação no trabalho. A análise do padrão de repostas aos itens comprovou a uniformidade e a estabilidade do instrumento para a amostra. Quanto aos sintomas, destaca-se a alta prevalência de níveis extremamente graves de depressão, ansiedade e estresse entre os participantes.

Conclusão  A DASS-21 apresentou adequada validade e confiabilidade para a avaliação de sintomas de depressão, ansiedade e estresse entre trabalhadores de enfermagem brasileiros.

Saúde ocupacional; Depressão; Ansiedade; Estresse psicológico; Psicometria; Escalas de Graduação Psiquiátrica; Profissionais de enfermagem; Inquéritos e questionários; Brasil

Resumen

Objetivo  Investigar las evidencias de validez de la Depression, Anxiety and Stress Scale (DASS-21) para analizar la prevalencia de síntomas de depresión, ansiedad y estrés en una muestra de trabajadores de enfermería brasileños.

Métodos  Estudio transversal con muestreo no probabilístico. Participaron en el estudio 4053 trabajadores de enfermería brasileños. Se evaluaron las propiedades psicométricas de la DASS-21 mediante el análisis de validez basada en le estructura interna (validez del constructo factorial, convergente y discriminante; invarianza del modelo factorial; fiabilidad), validez basada en las relaciones con medidas externas (validez convergente positiva y negativa de los factores depresión, ansiedad y estrés con los constructos daño emocional, satisfacción en el trabajo y variables individuales y laborales), validez basada en el patrón de respuesta a los ítems del instrumento (análisis de la función diferencial de los ítems en subgrupos distintos). Se comprobó la fiabilidad de los datos mediante la estimación del coeficiente alfa ordinal y la fiabilidad compuesta. La prevalencia de síntomas de depresión, ansiedad y estrés se estimó a partir de las recomendaciones de los autores originales de la DASS-21.

Resultados  Durante la validación de la estructura interna, el análisis factorial confirmatorio reveló que el modelo original de la DASS-21 presentó un ajuste adecuado para la muestra y se comprobó la invarianza de medida fuerte del modelo entre grupos distintos. El análisis basado en las medidas externas mostró que los factores depresión, ansiedad y estrés presentan correlaciones positivas y moderadas-fuertes con el dominio daño emocional y correlaciones negativas y moderadas con el constructo satisfacción en el trabajo. El análisis del patrón de respuestas de los ítems comprobó la uniformidad y la estabilidad del instrumento para la muestra. Respecto a los síntomas, se destaca la alta prevalencia de niveles extremadamente graves de depresión, ansiedad y estrés entre los participantes.

Conclusión  La DASS-21 presentó validez y fiabilidad adecuada para evaluar síntomas de depresión, ansiedad y estrés entre trabajadores de enfermería brasileños.

Salud laboral; Depresión; Ansiedad; Estrés psicológico; Psicometría; Escalas de Valoración Psiquiátrica; Profesionales de Enfermería; Encuestas y cuestionarios; Brasil

Introduction

Recent decades have shown a global concern regarding the high prevalence of psychological and mental disorders in the general population. This scenario was further aggravated by the COVID-19 pandemic. In this context, the World Health Organization (WHO) indicated depression as the most prevalent mental illness in the world in the coming years.(¹)

Among health and nursing professionals, it was recognized that the characteristics inherent to their work activities and environment in health institutions make them vulnerable to the occurrence of psychological and/or mental problems, and depression, anxiety, and stress were highlighted.(²-⁵) In this sense, scientific evidence(⁵-⁸) highlighted several factors related to the occurrence of mental disorders among nursing workers such as females,(⁵,⁶)presence of chronic diseases,(⁶)low self-efficacy, lack of social support, resilience,(⁶,⁷)and resources in the work environment,(⁷)long working hours, inadequate remuneration, and lack of professional recognition.(⁵,⁶,⁸)

The precariousness of the health work context was greatly aggravated by the COVID-19 pandemic, when nursing professionals faced significant changes in their work practices, including extra shifts, unpredictability of schedules, execution of atypical tasks, and reallocation of sectors and teams.(⁹) All these work and psychosocial factors generated negative impacts on the physical and mental health of these workers, highlighting the occurrence of symptoms of psychological suffering, fear, and anxiety,(¹⁰)as well as psychophysiological manifestations associated with depression and post-traumatic stress.(¹¹)

Aiming to evaluate symptoms of depression, anxiety, and stress, researchers developed the Depression, Anxiety, and Stress Scale (DASS-21).(¹²) This instrument was built from the theoretical assumptions of the Tripartite Model.(¹³) Their items were divided into two factors that characterize specific symptoms of depression and anxiety and one factor that brings together nonspecific and shared characteristics of these conditions (stress). According to the Tripartite Model, depression is characterized by feelings of hopelessness, sadness, worthlessness, and symptoms of psychomotor inhibition, anhedonia, apathy, and loss of appetite. Anxiety is characterized by physiological hyperarousal, hypervigilance, perception of threat or danger, and muscular tension and manifestation of symptoms of fear, panic, nervousness, avoidance, and instability.(¹³) Both factors share symptoms of psychomotor agitation, irritability, worry, low concentration, insomnia, fatigue, and feelings of helplessness, inferiority, guilt, and low self-esteem.(¹³)

DASS-21 has already been validated in different contexts and populations,(¹⁴-¹⁷)including samples of health professionals and nurses, especially during the COVID-19 pandemic.(⁴,¹⁸,¹⁹) In Brazil, DASS-21 was adapted and validated by Martins et al.(²⁰)among university students.

Therefore, the objective of this study was to investigate the validity evidence of the DASS-21 to analyze the prevalence of depression, anxiety, and stress symptoms in a sample of Brazilian nursing workers.

Methods

This was a cross-sectional study with non-probability convenience sampling. Data collection occurred between April and July 2022, using an electronic form created from the REDCap platform and sent by email by the Federal Nursing Council (COFEN) to registered nurses, technicians, and nursing assistants. A total of 779,337 e-mails were sent: 5979 professionals agreed to participate and 4053 of them completely responded to the form, comprising the sample for this study.

Inclusion criteria: to have a minimum professional experience of one year and be professionally working during the data collection period. Exclusion criteria: participants who did not fully respond to the psychometric instruments were excluded.

The data collection instrument in the sample was composed of two parts: 1) questions related to the individual and occupational characterization of participants (including mental health treatment) and evaluation of job satisfaction by answering the following question: On a 1-10 scale, indicate how satisfied you are with your nursing work (1: dissatisfied; 10: very satisfied); 2) Depression, Anxiety and Stress Scale (DASS-21),(¹²)composed of 21 items distributed in three factors (depression, anxiety, and stress), with a four-point Likert-type response scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time); Emotional Impairment scale, a domain of the Burnout Assessment Tool (BAT-general version),(²¹)composed of five items (related to feelings of anger, frustration, and irritability, exaggerated reactions, and inability to control one’s emotions) with a five-point Likert-type response scale, ranging from 1 (never) to 5 (always).

The psychometric properties of DASS-21 were assessed by analyzing the validity evidence based on the (i) internal structure of the instrument, (ii) relationships with external measures, and (iii) response process of the items.(²²)

Analysis of the internal structure comprised (a) assessment of the validity of the factorial, convergent, and discriminant construct; (b) invariance of the factorial model; and (c) analysis of data reliability. We highlight the initial analysis of the psychometric sensitivity of the items to confirm data normality by estimating the absolute values of skewness (sk) and kurtosis (ku), considered adequate if sk<3 and ku<7).(²³,²⁴)

Factorial validity was estimated using the Confirmatory Factor Analysis (CFA), Weighted Least Squares Means and Variances (WLSMV) estimation method, and the following goodness-of-fit indices: Comparative Fit Index (CFI); Tucker-Lewis Index (TLI); Root Mean Square Error of Approximation (RMSEA) with a 90% confidence interval; Standardized Root Mean Square Residual (SRMR); and factor weights (λ). Values of CFI and TLI >0.90, RMSEA<0.10, SRMR<0.08; λ≥0.50 were considered adequate.(²⁵,²⁶) Convergent construct validity was assessed by analyzing the Average Variance Extracted (AVE) of its factors (adequate if AVE ≥0.50).(²⁷) Discriminant construct validity was assessed by comparing the factors’ AVE (adequate when AVEi and AVEj ≥ρij2).(²⁷)

The Factorial invariance was confirmed through multigroup analysis between independent groups: test (n=2024) vs. validation (n=2029); male (n=543) vs. female (n=3484); individuals who reported undergoing psychological and/or mental health treatment (n=1076) vs. individuals who denied such treatment (n=2964). It was used the CFI difference test (ΔCFI) between the configural (M0), metric (M1), and scalar (M2) models, is considered adequate if ΔCFI<0.01.(²⁸)

Reliability was analyzed considering the ordinal alpha coefficient (α) and composite reliability (CR) of the DASS-21 factors (adequate if α and CR≥0.70).(²⁷)

Validity evidence based on external measures was estimated by analyzing the positive and negative convergent validity between the DASS-21 factors, the Emotional Impairment scale,(²¹)and the job satisfaction construct. It was expected that the symptoms of depression, anxiety, and stress would present positive associations with emotional impairment and negative associations with job satisfaction. Furthermore, correlations were tested between the DASS-21 factors and the following variables: gender, age, professional category, time working in nursing, weekly workload, and dual employment.

Validity evidence based on the response process of the items was assessed using the Differential Item Functioning (DIF) between two groups: individuals who underwent mental health treatment (group 1: mental health treatment; n=1076) and individuals who denied undergoing such treatment (group 2: no mental health treatment; n=2964). The Information-Weighted Mean Square (Infit), Unweighted Mean Square (Outfit), and Partial-Credit Model (PCM) indices. Infit and Outfit values between 0.5 and 1.5 were considered indicative of adequacy. The effect size of DIF was tested by evaluating the McFadden and Nagelkerke pseudo-R² coefficients, considering values <0.13 as negligible and the items with a significant total effect (p<0.01) as non-equivalent.(²⁹)

The DIF analysis is based on the Item Response Theory (IRT) which evaluates not only observable variables (items) but also considers the existence of latent traits (aptitudes) of individuals when they respond to the items of a psychometric instrument. The importance of DIF analysis is based on the possibility of identifying items that present differential functioning between distinct groups; hence, this analysis can indicate whether possible discrepancies between the responses of individuals belonging to different groups are associated with how the item operates the construct or whether they are related to external factors.(³⁰) It was also recognized that DIF analysis allows confirming (or refuting) the assessment of factorial invariance between distinct groups.(³¹)

Statistical analyses were performed using the R program, with the lavaan, SemTools, lordif, and eRm packages.(³²)

The prevalence of symptoms in the sample was calculated by point and 95% confidence interval, following the recommendations of the instrument’s original authors. Thus, the participants’ responses were initially added for each factor. The scores obtained were multiplied by two and the severity of participants’ symptoms was then classified as: Depression (>27: extremely severe; 27-21: severe; 20-14: moderate; 13-10: mild; 9-0: normal); Anxiety (>19: extremely severe; 19-15: severe; 14-10: moderate; 9-8: mild; 7-0: normal); Stress (>33: extremely severe; 33-26: severe; 25-19: moderate; 18-15: mild; 14-0: normal).(¹²)

This study was approved by the Research Ethics Committee (CAAE: 89678518.9.0000.5393). The standards of the National Health Council (Resolution 466/2012)(³³) and General Data Protection Law were followed. All participants signed the Informed Consent Form.

Results

Among the 4053 participants, a predominance of women (n=3484; 85.96%) with a mean age of 35.55 years (SD=8.93) was observed. Most participants included nurses (n=1925; 47.49%) and nursing technicians (n=1904; 46.92%) who worked in the nursing field within 1-10 years (n=2783; 68.68%) with working hours ≥40 h per week (n=2916; 71.95%), and without dual employment (n=2677; 65.8%). Regarding general health, 69.10% (n=2799) of participants reported psychological and/or mental problems; however, only 26.55% (n=1076) received mental health treatment. Regarding the origin of participants, they lived in the Southeast 3025 (74.64%), South 479 (11.82%), Northeast 327 (8.07%), Central-West 118 (2.91%), and North 88 (2.17%) of Brazil.

Regarding the validity of the internal structure of DASS-21, the psychometric sensitivity analysis showed a normal distribution of all items (skewness and kurtosis values ≤3 and ≤7, respectively). Table 1 shows the Confirmatory Factor Analysis (CFA) and assessment of DASS-21 reliability in different groups.

Table 1
Confirmatory Factor Analysis and DASS-21 reliability in different groups

Samples	n	Λ	TLI	CFI	RMSEA [90%CI]	SRMR	AVE	CR	α
DASS-21 Total	4053	0.640-0.904	0.975	0.977	0.058 [0.057-0.060]	0.032	0.629-0.697	0.921-0.941	0.919-0.938
DASS SOHM	4053	0.639-0.905	0.967	0.971	0.079 [0.077-0.081]	0.036	0.629-0.697	0.921-0.941	0.919-0.938
DASS Test	2024	0.638-0.908	0.972	0.975	0.075 [0.072-0.078]	0.035	0.626-0.699	0.920-0.941	0.918-0.939
DASS Validation	2029	0.641-0.902	0.965	0.969	0.080 [0.078-0.083]	0.039	0.626-0.699	0.920-0.941	0.919-0.938
DASS Female	3484	0.624-0.903	0.966	0.970	0.080 [0.078-0.082]	0.037	0.623-0.694	0.919-0.940	0.917-0.937
DASS Male	543	0.709-0.924	0.981	0.983	0.065 [0.059-0.070]	0.033	0.671-0.726	0.934-0.948	0.930-0.946
DASS Treatment	1076	0.614-0.880	0.958	0.963	0.082 [0.079-0.086]	0.045	0.606-0.667	0.914-0.933	0.908-0.928
DASS No treatment	2964	0.624-0.902	0.967	0.971	0.076 [0.073-0.078]	0.037	0.608-0.687	0.915-0.938	0.911-0.936

• DASS: Depression, Anxiety, and Stress Scale; SOHM: second-order hierarchical model; λ: factorial weights; TLI: Tucker-Lewis index; CFI: comparative fit index; RMSEA: root mean square error of approximation; 90%CI: 90% confidence interval; SRMR: standardized root mean square residual; AVE: average variance extracted; CR: composite reliability; α: ordinal alpha coefficient; DASS treatment: psychological and/or mental treatment; DASS no-treatment: no psychological and/or mental treatment.

An adequate fit of the original model was observed in sample subgroups: random (test vs. validation) and discriminant (female vs. male; psychological and/or mental treatment vs. no-treatment). We emphasize that the CFA of the original model confirmed the three-factor structure with 21 items; factor weights ≥0.64; and a strong correlation between the factors (r=0.84-0.89).

The convergent construct validity was shown to be adequate for all factors (AVE=0.629-0.697). However, the discriminant validity between factors (r²₌0.716-0.808) was not attested due to the high correlation between them. Adequate values of composite reliability and ordinal α of the factors were also observed, confirming the instrument reliability for the sample. After CFA, the invariance analysis confirmed strong-measurement invariance of the original DASS-21 model in all observed groups: 1) test (n=2024) and validation (n=2029): M₀₌0.972; M₁₌0.972; and M₂₌0.974 (∆CFI<0.01); 2) female (n=3484) and male (n=543): M₀₌0.972; M₁₌0.972; and M₂₌0.974 (∆CFI<0.01); 3) workers who underwent mental health treatment (n=1076) and workers who did not undergo treatment (n=2964): M₀₌0.970; M₁₌0.971; and M₂₌0.973 (∆CFI<0.01). The model fit in each group is shown in table 1.

Regarding validity based on external measures, statistically significant correlations between the depression, anxiety, and stress constructs and individual (gender; age) and occupational variables (professional category; working time in the nursing; weekly workload; and dual employment) were not confirmed. However, positive and moderate-strong correlations were observed between the emotional impairment domain and symptoms of depression (r=0.740; p<0.001), anxiety (r=0.695; p<0.001), and stress (r=0.793; p<0.001); and negative and moderate correlations were observed between the job satisfaction construct and symptoms of depression (r=-0.492; p<0.001), anxiety (r=-0.393; p<0.001), and stress (r=-0.434; p<0.001), confirming the initial assumptions of the study. We emphasize that the CFA of the Emotional Impairment scale was carried out, and an adequate fit was confirmed for the sample (n=3571; λ=0.782-0.900; CFI=0.990; TLI=0.981; RMSEA [IC90%]=0.160 [0.148-0.172]; and SRMR=0.027). We also highlight that, although the RMSEA value was >0.10, the influence of the large sample size concerning the Emotional Impairment scale (one-factor model with five items; low degrees of freedom) was recognized. In this case, the SRMR value confirmed the adequate model fit. The results of the DIF analysis of the DASS-21 items are shown in table 2, confirming the validity based on the response process of the items.

Table 2
Differential Item Functioning analysis between distinct groups (n=4040)

	Differential Item Functioning
Items	Group 1		Group 2		DIF	Pseudo R2
	Infit	Outfit	Infit	Outfit	p-value	McFadden	Nagelkerke
DASS1	1.114	1.138	1.197	1.330	0.101	0.000	0.001
DASS2	1.405	1.501	1.413	1.577	0.125	0.000	0.001
DASS3	1.047	1.183	1.240	1.822	0.238	0.000	0.001
DASS4	1.198	1.208	1.215	1.314	0.066	0.001	0.001
DASS5	1.176	1.216	1.105	1.093	0.088	0.001	0.001
DASS6	1.174	1.261	1.071	1.085	0.195	0.000	0.001
DASS7	1.169	1.197	1.122	1.100	0.001	0.002	0.002
DASS8	0.719	0.699	0.752	0.730	0.365	0.000	0.000
DASS9	0.788	0.771	0.857	0.830	0.000	0.003	0.003
DASS10	0.887	0.911	0.810	0.790	0.000	0.002	0.002
DASS11	0.948	0.952	0.900	0.887	0.445	0.000	0.000
DASS12	0.795	0.738	0.814	0.815	0.793	0.000	0.000
DASS13	0.756	0.742	0.743	0.718	0.117	0.000	0.000
DASS14	1.070	1.088	0.944	0.927	0.068	0.001	0.001
DASS15	0.783	0.733	0.781	0.683	0.020	0.001	0.001
DASS16	0.810	0.777	0.776	0.748	0.069	0.001	0.001
DASS17	0.899	0.902	0.855	0.830	0.000	0.002	0.002
DASS18	0.871	0.890	0.866	0.865	0.021	0.001	0.001
DASS19	0.967	0.948	1.009	0.997	0.980	0.000	0.000
DASS20	0.863	0.868	0.928	0.879	0.056	0.001	0.001
DASS21	0.926	0.899	0.966	0.893	0.378	0.000	0.000

• DIF: Differential Item Functioning; Group 1: psychological and/or mental treatment; and Group 2: no psychological and/or mental treatment.

The results showed that 100% of the items presented absolute Infit and Outfit values between 0.5-1.5, confirming suitability for PCM in both groups (group 1: psychological and/or mental treatment; n=1076; group 2: no psychological and/or mental treatment; n=2964). Items 7, 9, 10, and 17 presented DIF (p<0.01), indicating the non-equivalence of responses between groups. However, analysis of the DIF effect size between groups was not significant, as all items presented McFadden and Nagerkelke pseudo R²values <0.13.

Regarding the prevalence of symptoms, table 3 shows the predominance of normal levels of depression, anxiety, and stress among participants. However, we highlight the prevalence of extremely severe levels of depression and anxiety symptoms in the sample. When comparing genders, the results showed greater severity of symptoms among women, who had a higher prevalence of extremely severe levels of depression and anxiety compared to men.

Table 3
Prevalence of depression, anxiety, and stress symptoms among participants

Symptoms	Total DASS n(%)[95%CI]	female DASS n(%)[95%CI]	male DASS n(%) [95%CI]
Depression
Normal	1467(36.20) [34.72-37.68]	1240(35.59) [34.00-37.18]	222(40.88) [36.75-45.02]
Mild	445(10.98) [10.02-11.94]	387(11.11) [10.06-12.15]	56(10.31) [07.75-12.87]
Moderate	762(18.80) [17.60-20.00]	644(18.48) [17.20-19.77]	107(19.71) [16.36-23.05]
Severe	468(11.55) [10.56-12.53]	412(11.83) [10.75-12.90]	55(10.13) [07.59-12.67]
Extremely severe	911(22.48) [21.19-23.76]	801(22.99) [21.59-24.39]	103(18.97) [15.67-22.27]
Anxiety
Normal	1443(35.60) [34.13-37.08]	1197(34.36) [32.78-35.93]	240(44.20) [40.02-48.38]
Mild	232(5.72) [05.01-06.44]	202(5.80) [05.02-06.57]	30(5.52) [03.60-07.45]
Moderate	735(18.13) [16.95-19.32]	644(18.48) [17.20-19.77]	84(15.47) [12.43-18.51]
Severe	433(10.68) [09.73-11.63]	375(10.76) [09.73-11.79]	53(9.76) [07.26-12.26]
Extremely severe	1210(29.85) [28.45-31.26]	1066(30.60) [29.07-32.13]	136(25.05) [21.40-28.69]
Stress
Normal	1664(41.06) [39.54-42.57]	1400(40.18) [38.56-41.81]	257(47.33) [43.13-41.53]
Mild	458(11.30) [10.33-12.28]	388(11.14) [10.09-18.18]	65(11.97) [09.24-14.70]
Moderate	677(16.70) [15.56-17.85]	600(17.22) [15.97-18.48]	72(13.26) [10.40-16.11]
Severe	739(18.23) [17.04-19.42]	631(18.11) [16.83-19.39]	101(18.60) [15.32-21.88]
Extremely severe	515(12.71) [11.68-13.73]	465(13.35) [12.22-14.48]	48(8.84) [06.45-11.23]

• 95%CI: 95% confidence interval

Discussion

Assessment of the psychometric properties of DASS-21 in Brazilian nursing workers showed adequate validity and reliability of this instrument for the context studied. Furthermore, a high prevalence of depression, anxiety, and stress was observed in participants although normal levels predominated in all sample symptoms.

Confirmatory factor analysis revealed an adequate fit of the original DASS-21 model in the sample and confirmed the three-factor structure of 21 items, agreeing with the results of studies conducted in different contexts(¹⁴,¹⁶,¹⁷)and specifically among health professionals.(⁴,¹⁸,¹⁹)

Convergent validity analysis of the original DASS-21 model was shown to be adequate. However, attesting the discriminant validity of the model was not possible due to the strong correlation between factors, reinforcing the results of Brazilian studies.(⁴,²⁰) The strong-measure invariance of the original model was also observed in different subgroups, confirming the stability of the factorial structure in the sample and the instrument’s ability to homogeneously measure the symptoms of depression, anxiety, and stress in different groups.

The validity analysis based on external measures allowed to confirm the initial assumptions of the study, as positive correlations between the depression, anxiety, and stress symptoms and emotional impairment were found, indicating that the greater the severity of these symptoms, the greater the emotional damage of individuals. On the other hand, negative correlations between variables indicated that the greater the job satisfaction, the lower the severity of depression, anxiety, and stress symptoms among participants. These results confirm theoretical assumptions that positively associate the development of depression, anxiety, and stress symptoms with emotional impairment,(¹³,²¹)especially in nursing workers who faced challenging situations in the context of the COVID-19 pandemic.(³⁴,³⁵)

In theory, emotional impairment refers to a reduction in the ability to regulate emotional processes expressed by intense emotional reactions, feelings of being dominated by emotions, feelings of anger, frustration, and irritability.(²¹) Such processes are closely related to the assumptions of the Tripartite Model. They consider that emotional impairment or distress effectively contributes to developing anxiety and depression symptoms.(¹³) Similarly, a positive association between depression, anxiety, and stress symptoms and job satisfaction is supported by the Tripartite Model that relates the individuals’ positive affectivity to satisfaction in social activities, including work.(¹³)

Corroborating these findings, a study with Primary Health Care nurses identified a negative correlation between job satisfaction and stress, highlighting that job dissatisfaction increases the risk of emotional exhaustion.(³⁶) Similar results were found in a study conducted in India. It suggested that stressed, depressed, or anxious professionals tend to have lower job satisfaction.(³⁷) Another study showed an association between the development of depression, anxiety, and stress symptoms and the lack of professional recognition and satisfaction in nursing work.(⁸)

The process of evaluating the psychometric properties of DASS-21 was completed by validity analysis based on the item response pattern. The results of the analysis of differential item functioning (DIF) in distinct groups (individuals who underwent psychological and/or mental health treatment and individuals who did not undergo such treatment) confirmed the uniformity of DIF (reinforcing the invariance of the instrument) and the model stability in the sample, as the effect size analysis between groups showed to be negligible.

Regarding symptoms, the prevalence of severe and extremely severe levels of depression and anxiety in the sample (mainly in women) was significant, although the predominance of normal levels has been attested. It confirmed the results from international(³⁷,³⁸)and Brazilian studies(³⁹,⁴⁰)during and after the COVID-19 pandemic emphasizing the occurrence of psychological and mental impairments in Brazilian nursing workers. We highlight that data collection occurred from April to July 2022. This period was marked by a sharp global decline in the number of COVID-19 cases and is considered post-pandemic in many countries. However, it continued to disclose the physical, emotional, and psychological illness in health and nursing professionals due to their longstanding exposure to the high social and occupational demands during the pandemic.(⁵,⁴⁰)

In this sense, our results confirm the findings of a study that analyzed the prevalence of depression, anxiety, and stress in Indian nurses, in which 46.7% of participants had severe and extremely severe levels of anxiety.(³⁷) In Saudi Arabia, a study evaluated factors associated with the psychological well-being of nurses during the COVID-19 pandemic and identified the prevalence of moderate-to-severe levels of depression (46.1%), anxiety (48.0%), and stress (48.4%) among these professionals.(⁵)

Still on the prevalence of mental problems, recent report from the World Health Organization (WHO) exposed that almost one billion people in the world in 2019 lived with some type of mental disorder, which was the main cause of disability. According to this report, depression and anxiety increased by more than 25% in the first year of the pandemic alone. It has been estimated that depression is expected to become the most common illness in the world in the next 20 years, affecting more people than any other health problem.(¹)

Considering nursing workers’ health promotion and quality of care improvement, findings of systematic reviews on the worsening of the psychological and emotional suffering of these professionals highlighted the importance of a specialized provision of permanent psychological and emotional support and implementation of strategies to improve work environments in health services.(⁴¹,⁴²)

As limitations of the present study, we consider that (i) the cross-sectional design prevents establishing cause-effect relationships between symptoms of depression, anxiety, and stress and occupational and individual factors associated with mental illness in nursing workers and (ii) the lack of validity evidence based on the response process of DASS-21 items makes it impossible to compare the results obtained.

Therefore, conducting future longitudinal research is important not only to study the cause-and-effect relationships between variables involved in the physical and mental illness of nursing workers but also to propose effective interventions to promote the health and well-being of these professionals.

We also highlight the innovative character of validation studies of psychometric instruments that use robust analysis techniques based on item response patterns, thus contributing to the advancement of methodological knowledge related to the assessment of psychological constructs. In this aspect, the originality and relevance of this investigation were pointed out.

Conclusion

The results of this investigation confirmed the validity and reliability of DASS-21 in the studied context. It allowed the identification of predictive symptoms of mental illness in nursing workers, even not being a diagnostic tool. Furthermore, the prevalence of depression, anxiety, and stress symptoms among participants confirmed the impacts of the COVID-19 pandemic on the mental health of these professionals in Brazil. In this context, implementing strategies to mitigate individual and occupational factors related to the illness of nursing professionals by health service managers is essential.

Acknowledgments

To the Federal Nursing Council (COFEN) for their partnership, contribution to data collection, and support given during this research. Funding: Conselho Nacional de Desenvolvimento Científico e Tecnológico, CNPq (grant: 310705/2022-3).

References

• ¹ World Health Organization (WHO). World mental health report: transforming mental health for all. Geneva: WHO; 2022.
• ² Ghawadra SF, Abdullah KL, Choo WY, Phang CK. Psychological distress and its association with job satisfaction among nurses in a teaching hospital. J Clin Nurs. 2019;28(21-22):4087-97.
• ³ Al Maqbali M, Al Sinani M, Al-Lenjawi B. Prevalence of stress, depression, anxiety and sleep disturbance among nurses during the COVID-19 pandemic: A systematic review and meta-analysis. J Psychosom Res. 2021;141:110343.
• ⁴ Garcia GPA, Fracarolli IFL, Dos Santos HEC, et al. Depression, anxiety and stress in health professionals in the COVID-19 Context. Int J Environ Res Public Health. 2022;19 (7): 4402.
• ⁵ Al-Hadi Hasan A, Waggas D. Psychological wellbeing and associated factors among nurses exposed to COVID 19: findings from a cross sectional study. Int J Disaster Risk Reduct. 2022;76:103025.
• ⁶ Pouralizadeh M, Bostani Z, Maroufizadeh S, Ghanbari A, Khoshbakht M, Alavi SA, et al. Anxiety and depression and the related factors in nurses of Guilan University of Medical Sciences hospitals during COVID-19: A web-based cross-sectional study. Int J Afr Nurs Sci. 2020;13:100233.
• ⁷ Hu D, Kong Y, Li W, Han Q, Zhang X, Zhu LX, et al. Frontline nurses' burnout, anxiety, depression, and fear statuses and their associated factors during the COVID-19 outbreak in Wuhan, China: A large-scale cross-sectional study. EClinicalMedicine. 2020; 24:100424.
• ⁸ Assis BB, Azevedo C, Moura CC, Mendes PG, Rocha LL, Roncalli AA, et al. Factors associated with stress, anxiety and depression in nursing professionals in the hospital context. Rev Bras Enferm. 2022;75 (Suppl 3): e20210263.
• ⁹ Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors associated with mental health outcomes among health care workers exposed to coronavirus disease 2019. JAMA Netw Open. 2020;3(3):e203976.
• ¹⁰ Barros-Delben P, Cruz RM, Trevisan KR, Gai MJ, de Carvalho RV, Carlotto PA, et al. Saúde mental em situação de emergência: Covid-19. Rev Debates Psiquiatr. 2020;10(2):18-2.
• ¹¹ Paiano M, Jaques AE, Nacamura PA, Salci MA, Radovanovic CA, Carreira L. Mental health of healthcare professionals in China during the new coronavirus pandemic: an integrative review. Rev Bras Enferm. 2020;73(Suppl 2):e20200338.
• ¹² Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther. 1995;33(3):335-43.
• ¹³ Clark LA, Watson D. Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. J Abnorm Psychol. 1991;100(3):316-36.
• ¹⁴ Ahmed O, Faisal RA, Alim SM, Sharker T, Hiramoni FA. The psychometric properties of the Depression Anxiety Stress Scale-21 (DASS-21) Bangla version. Acta Psychol (Amst). 2022;223:103509.
• ¹⁵ Ali AM, Alameri RA, Hendawy AO, Al-Amer R, Shahrour G, Ali EM, et al. Psychometric evaluation of the depression anxiety stress scale 8-items (DASS-8)/DASS-12/DASS-21 among family caregivers of patients with dementia. Front Public Health. 2022;10:1012311.
• ¹⁶ Bibi A, Lin M, Zhang XC, Margraf J. Psychometric properties and measurement invariance of Depression, Anxiety and Stress Scales (DASS-21) across cultures. Int J Psychol. 2020;55(6):916-25.
• ¹⁷ Cao CH, Liao XL, Jiang XY, Li XD, Chen IH, Lin CY. Psychometric evaluation of the depression, anxiety, and stress scale-21 (DASS-21) among Chinese primary and middle school teachers. BMC Psychol. 2023;11 (1):209.
• ¹⁸ Al-Kalbani M, Al-Adawi S, Alshekaili W. Psychometric properties of the depression, anxiety, stress scales-21 (DASS-21) in a sample of health care workers in Oman. J Affect Disord Rep. 2022;10:100451.
• ¹⁹ Kakemam E, Navvabi E, Albelbeisi AH, Saeedikia F, Rouhi A, Majidi S. Psychometric properties of the Persian version of Depression Anxiety Stress Scale-21 Items (DASS-21) in a sample of health professionals: a cross-sectional study. BMC Health Serv Res. 2022; 22 (1):111.
• ²⁰ Martins BG, da Silva WR, Maroco J, Campos JA. Escala de Depressão, Ansiedade e Estresse: propriedades psicométricas e prevalência das afetividades [Internet]. J Bras Psiquiatr. 2019 ;68(1):32-41.
• ²¹ Schaufeli WB, Desart S, De Witte H. Burnout Assessment Tool (BAT)-Development, Validity, and Reliability. Int J Environ Res Public Health. 2020; 17 (24): 9495.
• ²² American Educational Research Association. American Psychological Association. National Council on Measurement in Education. Standards for Educational and Psychological Testing. Lanham (MD): American Educational Research Association; 2014.
• ²³ Kline RB. Principles and practice of structural equation modeling. New York: The Guilford Press; 2016.
• ²⁴ Marôco J. Análise de Equações Estruturais: fundamentos teóricos, software & aplicações. 2ª ed. Report Number: Pêro Pinheiro, Portugal; 2014.
• ²⁵ Arbuckle JL. AMOS 17.0 user's guide [Manual software]Chicago: Statistical Package for the Social Sciences; 2008.
• ²⁶ Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate Data Analysis. 7th ed. NJ: Prentice Hall; 2010.
• ²⁷ Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Mark Res. 1981;18(1):39-50.
• ²⁸ Nolte S, Elsworth GR. Factorial invariance. In: Michalos AC, editor. Encyclopedia of quality of life and well-being research. Dordrecht: Springer Netherlands; 2014. p. 2146-8.
• ²⁹ Choi SW, Gibbons LE, Crane PK. lordif: An R Package for Detecting Differential Item Functioning Using Iterative Hybrid Ordinal Logistic Regression/Item Response Theory and Monte Carlo Simulations. J Stat Softw. 2011;39(8):1-30.
• ³⁰ Pasquali L. TRI - Teoria de resposta ao item: teoria, procedimentos e aplicações. 1. Curitiba: Editora Appris; 2018.
• ³¹ Jaloto A. Funcionamento diferencial do item (DIF) e invariância da medida. In: Faiad, C, Baptista, MN, Primi, R, editores. Tutoriais em análise de dados aplicados à psicometria. São Paulo: Editora Vozes, 2021. p. 268-91.
• ³² The R Foundation. The R project for statistical computing. Vienna: R CORE TEAM, 2023.[cited 2022 Oct 30]. Available from: http://www.r-project.org/index.html
  » http://www.r-project.org/index.html
• ³³ Brasil. Conselho Nacional de Saúde. Resolução 466 de 12 de dezembro de 2012. Dispõe sobre as diretrizes e normas regulamentadoras de pesquisa envolvendo seres humanos. Diário Oficial da União. Brasília (DF), 13 junho 2013, seção I, p.59.
• ³⁴ Ribeiro CL, Maia IC, Pereira L P, Santos V F, Brasil RF, dos Santos JS, et al. Ansiedade e depressão em profissionais de enfermagem de uma maternidade durante a pandemia de COVID-19. Esc Anna Nery. 2022; 26(Spe):e20220041
• ³⁵ Liu SY, Kang XL, Wang CH, Chu H, Jen HJ, Lai HJ, et al. Protection procedures and preventions against the spread of coronavirus disease 2019 in healthcare settings for nursing personnel: lessons from Taiwan. Aust Crit Care. 2021;34(2):182-90.
• ³⁶ Garcia GP, Marziale MH. Satisfaction, stress and burnout of nurse managers and care nurses in Primary Health Care. Rev Esc Enferm USP. 2021;55:e03675.
• ³⁷ Kaushik A, Ravikiran SR, Suprasanna K, Nayak MG, Baliga K, Acharya SD. depression, anxiety, stress and workplace stressors among nurses in tertiary health care settings. Indian J Occup Environ Med. 2021;25(1):27-32.
• ³⁸ Pappa S, Ntella V, Giannakas T, Giannakoulis VG, Papoutsi E, Katsaounou P. Prevalence of depression, anxiety, and insomnia among healthcare workers during the COVID-19 pandemic: A systematic review and meta-analysis Brain Behav Immun. 2020 ;88:901-7. Correction in Brain Behav Immun. 2021 Feb;92:247.
• ³⁹ Appel AP, Carvalho AR, dos Santos RP. Prevalence and factors associated with anxiety, depression and stress in a COVID-19 nursing team. Rev Gaúcha Enferm. 2021;42(Spe):e20200403.
• ⁴⁰ Sandoval LB, Rodrigues KM, Júnior RR, Pereira JR, Dias CA, de Carvalho RL, et al. Anxiety disorder developed during the pandemic COVID-19 in nursing staff. Braz J Hea Rev. 2021;4(6):29225-4.
• ⁴¹ Woo T, Ho R, Tang A, Tam W. Global prevalence of burnout symptoms among nurses: A systematic review and meta-analysis. J Psychiatr Res. 2020;123:9-20.
• ⁴² Girma B, Nigussie J, Molla A, Mareg M. Occupational stress and associated factors among health care professionals in Ethiopia: a systematic review and meta-analysis. BMC Public Health. 2021;21(1):539.

Edited by

• Associate Editor:
  Thiago da Silva Domingos (https://orcid.org/0000-0002-1421-7468) Escola Paulista de Enfermagem, Universidade Federal de São Paulo, São Paulo, SP, Brazil

Publication Dates

• Publication in this collection
  02 Dec 2024
• Date of issue
  2025

History

• Received
  5 Dec 2023
• Accepted
  13 May 2024