Validation Process of a Work Engagement Scale for the Brazilian Population

Abstract

The objective of this study is divided into two parts: the first part aimed at presenting the process of translation and adaptation of Porto-Martins and Benevides-Pereira's (2008) version of the Utrecht Work Engagement Scale (UWES); the second part focused on the validation of the instrument considering samples from five distinct and independent studies, with N=1,934. The adaptation process was composed of 13 steps, which included, among other steps, translation and back translation. Descriptive, reliability and confirmatory factor analyses were performed. The results denoted statistical quality and agreement with the data in the official UWES manual. Therefore, the validity of the corresponding version of the UWES is concluded, and future studies are encouraged in order to solidify this relevant construct in the national context.

Keywords:
work engagement; psychological evaluation; occupational health psychology; UWES; organizational psychology

Resumo

O objetivo deste estudo é dividido em duas partes: na primeira, visou-se expor o processo de tradução e adaptação da versão de Porto-Martins e Benevides-Pereira (2008) do Utrecht Work Engagement Scale (UWES); na segunda parte, focou-se a validação do instrumento, considerando amostras de cinco estudos distintos independentes, com N=1934. O processo de adaptação foi composto por 13 passos, que incluíram, entre outras etapas, tradução e back translation. Foram realizadas análises descritivas, de confiabilidade e fatoriais confirmatórias. Os resultados denotaram qualidade estatística e consonância com os dados do manual oficial do UWES. Conclui-se, portanto, a validade da respectiva versão do UWES, assim como instigam-se futuros estudos, a fim de solidificar esse relevante constructo no contexto nacional.

Palavras-chave:
engajamento no trabalho; avaliação psicológica; psicologia da saúde ocupacional; UWES; psicologia organizacional

The objective of this study is divided into two parts: to share the translation and adaptation process of the Utrecht Work Engagement Scale for Students (UWES) carried out by Porto-Martins and Benevides-Pereira (2008) and to verify its psychometric quality.

The topic engagement or work engagement is considered recent and with growing interest in the literature (Kulikowski, 2017). The first scholar to conceptualize it was William A. Kahn, in the early 1990s (Salanova & Schaufeli, 2009; Schaufeli, 2012). The construct is considered a positive, cognitive and affective condition characterized by a high level of vigor, dedication, and absorption (Pocinho & Perestrelo, 2011; Schaufeli, 2012, 2016, 2017; Yulita, Dollard & Idris, 2017), always related to the world of work, motivational and social in nature, increasing occupational health (Salanova & Schaufeli, 2009).

The construct exerts a positive effect on both individual and collective levels and is closely connected with organizational development (Salanova & Schaufeli, 2009).

Regarding the measurement of engagement, the UWES is considered the international standard instrument (Salanova & Schaufeli, 2009), presenting versions in more than 19 languages, as well as being the most popular for its measurement (Kulikowski, 2017). This instrument analyzes the three dimensions (vigor, dedication, and absorption), in addition to presenting a global scale, which encompasses a general index of the construct and contemplates the three dimensions concomitantly (Salanova & Schaufeli, 2009; Schaufeli & Bakker, 2003).

Regarding the UWES, based on Pocinho and Perestrelo (2011), Schaufeli (2012, 2016, 2017), and Schaufeli and Bakker (2003), the dimensions are described as:

Vigor (VI - six items): characterized by high levels of energy, persistence, desire to strive at work, and mental resilience when considering organizational activities.

Dedication (DE - five items): related to being fully concentrated on doing the work, feeling inspired, proud, challenged by work activities, with a sense of objectivity, meaning, and enthusiasm.

Absorption (AB - six items): characterized by high concentration and happiness in performing the work and the feeling that time passes quickly.

The 17 items make up a seven-point Likert-type scale, ranging from 0, for "never/not at all", to 6, for "always/every day".

Historically, the study of engagement has focused on four major areas: individual engagement, organizational engagement, collective engagement, and its relationship with burnout syndrome. These aspects support the breadth of the concept and its importance for both organizations and the individual (Burton, Chen, Li & Schultz, 2017; Schaufeli & De Witte, 2017). Inclusively, these perspectives are in line with worker appreciation and form an important avenue for the promotion of organizational strategies, highlighting the relevance of engagement for the management field (Salanova & Schaufeli, 2009; Schaufeli, 2016).

As engagement is a construct of academic and organizational interest (Salanova & Schaufeli, 2009), it is considered important to evaluate the availability of information about the existing instruments, especially the version of Porto-Martins and Benevides-Pereira (2008), since this is considered the first, being present in the manual of the instrument and also in the version available on the website of the main author of the original version (http://www.wilmarschaufeli.nl/).

In order to map the current state of publications with the UWES within the Brazilian context, a search was conducted in scientific databases, using the search criteria: "engagement", "engagement" and "UWES". From the articles found, 120 articles were preliminarily analyzed. Of these, 23 were selected because they involved the application of the 17-item version of the UWES in Brazilian samples.

The characteristics of the two translation versions of the UWES found are described chronologically below:

1 - Porto-Martins and Benevides-Pereira (2008) version:

1. Machado (2010); Machado, Porto-Martins and Benevides-Pereira (2014), in complementary studies with N=212 workers - bus drivers and collectors. They showed Cronbach's alpha index > 0.7 (VI = 0.88; AB = 0.77; DE = 0.84) and positive and significant Pearson's correlations between r = 0.76 for VI((AB and r = 0.95 for VI((Engagement, all with p < 0.01.

2. Machado, Porto-Martins and Amorim (2012) with N=63 teachers, also with α > 0.7 (VI = 0.92; DE = 0.94; AB = 0.78; overall = 0.96).

3. Machado and Porto-Martins (2013, 2015), in complementary studies with a multi-occupational sample of N=701, showed α > 0.8 (VI = 0.89; DE = 0.93; AB = 0.83; global = 0.95), with positive and significant Pearson correlations between r = 0.82 for VI((AB and r = 0.95 for DE((Engagement, all with p < 0.01. The studies also revealed confirmatory factor analyses, with RMSEA = 0.10; NNFI = 0.90 and CFI = 0.91.

4. Teixeira et al. (2017) with N=36 physicians, with α indices > 0.80 (VI = 0.75; DE = 0.90; AB = 0.71; overall = 0.92).

5. Gonsalez et al. (2017) with N=82 professionals from health professional development and enhancement programs, showing α values ≥ 0.69 (VI = 0.81; DE = 0.80; AB = 0.69; overall = 0.90).

6. Hansen, Fabricio, Rotili, and Lopes (2018) with N=132 managers, found α ≥ 0.80 (VI = 0.93; DE = 0.92; AB = 0.80 and overall = 0.95).

7. Rodrigues, Mattos, Cardoso & Mancebo (2019) with N=210 federal servers at the Federal University of Pará (UFPA), a study that showed global α = 0.90

8. Alves, Mattos, Santiago & Mancebo (2020) with N=84 subjects with α >0.70 (VI = 0.84; DE = 0.9; AB = 0.72 and global = 0.93). Research that showed high internal consistency between the dimensions.

9. Porto-Martins, Machado and Camou (2020) with N=423 telecare workers, found in the tri-factor model CFI = 0.96 and CMIN/DF, strong internal correlation, α > 0.70 in the three dimensions and also temporal validation considering the interval between applications of one month.

10. Obregon, Schwaab, Lopes and Ceretta (2021) with N=1,511 respondents and α > 0.70 (VI = 0.81; DE = 0.86 and AB = 0.78).

11. Moreira Mattos, Mancebo and Corrêa (2021) with N=100 public servants and α > 0.70 (VI = 0.71; DE = 0.73 and AB = 0.76).

Four other studies were also found that did not address psychometric indicators of the UWES:

1. Freitas and Charão-Brito (2016) with N=40 multidisciplinary health professionals in a municipality in the Southern region.

2. Moura, Charão-Brito & Lopes (2017) with N=47 postgraduate professionals from a private higher education institution.

3. Pereira & Lopes (2019) with N=72 servers of a federal educational institution.

4. Santos, Quel Vieira & Rosini (2019) with N=185 managers.

2 - Version of Vazquez, Magnan, Pacico, Hutz and Schaufeli (2015), which also follows the structure of the original, three-dimensional version (with 17- and nine-item formats) and presents studies such as those of:

1. Magnan, Vazquez and Pacico (2016) and Vazquez et al. (2015) with N=1,167 workers. Further studies that showed positive and significant (p < 0.01) internal correlation of 0.81 for VI((Concentration and 0.82 for VI((DE, with α = 0.95 for the overall scale and confirmatory factor analyses of CFI > 0.96 and RMSEA = 0.10.

Studies were also found that did not address psychometric aspects:

1. Dalanhol, Freitas, Machado, Hutz and Vazquez (2017), with N=200 judicial officers from Porto Alegre and metropolitan region, not specifying reliability values and correlations.

2. Santos, Fiorentin, Stefano, and Abreu (2019) with N=78 industry professionals.

3. Mercali and Costa (2019) with N=506 higher education teachers.

4. Dantas and Guedes (2020) with N=302 police officers.

In the search process, studies were identified in which the version used to assess engagement was not presented, such as those of Castro et al. (2019) with 31 oral health professionals. Garcia, Pinto & Canille (2020) with 41 professionals from pediatric hospital inpatient units. Cunha and Fabricio (2020) with 67 workers and 12 commercial establishments in the city of Ijuí.

It is worth mentioning that studies were also located with misdescription of the UWES version used, as illustrated by the citations described below:

1. Moreira Mattos, Mancebo and Corrêa (2021, p. 6): "translated into Portuguese by Angst, Benevides-Pereira and Porto Martins (2009) and validated by Magnan, Vazquez and Pacico (2016)"

2. Alves, Mattos, Santiago, and Mancebo (2020, p. 101) "translated into Portuguese, in Brazil, by Angst, Benevides-Pereira, and Porto-Martins (2009) and subsequently validated by Vazquez et al. (2015)"

3. Rodrigues et al. (2019, p. 7) "translated into Portuguese by Angst, Benevides-Pereira and Porto-Martins (2009) and validated by Vazquez et al. (2015)."

It is possible to identify misconceptions when considering the version of Porto-Martins and Benevides-Pereira (2008) as being the same used by the authors Vazquez et al. (2015) when in reality they are different versions, both concerning the translation and validation process. Another point is to cite the version of the instrument by Porto-Martins and Benevides-Pereira (2008) as being that of Angst, Benevides-Pereira and Porto-Martins (2009) because the latter is the national version of the instrument manual, which contains in its appendices the version of Porto-Martins and Benevides-Pereira (2008). Misunderstandings, even if subtle, can hinder the study and development of the theme "engagement" in the Brazilian context, making it essential to clarify and disseminate data that clarify this scenario.

Next, the methodological procedures of the study are presented.

Method

The methodology related to the first part of the objective - translation and validation process of the UWES performed by Porto-Martins and Benevides-Pereira (2008) - will be presented in the results section.

The process of instrument adaptation can be considered relevant for the advancement of science, but it deserves care, since it requires attention, time, and dedicated people (Silva & Domingues, 2017), for providing benefits regarding the construction of a measure, as well as, for example, for enabling direct comparisons between instruments (Borsa et al., 2012; Pereira, Lam, & Gir, 2017; Silva & Domingues, 2017). In order to perform the corresponding translation and adaptation process, it was taken into account the fact that adjusting a psychological scale is a complex process, which requires planning and rigor to maintain the content for the new population (Borsa et al., 2012).

The steps of the process should consider translation procedures, synthesis of the translated versions, evaluation of the synthesis by expert judges, evaluation of the instrument by the target audience, back-translation and pilot study (Borsa et al., 2012; Pieri, Pires, Filgueiras & Oliva, 2017; Zumpano et al., 2017), the consensus among judges, semantic validation, and verification of internal consistency by Cronbach's alpha coefficient (Pereira et al., 2017). Furthermore, Borsa et al. (2012) highlight the possibility of increasing the process with the addition of the following steps: factor analysis of the data, conceptual evaluation of the items by the target population, and discussion with the author of the original instrument.

Considering the aforementioned authors, the adaptation process of this study, Part 1, was elaborated based on the following steps: (i) review of existing studies; (ii) authorization with the authors of the original version; (iii) alignment and design of the stages; (iv) preliminary translation; (v) first single version; (vi) cultural and semantic verification; (vii) first pre-test; (viii) first back-translation; (ix) consolidation of back-translation into a single version; (x) comparison between versions; (xi) second pre-test; (xii) overall analysis; (xiii) submitting to the author of the instrument.

After these procedures, Part 2 was structured to analyze the model validity of the UWES. Analyses were performed to assess the psychometric quality of this version of the instrument, such as descriptive analyses (mean, standard deviation, and Pearson correlation); reliability analysis (Cronbach's alpha); confirmatory factor analyses: Degrees of Freedom in a Chi-Square (CMIN/DF), Comparative Fit Index (CFI), Adjusted Goodness of Fit Index (AGFI), Root Mean Square Error of Approximation (RMSEA), Average Variance Extracted (AVE), Composite Reliability Coefficients (CR) and structural equation models. For the statistical analyses, Excel 2016, Statistical Package for the Social Sciences (SPSS) and AMOS software were used, the latter in version 22.

Part 2 was segmented into two stages: in the first stage, a sample of 5 independent studies was analyzed to verify the psychometric quality and characteristics of the data, as well as to compare them to other similar studies with national and international samples; the second stage included the stratification of the sample to verify the adequacy of the scale in relation to the gender of the participants and studies with different occupational groups.

Sample

The first part - the translation and validation process of the UWES of Porto-Martins and Benevides-Pereira (2008) - had a team of nine people, including researchers and translators, as well as the application of the instrument with 20 people throughout the stages.

In the second part, the sample used for validation was made up of N=1,934 workers, from 5 different and independent studies: Study 1 - N=701 workers from the industrial sector of Southern Brazil; Study 2 - N=423 telemarketing operators from the city of Curitiba; Study 3 - N=212 drivers and collectors of public transportation from Curitiba and the metropolitan region; Study 4 - N=120 university professors from several cities in Paraná; and Study 5 - N=478 flight attendants from several regions of Brazil.

The data from the studies had the following exclusion criteria: not completing any of the items of the UWES; answering all the questions equally and participants who expressed the desire not to include the data in the research. The sample size is adequate since it respected the minimum size of five times the number of items to be evaluated, as well as the minimum of 100 to 150 respondents when using the method of maximum likelihood (maximum likelihood) in the calculation of parameters (Hair, Black, Babin & Anderson, 2010).

Ethics

This study comes from different research projects, submitted and approved by Research Ethics Committees. All participants signed the Informed Consent Form (ICF), and the necessary feedback was given to the parties involved. Thus, the research respected CNS Resolution No. 466/2012 as well as the 35 ethical principles present in the Declaration of Helsinki, the latter present in the official website of the World Medical Association and considering integrity, voluntariness and knowledge of the objectives by the subjects and researchers involved.

Results and discussion

The results of the 13 stages performed in Part 1 of this study are described below:

First stage (review): a review of the construct "work engagement," its history, evaluation, concepts, and other related aspects, was conducted. With this review, it was possible to highlight the scarcity of Brazilian publications on the theme, which is supported by Porto-Martins et al. (2013) and Machado et al. (2014), reinforcing the importance of validation studies about the construct.

Second stage (authorization): the researcher responsible for the instrument, Professor Schaufeli from Utrecht University in the Netherlands, was contacted via e-mail to grant authorization to validate the instrument for the Brazilian population.

The third stage (alignment and team building): we studied the references about the steps recommended for adapting the scale and delineating the procedures to be adopted. After the design of the steps, the work team was formed, with:

N=1 "responsible researcher", graduated in Psychology, with knowledge of the construct engagement and fluent in Portuguese and English. This professional was responsible for conducting the entire process.

N=2 "expert researchers", with previous knowledge in the area of health, labor psychology, as well as fluency in Portuguese and English, but without previous contact with the UWES. These researchers were mainly responsible for supervising and monitoring the whole process.

N=6 "translators", who, like the "expert researchers", had prior knowledge of the health area, labor psychology, as well as fluency in Portuguese and English. They had no previous contact with the UWES and contributed, in particular, to the translations and back translation process.

After the formation of the team, the researcher in charge prepared a training session about the construct, with the objective of leveling the knowledge about work engagement, as well as explaining the steps described in the sequence.

Fourth stage (translation of the preliminary versions): the translation of the original version from English into Portuguese was carried out by three translators separately, generating three independent versions. This stage focused on checking the language of common and divergent aspects used by the three experts and took into account the recommendations of Borsa et al. (2012), which recommend, at this stage, the care not to perform a literal translation, since it may hurt the cultural, contextual and scientific aspects of the new target audience.

Fifth stage (first single version): the first single version was prepared by expert researcher 1, based on the three versions generated in the previous stage. After this procedure, a meeting was held between expert researcher 1 and the researcher in charge to verify cultural adaptation issues and make necessary adjustments. The step aimed to minimize semantic, conceptual, linguistic, and contextual discrepancies, generating the first single version (Borsa et al., 2012).

Sixth step (cultural and semantic verification): the verification of the previous step was deepened, now also focusing on cultural issues. This step was performed by the two expert researchers and the responsible researcher, being called "expert evaluation of the synthesis" by Borsa et al. (2012). At this point, it was decided to take the risk of dealing with redundancy when performing the translation, i.e., to insert explanatory terms in parentheses to avoid difficulty in understanding the items, a decision also advocated by other authors, such as Reichenheim and Moraes (2007), and Serralta, Nunes, and Eizirik (2007). These cases are presented below (from the original to the translation):

• Question 1: "I feel bursting with energy" for "I feel full of energy".

• Question 4: "At my job, I feel strong and vigorous" for "At work, I feel strong and vigorous (vitality)".

• Question 15: "At my job, I am very resilient, mentally" for "At my job, I am a mentally resilient person (versatile)".

Thus, of the 17 items, three had terms in parentheses in the Portuguese translation, in order to facilitate the understanding of the items in different contexts.

Seventh step (application and verification): the first pre-test of the Portuguese version was carried out in a sample of N=10 subjects, being N=5 psychologists with knowledge in the mental and labor health area and N=5 with no technical knowledge about the theme. This step was performed in order to verify the understanding of the instructions and items (Borsa et al., 2012). The application occurred at different times, but always in the presence of the responsible researcher. After the end of each application, the researcher interviewed the subjects in order to verify the degree of difficulty in understanding the questions. The ten participants claimed to have had no difficulty in understanding and filling out the instrument.

Eighth stage (back translation): the process of back translation of the Portuguese version into English was performed, considered a process of quality control verification (Borsa et al., 2012). This step was performed by the three translators who did not participate in the translation from English into Portuguese. One of the purposes of back translation was to detect errors or deviations from the original instrument, as advocated by Barbosa, Steiner-Oliveira and Gavião (2010). This verification was made by the responsible researcher, together with expert researchers, who found a high similarity between the three versions resulting from back translation and the original English version. It is noteworthy that back translation is also used to ensure that the content of the instrument in the second language reflects the content of the original version (Silva & Domingues, 2017).

Ninth stage (single version back translation): a single version in English was structured by the two expert researchers. This step aimed to synthesize the versions of the back translation into a single version, favoring the verification of common and predominant elements in the three versions, enabling the investigation of possible inconsistencies in them.

The tenth stage (comparison): the two expert researchers and the responsible researcher compared the single version of the back translation into English with the original version of the UWES (also in English). The three researchers concluded that the versions were similar, noting only the presence of redundant bracketed terms in items 1, 4, and 15, as expected.

Eleventh step (application of back translation): the version of back translation was then applied in English in a sample of ten subjects, different from the subjects of the seventh step, being N=5 psychologists with knowledge in the area of mental and occupational health and N=5 without technical knowledge in this area, all fluent in English, with the same purpose of that step.

The application occurred at different times, but always in the presence of the responsible researcher. After the end of each application, the researcher interviewed the subjects in order to verify the degree of difficulty in understanding the questions. The ten subjects claimed to have had no difficulty in understanding, as well as in filling out the instrument. With this result, it was considered that the translation and back translation process by the team managed to maintain the original characteristics of the instrument.

Twelfth stage (general analysis): a meeting was held with all those directly involved - one researcher, two expert researchers, and six translators. After reviewing and discussing the stages and the results, it was concluded that there was no need to change the structured version in Portuguese.

Thirteenth stage (sending to the author): to conclude the process, the Portuguese version was sent, via e-mail, to Professor Wilmar Schaufeli, who approved the translation and published it on his personal website (http://www.wilmarschaufeli.nl/).

Regarding Part 2 of this manuscript - referring to the psychometric validation process of the scale - the total sample was N=1,934 workers. For comparison of results, we used as parameter mainly the data from the manual of the instrument (Schaufeli & Bakker, 2003), which counted N=12,161.

Regarding the results of the weighted averages of the UWES scales, it is noteworthy that in the sample of this study there were higher levels in the three dimensions and, consequently, of the global scale when compared to the data from the manual. These values are described in detail in Table 1.

Table 1.
Weighted averages and standard deviation of the UWES scales in the sample of this study (N=1,934) and the manual (N=12,161).

Scale	Average	DP	Manual Average	Manual DP
VI	4.36	1.27	4.24	1.09
DE	4.45	1.43	4.33	1.36
AB	4.05	1.29	3.77	1.28
Global	4.28	1.25	4.10	1.11

• Note: VI = vigor. DE = dedication. AB = absorption. SD = standard deviation.

As shown in Table 1, the Brazilian study presented higher means in all scales when compared to the manual. Also, in both studies, the levels of DE were higher in relation to the other dimensions, followed by VI, and finally AB.

When the means of the items are analyzed separately, we highlight items 17, with the highest weighted mean (4.80) and lowest standard deviation (1.37), and 16, which inversely had the lowest mean (3.31) and greatest oscillation (SD = 1.93), i.e., the item with the highest mean had a behavior with greater similarity of response among the sample, unlike the item with the lowest mean, according to Table 2. The identification of these averages makes it possible to identify the behavior of the items in an isolated manner, as well as in relation to their corresponding dimension.

Table 2.
Item averages and standard deviation.

Item	Dimension	Average	DP
1 - In my work, I feel full of energy.	VI	4.22	1.56
2 - I find that the work I do is full of meaning and purpose.	DE	4.63	1.61
3 - "Time flies" when I am working.	AB	4.46	1.68
4 - At work, I feel strong and vigorous (vitality).	VI	4.20	1.55
5 - I am enthusiastic about my work.	DE	4.33	1.64
6 - When I'm working, I forget everything around me.	AB	3.63	1.86
7 - My work inspires me.	DE	4.20	1.75
8 - When I get up in the morning, I feel like going to work.	VI	4.23	1.70
9 - I feel happy when I work intensely.	AB	4.25	1.66
10 - I am proud of the work I do.	DE	4.74	1.56
11 - I feel involved with the work I do.	AB	4.81	1.50
12 - I can continue working for long periods.	VI	4.14	1.71
13 - For me, my work is challenging.	DE	4.35	1.74
14 - I "let myself go" with my work.	AB	3.86	1.83
15 - In my work, I am a mentally resilient (versatile) person.	VI	4.58	1.46
16 - It is difficult to disconnect from work.	AB	3.31	1.93
17 - At work, I am persistent, even when things are not going well.	VI	4.80	1.37

• Note: SD = standard deviation. VI = vigor. DE = dedication. AB = absorption.

Regarding the correlation between all dimensions, they were positive and significant (p < 0.01), being: AB-VI (r = 0.82); DE-AB (r = 0.82); DE-VI (r = 0.87); global-VI (r = 0.95); global-AB (r = 0.94) and global-DE (r = 0.95). Furthermore, the positive and significant internal correlation was confirmed when the Intraclass Correlation Coefficient (ICC) was calculated. These data denote internal consistency of the instrument, as well as convergence with the theoretical assumption and the data found in the manual, showing that the version of Porto-Martins and Benevides-Pereira (2008) maintained the same pattern of internal consistency as the original English version, which is is in line with those found by other studies that used the version of Porto-Martins and Benevides-Pereira (2008) as Alves, Mattos, Santiago, Mancebo (2020); Machado (2010); Machado, Porto-Martins and Benevides-Pereira (2014); Machado and Porto-Martins (2013, 2015); Porto-Martins, Machado and Camou (2020). The results described above also converge with the study of the national version by Vazquez et al. (2015), which corroborates the premise that engagement is a single construct, but it can be segmented into three dimensions.

When reliability was verified through Cronbach's alpha, all indices were adequate: VI (α = 0.90), DE (α = 0.91), AB (α = 0.83), and global (α = 0.95), meeting the criterion of α > 0.70 (Hair et al., 2010; Pereira et al., 2017). Results aligned with other studies, which presented all dimensions above 0.70 and AB with the lowest value among the dimensions Machado (2010); Machado, Porto-Martins, and Benevides-Pereira (2014); Machado, Porto-Martins, and Amorim (2012); Teixeira et al. (2017); Hansen, Fabricio, Rotili, and Lopes (2018); Alves, Mattos, Santiago, and Mancebo (2020); Obregon, Schwaab, Lopes, and Ceretta (2021). Regarding the original version, the data are similar to that presented in the manual of the instrument (Schaufeli & Bakker, 2003), since in this one the results are VI (α = 0.82), DE (α = 0.89), AB (α = 0.83) and global (α = 0.93), data also aligned with the version of Vazquez et al. (2015) that obtained α = 0.89 for the global scale.

Regarding the confirmatory factor analyses, a three-factor model was structured, considering the VI, DE and AB dimensions, as well as a one-factor model was structured, considering engagement as a global scale. The calculations of both models were structured in line with the theoretical premise that the UWES is a three-factor instrument, but can also be used to verify the global scale of engagement (Schaufeli & Bakker, 2003; Porto-Martins, Machado & Camou, 2020).

The three-factor and one-factor models presented satisfactory results for the analyses performed, which contemplated the following fit indices and parameters: CFI > 0.90 (Hair et al., 2010), AGFI > 0.80 (Hair et al., 2010) and RMSEA < 0.80 (Byrne, 2010). On the other hand, only CMIN/DF, which should be less than 5.00 (Hair et al., 2010), did not meet the parameter used, a result similar to that of other studies, such as the national one by Vazquez et al. (2015) and the international ones by Schaufeli and Bakker (2003) and Seppala et al. (2009), as it can be seen in Table 3.

Table 3.
Comparison of the confirmatory factor analysis between the three and one-factor models.

Study	Factor	N	X2	GL	X2/GL	CFI	AGFI	RMSEA
This Study	1	1.934	1,346.43	119	11.32	0.93	0.84	0.09
	3		1,313.29	116	11.32	0.93	0.84	0.09
Vazquez et al. (2015)	1	1.167	1,804.00	119	15.16	0.96	--	0.10
	3		1,667.99	122	13.67	0.96	--	0.10
Schaufeli e Bakker (2003)	1	2.313	3,554.65	119	29.87	0.87	0.78	0.11
	3		1,859.93	116	16.03	0.91	0.83	0.10
Seppala et al. (2009)	1	2,314	1,483.74	119	12.47	0.93	--	0.07
	3		1,311.81	116	11.31	0.94	--	0.07
Torrente, Salanova, Lorens e Schaufeli (2013)	1	54	342.74	135	2.54	0.73	--	0.17
	3		311.85	132	2.36	0.77	--	0.16
Simbula, Guglielmi, Schaufeli e Depolo (2013)	1	488	805.30	119	6.77	0.86	--	0.11
	3		668.64	116	5.76	0.89	--	0.10
Schaufeli, Salanova, González-Romá e Bakker (v2002)	1	933	1,237.47	238	5.19	0.85	0.80	0.07
	3		952.66	232	4.11	0.90	0.85	0.05
Porto-Martins, Machado e Camou (2020)	1	423	-		3.500	0.94	0.86	0.08
	3		-		3000	0.96	0.89	0.07

• Notes: X² = chi-square; GL=degree of freedom.

In order to complement the data from the confirmatory factor analyses, the AVE and CR indexes were calculated for both models:

• AVE = 0.60 (VI); 0.68 (DE); 0.47 (AB) and 0.56 (overall - single-factor), the reference value being greater than 0.50 (Hair et al., 2010).

• CR = 0.90 (VI); 0.91 (DE); 0.84 (AB) and 0.96 (global - single-factor), the reference value being greater than 0.70 (Hair et al., 2010).

Regarding factor loadings (β), there was a range between 0.42 (item 16) and 0.89 (items 4 and 7) for the three-factor model, that is, all values were higher than the parameter greater than 0.40 (Gomes, Dagostini, Silva & Cunha, 2012). As for the one-factor model, the variation occurred between item 16 (β = 0.36) and item 7 (β = 0.88), that is, with the presence of only item 16 below the mentioned parameter, a fact corroborated by the UWES manual and by a study by Ramalho, Teles, Ramalho, and Ribeiro (2017), when addressing the issue of item quality. In general, high factor loadings indicate adequate item strength in relation to their corresponding dimensions (latent variables).

Concerning covariances, they were all significant and positive between the three dimensions of engagement and higher than 0.97 VI((DE / DE((AB / VI((AB, indicating a solid association between the variables. In Figure 1, both models are displayed, with all their corresponding β and covariances.

Figure 1.
Suggested title: Standardized results of the UWES three-factor and one-factor models.

In summary, considering the data from the manual and the studies cited, it is considered that the version carried out for the Brazilian population presented adequate data for both models (three-factor and one-factor), both regarding the fit indices and the regressions and covariances.

With regard to the second stage of Part 2 of this study, as described in the method, it was structured with the objective of expanding the analysis of data from the three-factor model, since it is considered to be more adjusted than the one-factor model, both in this study and in the manual. In order to do so, the sample was stratified by gender and study, with the purpose of measuring the model compared to different professional categories.

As for gender, there was a sample loss due to the partial lack of this information in the studies, which implied an N=1,853. Of these, most were women (N=1,138). For both genders, the indices were adequate: CFI = 0.94 for both; AGFI = 0.87 for men and 0.86 for women; RMSEA = 0.08 for both. Regarding the dimensions, the CR values for men (VI = 0.88; DE = 0.89; AB = 0.78) and women (VI = 0.91; DE = 0.92; AB = 0.86), as well as the AVE for men (VI = 0.57; DE = 0.63; AB = 0.40) and women (VI = 0.62; DE = 0.71; AB = 0.52), met the criterion of CR > 0.70 and AVE > 0.50, respectively, with the only exception for men in the AB dimension, which showed AVE = 0.40. For both stratifications, Cronbach's alpha values were greater than 0.70. The regressions ranged between 0.28 (item 16) and 0.87 (item 10) for men and between 0.40 (item 16) and 0.90 (item 7) for women. It is considered that all β were adequate (> 0.40), except for item 16 in the model calculated for males. These data allow us to attest to the psychometric quality of the UWES for both the male and female population of this study.

For the data stratified by study, the total N remained unchanged. It is noteworthy that in different cases the CR values were lower than the parameter adopted in the present study, especially in the AB scale. The AGFI values of studies 1 and 4 also presented values below the parameter. On the other hand, when a general analysis of all analyzed indexes is made, it is possible to infer that most of them met the parameters used for the study, according to Table 4. In all the studies mentioned, Cronbach's alpha values were higher than 0.70. As for the regressions, the averages of the regressions were adequate in all 5 studies, ranging from ΔAB=0.58 in study 3 to ΔDE= 0.86 in study 1.

Table 4.
Results of the confirmatory factor analyses stratified by study.

E	CFI	AGFI	R	Δ Beta				AVE				CR
				VI	AB	DE	U	VI	AB	DE	U	VI	AB	DE	U
1	0.89	0.73	0.12	0.76	0.66	0.86	0.76	0.89	0.83	0.93	0.96	0.59	0.46	0.74	0.59
2	0.96	0.85	0.08	0.77	0.65	0.79	0.73	0.90	0.82	0.90	0.95	0.60	0.45	0.64	0.56
3	0.93	0.81	0.08	0.74	0.58	0.73	0.68	0.88	0.76	0.86	0.94	0.56	0.37	0.56	0.49
4	0.87	0.67	0.11	0.69	0.61	0.78	0.69	0.85	0.79	0.88	0.94	0.51	0.40	0.61	0.50
5	0.93	0.87	0.73	0.69	0.57	0.77	0.67	0.85	0.75	0.88	0.94	0.49	0.37	0.60	0.48

• Notes: E = study; R = RMSEA; U = global scale.

Finally, in general, the instrument is considered adequate for diversified samples with regard to sex and occupation, data that indicate the possibility of the future use of this version of the UWES for different studies, in different occupational contexts.

Final considerations

It is considered that the objective of the study was met and, as the main aspect, it can be concluded that the adapted version of the UWES by Porto-Martins and Benevides-Pereira (2008) presented adequate psychometric qualities, is aligned with the other national and international versions of the instrument, as well as follows the main steps recommended for the scale validation. In addition, it contributes to the elucidation of the different versions of the UWES present in the Brazilian context and their characteristics.

It is worth noting that defining the adequacy of a model is a complex decision and there is no standard (Byrne, 2010). However, the results of the UWES applications in this study presented, for the most part, indices considered adequate, according to the parameters used for both the one-factor and three-factor models of work engagement. Even these data were maintained by stratified analyses by sex and occupational category.

As main limitation, we highlight the fact that the sample consisted mainly of participants from Southern Brazil, denoting the need to expand the application of the instrument to other regions, as well as to increase the number and occupational diversity of the sample. It is also emphasized that it is important to conduct future analyses regarding the use of the instrument, such as structuring cross-cultural studies in order to verify the (in)variance of its structure and parameters when applied to different groups and cultural contexts (Borsa et al., 2012) as well as to conduct predictive and discriminant validity investigations of engagement.

Finally, we highlight that work engagement is a topic of academic and organizational interest, being considered an essential aspect for strategic management practices and occupational health, and the UWES is one of the quantitative tools that contribute to this process. The evaluation of this construct provides important subsidies both for translation processes of psychological instruments and for the promotion of prevention and/or intervention actions in companies.

References

• Agnst, R., Benevides-pereira, A. M. T. & Porto-Martins, P. C. (2009). Tradução do manual do UWES Work Engagement Scale In: Schaufeli, W. B. & Bakker, A. B. 2003. Recuperado de http://www.wilmarschaufeli.nl/downloads/test-manuals-2/
  » http://www.wilmarschaufeli.nl/downloads/test-manuals-2/
• Alves, I. M. P., de Mattos, C. A. C., Santiago, A. M., & Mancebo, C. H. A. (2020). Engajamento no trabalho da administração pública: uma pesquisa no poder judiciário no estado do Pará, Brasil. Revista de Administração, Contabilidade e Economia da Fundace, 11(1). DOI: http://doi.org/10.13059/racef.v11i1.675
  » http://doi.org/10.13059/racef.v11i1.675
• Barbosa, T. S., Steiner-Oliveira, C. & Gavião, M. B. D. (2010). Tradução e adaptação brasileira do Parental-Caregiver Perceptions Questionnaire (P-CPQ). Saúde e Sociedade, 19 (3), 698-708.
• Borsa, J. C., Damásio, B. F. & Bandeira, D. R. (2012). Adaptação e Validação de Instrumentos Psicológicos entre Culturas: Algumas Considerações. Paidéia, 22 (53), 423-432.
• Burton, W. N., Chen, C. Y., Li, X. & Schultz, A. B. (2017). The Association of Employee Engagement at Work With Health Risks and Presenteeism. JOEM, 20 (10), 1-5.
• Byrne, B. M. (2010). Structural equation modeling with Amos: Basic concepts, applications, and programming. New York: Routledge Taylor & Francis.
• Castro, J. R., Gazetta, C. E., da Silva, A. G., Sodré, P. C., & Lourenção, L. G. (2019). Estresse ocupacional e engajamento em profissionais de saúde bucal. Revista Brasileira em Promoção da Saúde, 32. DOI: https://doi.org/10.5020/18061230.2019.9157
  » https://doi.org/10.5020/18061230.2019.9157
• Da Cunha, L. G., & Fabricio, A. (2020). Síndrome de Burnout e Engajamento no Trabalho: um Estudo a Partir de Trabalhadores do Comércio. Saber Humano: Revista Científica da Faculdade Antonio Meneghetti, 10(16). DOI: https://doi.org/10.18815/sh.2020v10n16.403
  » https://doi.org/10.18815/sh.2020v10n16.403
• Da Silva Pereira, A., & Lopes, D. D. (2019). Comprometimento e Entrincheiramento Organizacional e suas relações com o Engajamento no Trabalho: um estudo com servidores técnico-administrativos de uma Instituição Federal de Ensino. Desenvolvimento em Questão, 17(48), 139-158. DOI: https://doi.org/10.21527/2237-6453.2019.48.139-158
  » https://doi.org/10.21527/2237-6453.2019.48.139-158
• Dalanhol, N., de Freitas, C. P. P., de Lara Machado, W., Hutz, C. S., & Vazquez, A. C. (2017). Engajamento no trabalho, saúde mental e personalidade em oficiais de justiça. Psico, 48(2), 109-119.
• Dantas Guedes, H., & Guedes Gondim, S. M. (2020). Trabalho emocional e engajamento no trabalho em policiais militares. Quaderns de psicologia, 22(2). DOI: https://doi.org/10.5565/rev/qpsicologia.1584
  » https://doi.org/10.5565/rev/qpsicologia.1584
• De Freitas, L. A. V., & Charão-Brito, L. (2016). Engajamento no Trabalho: Um Estudo em Profissionais Multidisciplinares de Saúde em um Município da Região Sul. Revista Cesumar - Ciências Humanas e Sociais Aplicadas, 21(2), 407-419.
• De Souza Santos, J., Fiorentin, B. E., Stefano, S. R., & de Abreu, B. B. (2019). Engajamento no trabalho: Uma análise dos profissionais de indústrias. Research, Society and Development, 8(11). DOI: e478111469-e478111469 .
  » https://doi.org/e478111469-e478111469
• Garcia, L. G. G., Pinto, M. H., & da Silva Canille, R. M. (2020). Engajamento do profissional da enfermagem no trabalho com crianças em tratamento oncológico. Enfermagem em Foco, 11(5), p152-158.
• Gomes, G., Dagostini, L., Silva, J. C. & Cunha, P. R. (2012). Equações estruturais aplicadas ao grau de satisfação dos estudantes do Curso de Ciências Contábeis: estudo em uma faculdade do sudoeste do Paraná. VII Seminário de Ciências Contábeis FURB, Blumenau.
• Gonsalez, E. G., Lourenção, L. G., Teixeira, P. R., Rotta, D. S., Gazetta, C. E., Beretta, D., & Pinto, M. H. (2017). Engagement no trabalho em profissionais de programas de aprimoramento profissional em saúde/Work engagement in employees at professional improvement programs in health. Cadernos Brasileiros de Terapia Ocupacional, 25(3), 509-517.
• Hair, J. F., Black, W. C., Babin, B. J. & Anderson, R. E (2010). Multivariate data analisys Englewood: Prentice Hall.
• Hansen, R., Fabricio, A., Rotili, L. B., & Lopes, L. F. D. (2018). Inteligência Emocional e Engajamento no Ambiente de Trabalho: Estudo Empírico a Partir de Gestores e Equipes. Revista Gestão Organizacional, 11(1). DOI: 10.22277/rgo.v11i1.3980.
  » https://doi.org/10.22277/rgo.v11i1.3980.
• Kulikowski, K. (2017). Do we all agree on how to measure work engagement? Factorial validity of Utrecht Work Engagement Scale as a standard measurement tool: a literature review. International Journal of Occupational Medicine and Environmental Health, 30, 2, 161-175. DOI: https://doi.org/10.13075/ijomeh.1896.00947
  » https://doi.org/10.13075/ijomeh.1896.00947
• Machado, P. G. B. & Porto-Martins, P. C. (2013). Condições organizacionais enquanto terceiras variáveis entre burnout e engagement. Diaphora 13(1), 35-44.
• Machado, P. G. B. (2010). Engagement e burnout em motoristas e cobradores de ônibus. In: Jornada de Saúde Mental e Psicanálise da PUC-PR, V, Anais...: Curitiba. Disponível em Disponível em http://www2.pucpr.br/reol/index.php/jm?dd1=4427ⅆ99=view Acesso em: 23 jun. 2013.
  » http://www2.pucpr.br/reol/index.php/jm?dd1=4427ⅆ99=view
• Machado, P. G. B., & Porto-Martins, P. C. (2015). Proposta de protocolo: indicadores de saúde-enfermidade laboral/Proposition of a laboural health/illness protocol. Trabalho & Educação, 24(2), 281-293.
• Machado, P. G. B., Porto-Martins, P. C. & Amorim, C. (2012). Engagement no trabalho entre profissionais da educação. Intersaberes, 7 (13), 193-214.
• Machado, P. G. B., Porto-Martins, P. C. & Benevides-Pereira, A. M. T. (2014). Engagement no trabalho em profissionais de uma empresa de transporte público urbano. Psicologia Argumento, 32(79), 175-185.
• Magnan, Emília dos Santos, Vazquez, Ana Claudia Souza, Pacico, Juliana Cerentini, & Hutz, Claudio Simon. (2016). Normatização da versão Brasileira da Escala Utrecht de Engajamento no Trabalho. Avaliação Psicológica, 15(2), 133-140.
• Martins, V., Costa, L. V., & Siqueira, M. M. M. (2015). O impacto do comprometimento afetivo e do engajamento no trabalho sobre os comportamentos de cidadania organizacional. Revista de Administração, Contabilidade e Economia da Fundace, 6(2).
• Mercali, G. D., & Costa, S. G. (2019). Antecedentes do engajamento no trabalho dos docentes de ensino superior no Brasil. RAM. Revista de Administração Mackenzie, 20(1). DOI: 10.1590/1678-6971/eRAMG190081.
  » https://doi.org/10.1590/1678-6971/eRAMG190081
• Moreira, M. L. R., de Mattos, C. A. C., Mancebo, C. H. A., & de Castro Corrêa, A. (2021). Engajamento no trabalho: uma pesquisa com servidores públicos estaduais. Navus-Revista de Gestão e Tecnologia, 11, 1-15. DOI: http://doi.org/10.20503/recape.v10i1.44284
  » http://doi.org/10.20503/recape.v10i1.44284
• Moura, L. R. N. I., Charão-Brito, L., & Lopes, L. F. D. (2017). Vigor, dedicação, absorção: uma análise da percepção de profissionais pós-graduandos sobre engajamento no trabalho. Revista Cesumar-Ciências Humanas e Sociais Aplicadas, 22(2), 229-245.
• Obregon, S. L., Schwaab, K. S., Lopes, L. F. D., & Ceretta, P. S. (2021). Engajamento no trabalho e síndrome de burnout: uma análise estrutural com abordagem quantílica. Revista Alcance (Online), 28(1), 67-81.
• Pereira, F. M. V., Lam, S. C., & Gir, E. (2017). Adaptação Cultural e Confiabilidade da Compliance with Standard Precautions Scale (CSPS) para enfermeiros no Brasil. Revista Latino-Americana de Enfermagem, 25, 2850.
• Pieri, R., Pires, P., Filgueiras, A., & Oliva, Â. D. (2017). Adaptação Transcultural e Validação de Conteúdo do Questionário de Coesão no Esporte Infantil para o Português do Brasil. Revista Brasileira de Psicologia do Esporte, 6(2), 14-25.
• Pocinho, M. & Perestrelo, C. X. (2011). Um ensaio sobre burnout, engagement e estratégias de coping na profissão docente. Educação e Pesquisa, 37 (3), 513- 528.
• Porto-Martins, P. C., Machado, P. G. B., Benevides-Pereira, A. M. T. (2013). Engajamento no trabalho: uma discussão teórica. Fractal, 25 (3), 629-644.
• Porto-Martins, P. C; Benevides-Pereira, A. M. T. (2008). Work & Well-being Survey (UWES) Questionário do Bem-estar e Trabalho. Recuperado de http://www.wilmarschaufeli.nl/publications/Schaufeli/Tests/UWES_BRA_17.pdf
  » http://www.wilmarschaufeli.nl/publications/Schaufeli/Tests/UWES_BRA_17.pdf
• Porto-Martins, Paulo Cesar, Machado, Pedro Guilherme Basso, & Viacava, Juan José Camou. (2020). Engajamento no Trabalho: Características Psicométricas e Invariância Temporal do UWES. Avaliação Psicológica, 19(3), 277-288. DOI: https://doi.org/10.15689/ap.2020.1903.15631.06
  » https://doi.org/10.15689/ap.2020.1903.15631.06
• Ramalho, N., Teles, H., Ramalho, V., & Ribeiro, S. (2017). Adaptação e validação da Utrecht Work Engagement Scale (UWES) aplicada a assistentes sociais em Portugal. Revista Portuguesa de Investigação Comportamental e Social, 3(2), 10-20.
• Reichenheim, M. E. & Moraes, C. L. (2007). “Operacionalização de adaptação transcultural de instrumentos de aferição usados em epidemiologia. Revista de Saúde Pública, 41 (4), 665-73.
• Rodrigues, R. A. P., Mattos, C. A. C. D., Cardoso, H. G., & Mancebo, C. H. A. (2019). Motivação e engajamento dos servidores públicos federais da Universidade Federal do Pará. Colóquio Internacional de Gestão Universitária UFSC: Florianópolis
• Salanova, M. & Schaufeli, W. B. (2009). El engagement en el trabajo Madrid: Alianza Editorial.
• Santos, M. M. de O., Quel, L. F., Vieira, A. M., & Rosini, A. M. (2019). Indicadores de desempenho e engajamento profissional em organizações inovadoras. GESEC, 10(1),192-212.
• Schaufeli, W. B. & Bakker, A. (2003). Utrecht Work Engagement Scale (UWES). Occupational Health Psychology Unit: Utrecht.
• Schaufeli, W. B. (2012). Work Engagement: what do we know and where do we go? Romanian Journal of Applied Psychology, 14 (1), 3-10.
• Schaufeli, W. B. (2016). Heavy work investment, personality and organizational climate. Journal of Managerial Psychology, 31, 1057-1073.
• Schaufeli, W. B. (2017). The Job Demands-Resources model: a ‘how to’ guide to increase work engagement and prevent burnout. Organizational Dynamics, 46, 120-132.
• Schaufeli, W. B.; Salanova, M.; González-Romá, V. & Bakker, A. B. (2002). The measurement of engagement and burnout: a two sample confirmatory factor analytic approach. Journal of Happiness Studies, 3, 71-92.
• Schaufeli, W.B. & De Witte, H. (2017). Work engagement in contrast to burnout: real or redundant? Burnout Research, 5, 1-2.
• Seppala, P., Mauno, S., Feldt, T., Hakanen, J., Kinnunen, U., Tolvanen, A. & Schaufeli, W. (2009). The construct validity of the Utrecht Work Engagement Scale: multisample and longitudinal evidence. J Happiness Stud, 10, 459-481.
• Serralta, F. B., Nunes, M. L. T. & Eizirik, C. L. (2007). Elaboração da versão em português do Psychotherapy Process Q-Set. Revista de Psiquiatria do Rio Grande do Sul, 29(1), 44-55.
• Silva, J. V., & Domingues, E. A. R. (2017). Adaptação cultural e validação da escala para avaliar as capacidades de autocuidado. Arquivos de Ciências da Saúde, 24(4), 30-36.
• Simbula, S.; Guglielmi, D.; Schaufeli W. B.; Depolo, M. (2013). An Italian validation of the Utrecht Work Engagement Scale: characterization of engaged groups in a sample of schoolteachers. Bollettino di Psicologia Applicata, 268, 43-54.
• Teixeira, P. R., Lourenção, L. G., Gazetta, C. E., Gonsalez, E. G., Rotta, D. S., Pinto, M. H., Peres, L. & Beretta, D. (2017). Engagement no Trabalho em Residentes Médicos de Pediatria. Revista Brasileira de Educação Médica, 41(1), 126-133.
• Torrente, P., Salanova, M., Llorens, S. & Schaufeli, W. B. (2013). From “I” to “We”: The factorial validity of a Team Work Engagement Scale. In: Sónia P. Gonçalves. Occupational Health Psychology: from burnout to well-being. Editora: Cidade. pp. 333-356.
• Vazquez, A. C. S., Magnan, E. S., Pacico, J. C., Hutz, C. S. & Schaufeli, W. B. (2015). Adaptation and Validation of the Brazilian Version of the Utrecht Work Engagement Scale. Psico-USF, 20(2), 207-217.
• Yulita, -; Dollard, M. F. & Idris, M. A. (2017). Climate congruence: how espoused psychosocial safety climate and enacted managerial support affect emotional exhaustion and work engagement. Safety Science, 96, 132-142.
• Zumpano, C. E., Mendonça, T. M. S., da Silva, C. H. M., Correia, H., Arnold, B., & Pinto, R. D. M. C. (2017). Adaptação transcultural e validação da escala. Cad. Saúde Pública, 33(1), e00107616.

Publication Dates

• Publication in this collection
  28 Oct 2022
• Date of issue
  2022

History

• Received
  17 Mar 2018
• Accepted
  07 June 2021