Cross-cultural adaptation and reliability analysis of the Brazilian version of the Work Rehabilitation Questionnaire (WORQ)

Abstract

Introduction

The Work Rehabilitation Questionnaire (WORQ) is an international instrument, based on the International Classification of Functioning, Disability and Health (ICF), used to assess the functional capacity of people with work disabilities.

Objective

To perform a cross-cultural adaptation and a reliability analysis of this instrument for the Brazilian population.

Method

The cross-cultural adaptation consisted of translation, synthesis of translations, back translations, analysis by the committee of judges, pre-test, and pilot test. The results were obtained by calculating the agreement rate for semantic, idiomatic, experiential, and conceptual equivalence. Reliability was tested by analysis of stability, equivalence, and internal consistency by calculating Cronbach’s Alpha and Intraclass Correlation Coefficient (ICC).

Results

Five translators, eight judges, and 14 possible users in the pre-test participated in the first stage. The agreement rate was <90% only for idiomatic equivalence, resulting in orthographic adjustments without changing the instrument construct. The reliability was tested on 34 workers with disabilities, with a mean age of 40.3 (±4.2) years, of both sexes, employees of an educational institution in the state of São Paulo, Brazil, hired through the Quotas Law (8,213/91). The internal consistency showed values >80% in all domains and was >90% for the total score. Fifteen days after the first assessment, 20 of these participants were randomly selected for reassessment. The ICC values for stability and equivalence were 82.5% (p=0.008) and 95.4% (p<0.001), respectively.

Conclusion

The WORQ was translated and cross-culturally adapted to Brazilian Portuguese and reliably responded to the ICF domains. The Brazilian version of the WORQ is available at http://www.myworq.org/questionnaire_en.php.

Keywords:
Disability Evaluation; International Classification of Functioning; Disability and Health; Vocational Rehabilitation; Surveys and Questionnaires

Resumo

Introdução

O Work Rehabilitation Questionnaire (WORQ) é um instrumento internacional baseado na Classificação Internacional de Funcionalidade, Incapacidade e Saúde para avaliar a capacidade funcional em pessoas com deficiência para o trabalho.

Objetivo

Realizar adaptação transcultural e análise da confiabilidade para a população brasileira.

Métodos

A adaptação transcultural consistiu em tradução, síntese das traduções, retrotraduções, análise do comitê de juízes, pré-teste e teste piloto. Os resultados foram obtidos pelo cálculo da taxa de concordância para equivalência semântica, idiomática, experiencial e conceitual. A confiabilidade foi testada por análise de estabilidade, equivalência e consistência interna por meio do cálculo de α-Cronbach e Coeficiente de Correlação Intraclasse (ICC).

Resultados

Participaram da primeira etapa 05 tradutores, 08 juízes e 14 possíveis usuários no pré-teste. A taxa de concordância foi menor que 90% apenas na equivalência idiomática, resultando em ajustes ortográficos sem alterar o construto do instrumento. A confiabilidade foi testada em 34 trabalhadores com deficiência, idade média de 40,3 (±4,2) anos, ambos os sexos, funcionários de uma instituição de ensino de São Paulo inseridos pela lei de cotas (8.213/91). A Consistência Interna apresentou valores superiores a 80% em todos os domínios e foi maior que 90% para o escore total. Após 15 dias da primeira avaliação, foram sorteados 20 desses participantes para reavaliação. O ICC para estabilidade foi 82,5% (p=0,008) e equivalência foi 95,4% (p<0,001).

Conclusão

O questionário WORQ foi traduzido e adaptado transculturalmente para o português brasileiro e apresentou confiabilidade para responder aos domínios da CIF. A versão brasileira está disponível na home page do WORQ, no endereço eletrônico http://www.myworq.org/questionnaire_en.php.

Palavras-chave:
Avaliação da Deficiência; Classificação Internacional de Funcionalidade; Incapacidade e Saúde; Reabilitação Profissional; Inquéritos e Questionários

Introduction

The inclusion of people with disabilities in the labor market is one of the priority themes in the world’s search for a more inclusive society (Soares, 2019). Professional activity allows human beings to express themselves and be included in society, enabling them to transform their reality, survive, and form their identity (Tolfo & Piccinini, 2007).

Disabilities resulting from accidents, injuries, and health problems directly impact the inclusion of these professionals in the working environment (Glässel et al., 2011; Finger et al., 2012; Padkapayeva et al., 2017). Access and return to work require assertive, detailed, and continuous classification methodologies (Simonelli et al., 2013), with a special focus on functionality, individual disability, and ergonomics for the execution of tasks, so that an accessible and safe workplace can be ensured (Wuellrich, 2010).

In this context, the International Classification of Functioning, Disability and Health (ICF) describes and defines the relationship between work and health (Finger et al., 2012; Simonelli et al., 2013; Momsen et al., 2019). The biopsychosocial perspective provided by the ICF favors interdisciplinary communication among health professionals and facilitates dialogue with employers. Therefore, it helps professionals with disabilities to be admitted to and remain at work, valuing their ability and not the difficulties arising from their disability, thus guaranteeing the necessary working conditions for professionals to feel respected in the employment relationship (Finger et al., 2014, 2019).

The Work Rehabilitation Questionnaire (WORQ) (Finger et al., 2014) was developed in English based on the brief ICF Core Set for Vocational Rehabilitation (VR) and consists of a set of 13 subcategories of its full version. This set is a minimum reference standard to assess and describe the relevant factors of workers’ functionality, regardless of health status, or to define VR. This instrument was developed to be applied by an interviewer and/or in a self-response version, in which only the filling instructions are different. Aiming to assess an individual’s degree of functionality, the WORQ indicates the capacity and performance of people with disabilities to perform work activities. It can be applied to support therapeutic decision-making, management, and the planning of more effective actions and policies (Finger et al., 2014).

The biopsychosocial approach of the WORQ enables the assessment of interactions between the biological dimensions of functioning and disability (body functions and structures), with dimensions related to activities and participation and environmental factors (performance) (Wuellrich, 2010; Finger et al., 2012). In certain situations, the disability does not prevent work, and it is necessary to adjust the environment physically, cognitively, organizationally, and ergonomically to favor the work conditions regardless of the level and extent of the disability (Wuellrich, 2010; Martins, 2015).

In Brazil, public policies have advanced regarding the right to work for people with disabilities(Soares, 2019); however, inequality remains (Oliveira et al., 2017; Paiva & Bendassolli, 2017). The difficulty of access and the fundamental conditions to guarantee employment are attributed to the lack of supervision and investment in actions to develop the skills of this population (Simonelli et al., 2013).

Reports on instruments to assess the functionality of workers with disabilities inserted in or provisionally removed from work are limited. Therefore, it is relevant to perform the cross-cultural adaptation and reliability analysis of the WORQ so that it can be applied to the Brazilian population.

Thus, it will be possible to provide an instrument capable of identifying and subsidizing promotion and intervention actions in different contexts of VR, such as medical expertise, professional social work, psychology, occupational therapy, physiotherapy, and other areas related to the process (Santos & Lopes, 2021).

Methods

This study conducted the cross-cultural adaptation and reliability analysis of the Work Rehabilitation Questionnaire (WORQ). It was submitted and approved by the Ethics and Human Research Committee of Mackenzie Presbyterian University under protocol CAAE - 54515215.3.0000.0084 according to the norms established by Resolution no. 466/12 of the National Health Council (Brasil, 2012). The study was organized in two phases: in the first phase, the instrument was translated and linguistically and cross-culturally adapted to Brazilian Portuguese according to international assumptions (Guillemin et al., 1993; Reis et al., 2015) and following the guidelines in Beaton et al. (2007) and Fortes & Araújo (2019). The following methodological stages were considered in this phase: I. Authorization of the authors of the instrument; II. Translation of the instrument into Brazilian Portuguese; III. Synthesis of the translations; IV. Back translation, synthesis, and approval by the authors; V. Evaluation of the translated version by a committee of expert judges; VI. Pretest of the preliminary version; VII. Back translation of the final version; VIII. Submission of the instrument for approval by its authors. The second phase of the study consisted in testing the instrument reliability property measures, as shown in the scheme in Figure 1. These stages are detailed in the following paragraphs.

Figure 1
Schematic presentation of the methodological stages of the cross-cultural adaptation of the Work Rehabilitation Questionnaire (WORQ). Source: Prepared by the authors.

The instrument

There are two versions of the Work Rehabilitation Questionnaire (WORQ) (Finger et al., 2014). Both of them are divided into two parts and differ only regarding the filling instructions.

Part 1 contains 17 questions on sociodemographic characteristics and general health conditions related to vocational rehabilitation (VR) to assess three identified environmental categories (e310 - Immediate family, e330 - People in positions of authority, e580 - Health services, e590 - Labor and employment services, systems and policies) and six categories from activities and participation describing vocational education or work situations (d825 - Vocational training, d830 - Higher education, d840 - Apprenticeship (work preparation), d845 - Acquiring, keeping and terminating a job, d850 - Remunerative employment, d855 - Non-remunerative employment) from the brief ICF Core Set.

Part 2 of the WORQ (Main Section) assesses the functionality of individuals in VR and presents 42 questions answered from a baseline question: “Overall in the past week, to what extent did you have problems with...”. Respondents rate their level of problem in the past week for each question from 0 = no problem to 10 = complete problem.

Methodological stages of the study - Phase 1

1. Authorization of the authors of the instrument: Communication with the authors occurred electronically at all stages of this study (Borsa et al., 2012; Finger et al., 2014, 2019).

2. Translation of the instrument into Brazilian Portuguese: This phase includes two independent translations of the WORQ into Brazilian Portuguese: the first by a certified translator (T1) and the second by a translator specialized in the field of rehabilitation (T2).

3. Synthesis of the translations: Consensus synthesis of translations T1-2 was performed by three experts. At this stage, both translations (T1 and T2) were analyzed by a committee of specialists composed of three Brazilian university professors with knowledge of the original language of the instrument (English), namely, one with an M.S. in Physical Therapy, one with a Ph.D. In Physical Therapy, and one with a Ph.D. in Education.

4. Back translation, synthesis, and approval by the authors: Version T1-2 was back-translated by two bilingual translators who are native speakers of English with linguistic and cultural mastery of Brazilian Portuguese. Neither of them was aware of the contents of the instrument and independently translated the T1-2 version into English, resulting in the BT1 and BT2 versions (Ramada-Rodilla et al., 2013; Chaves et al., 2017). The authors received the detailed reports of the back-translated syntheses electronically.

5. Evaluation of the translated version by a committee of expert judges: Version BT1-2 was updated and sent to the committee of expert judges for analysis. The five judges were selected according to the following criteria: having professional and scientific experience in the area of disability, rehabilitation, construction and validation of measuring instruments, ICF, and a doctor’s degree (Borsa et al., 2012). One of the judges was selected from another Brazilian state to mitigate the regional bias (Cassepp-Borges et al., 2010). The judges received the guidelines and materials to be evaluated electronically. The objective of this stage was to consolidate the construct of the translated version, and the judges were instructed to assess the adequacy of the items considering the target population of the instrument (Cassepp-Borges et al., 2010; Borsa et al., 2012). For each instrument item, the judges were instructed to choose “agree” or “disagree” considering the following aspects (Guillemin et al., 1993):

a) semantic equivalence - assess whether the words have the same meaning, the item has more than one meaning, and there are grammatical errors in the translation;

b) idiomatic equivalence - evaluate whether items from the original instrument that were difficult to translate were replaced by an equivalent expression that has not changed its cultural meaning;

c) experiential equivalence - assess whether a given item of the instrument is applicable in the new culture and, if not, replace it with an equivalent item;

d) conceptual equivalence - evaluate whether a given term or expression, even if properly translated, informs the same aspect in the new culture.

By checking the “disagree” option in any of the items, the judge should objectively suggest an amendment proposal. With these adjustments, a preliminary version was established, designated as V1.

1. Pretest of the preliminary version (V1): Version V1 was sent back to the expert judges for reassessment accompanied by a report containing the Agreement Rates and the explanations on which the new version was based. This version was preliminarily tested in a convenience sample composed of People with Disabilities. This sample was chosen according to the literature, which recommends that the preliminary version of an instrument should be tested on its potential users. These users should be less skilled than the expert judges in testing the clarity and understanding of the instrument. The participants’ observations were weighted and those considered relevant because they did not change the content and facilitate understanding were modified (Pasquali, 2003; Anthoine et al., 2014; Coluci et al., 2015). The weighted changes of the post-test considerations resulted in a new version, named V2.

2. Back translation of the final version (BTV2): Version V2 was back-translated (BTV2) by a Brazilian university professor (Ph.D.) who masters English and was not aware of the contents of the instrument.

3. Submission of the instrument for approval by its authors: After verification by the committee of experts who participated in the other syntheses, version V2, together with a detailed report on the Transcultural Adaptation process, was sent to the authors for final approval, generating the final version V3.

Methodological stages of the study - Phase 2

Evidence of validity - Testing of instrument reliability psychometric measures: An adequate process of translation and cross-cultural adaptation of measurement instruments should ensure that the psychometric properties of the original instrument be preserved (Ramada-Rodilla et al., 2013; Coluci et al., 2015). In the second phase of this study, the psychometric measures of the WORQ version were tested for comparison with the model adopted in the cross-cultural adaptations of the instrument to German (Finger et al., 2014) and French (Finger et al., 2019). A reliability analysis, which is considered the first measure in cross-cultural adaptation studies, was performed to ensure that the adapted version meets the criteria established in the original version (Ramada-Rodilla et al., 2013). Instrument reliability measures include 1. Internal Consistency, 2. Stability, and 3. Equivalence (Alexandre et al., 2013; Coluci et al., 2015). Thus, these measures were tested in the final version (V3) of the WORQ in an independent sample.

To this end, the participants were evaluated at two moments by two physical therapists specialized in Functional Health who had not had previous contact with the sample at any stage of the study. Initially, the first interview, designated as I1 (test), was conducted by one of the professionals for internal consistency analysis. After 15 days, the sample was randomly separated and the second interview, designated as I1-2 (retest), was carried out by the same professional for stability analysis and by another professional, designated as I2, for equivalence analysis.

a) Internal Consistency: Internal consistency comprises the correlation between the items that make up an instrument. A good internal consistency means that the items are homogeneous and measure the same content. It is usually evaluated by calculating Cronbach’s Alpha coefficient, which considers the sum of scores for a total set of items and/or sub-items in which all total scores or sub-items point in the same direction (Alexandre et al., 2013).

b) Stability and Equivalence: The stability of a measurement instrument is associated with the ability of the measure to be repeated, regardless of time, when it is used with the same population and is represented by the correlation between the test and retest measures. Equivalence, on the other hand, assesses reliability by considering the degree of accuracy of the measures of the same instrument when applied to the same population by two or more examiners, and is represented by the correlation between the examiners’ measures (Alexandre et al., 2013; Miot, 2016).

Statistical analysis

The sample was analyzed using descriptive statistics. In the first phase, an Agreement Rate (AR) was calculated through qualitative empirical analysis by dividing the number of “agree” responses by the total number of judges and multiplying it by one hundred (Fortes & Araújo, 2019). The AR was calculated for each of the items related to semantic, idiomatic, experiential, and conceptual equivalence (Guillemin et al., 1993). The mean AR was calculated for each area of equivalence for each judge. The AR was considered acceptable if the values achieved were ≥90%.

In the second phase, internal consistency was examined using Cronbach’s Alpha coefficient. This was considered the main section of the instrument (Alexandre et al., 2013; Ramada-Rodilla et al., 2013; Coluci et al., 2015). The calculation was performed with 40 items divided into two dimensions: a) body structures and functions - 18 items; b) activities and participation - 22 items. To this end, both the total of the items and the scores of the dimensions were considered, based on the assumed value α≥0.80 (Alexandre et al., 2013; Finger et al., 2019).

The Intraclass Correlation Coefficient (ICC) — a reliable statistical test for continuous variables in clinical and experimental studies — was used to calculate stability and equivalence. Calculation of the average of the total score obtained in the 40 questions of the main section in the two evaluated conditions was performed, and a reliability value of ≥0.801 was adopted. The paired Student’s t-test was used to verify the equality between means and was considered statistically different when p≤0.05. Stability was calculated based on data collected in the 1st (I1) and 2nd (I1-2) interviews. The stability results were classified as very strongly correlated when ≥0.80 (Alexandre et al., 2013; Miot, 2016). The data were processed using the PASW Statistics for Windows^® software (International Business Machines Corporation, 2017).

Results

The results are subdivided according to study phases.

• PHASE 1:

The authors’ consent was obtained (stage I) and the two translations were performed (stage II).

1. Synthesis of the translations: The three specialists read the three versions individually and concluded that they were similar. Version T1 presented literal aspects of the original version and version T2 contained grammatical and interpretation adjustments that led to cross-cultural adaptation. The experts then analyzed these two versions item by item and consulted the original version in English when they found differences (Tsang et al., 2017).

At the end of the analyses, they adopted the translation closest to the original language. In total, 18 adjustments were made to the instrument: eight in Part 1 and 10 in Part 2, related to grammatical adequacy. There was a preference for the T1 translation (certified) over the T2 translation (specialized). Thus, the non-specialized translator is less likely to deviate from the meaning of the items, producing a more literal version of the original language (Ramada-Rodilla et al., 2013). This result was expected, as it is known that this stage consisted only in translating the instrument into Brazilian Portuguese and that cultural differences should be evaluated by the committee of expert judges (Epstein et al., 2015). A detailed report of this step (Ramada-Rodilla et al., 2013; International Test Commission, 2017), which originated the T1-2 version, was produced.

1. Back translation, synthesis, and approval by the authors: Since the versions were similar, an option was made for the back translation that was closest to the original English version, resulting in the common translation designated as RT1.2. This version was forwarded to the authors who requested to change the scale from 0 to 100 to 0 to 10 because of changes suggested by studies on translation and cross-cultural adaptation conducted in other countries.

2. Evaluation of the translated version by a committee of expert judges: The completed forms were returned after 15 days. The mean and individual AR values were >90%, except for idiomatic equivalence, whose mean AR was 86.1%. This gave rise to a literature review (Ramada-Rodilla et al., 2013; Epstein et al., 2015) of the Brazilian Federal Constitution, descriptors of the Virtual Health Library, and scientific publications addressing Professional Rehabilitation to adapt the terms or expressions of the items with AR values <90% (Coluci et al., 2015). Adjustments were required mainly in sentence construction, thus spelling and punctuation adjustments were made and pronouns, prepositions, and linking verbs were added as essential words to adapt the language to that spoken in Brazil (Coluci et al., 2015; Reis et al., 2015).

3. Pretest of the preliminary version (V1): In this stage, the AR was >90% for all items, and V1 was used for testing in the target population. The sample consisted of 20 employees hired through the Quotas Law intended for People with Disabilities, but only 14 showed availability of schedule, thus sampling was purposive. Sample description: 14 People with Disabilities of both sexes (9 male and 5 female), with a mean age of 45.35 (SD ±13.86) years, 13 with physical disabilities and 01 with multiple disabilities, all employees of a higher education institution in the city of São Paulo, hired through the Quotas Law (8.213/91), who had been working for 10.3 (SD ±7.8) years on average, 04 part-time and 10 full-time workers. As for the level of education, 02 had graduate education, 06 had a college degree, and 06 had completed high school. Regarding their positions, 10 were administrative assistants and 04 were administrative technicians.

The participants were gathered in a single location and answered the V1 version of the WORQ. The average completion time was 15 min and 30 s and the respondents reported good understanding of the instrument. At the end, they suggested rephrasing four questions: three from Part 1 and one from Part 2 (Main Section), namely, a) question 1 - não se sentir descansado e revigorado durante o dia?, replace with: sentir-se cansado e não revigorado durante o dia?; b) question 2 - dormir, como adormecer, acordar frequentemente durante a noite ou acordar muito cedo pela manhã?, replace with: com sono, como adormecer, acordar frequentemente durante a noite ou acordar muito cedo pela manhã?; c) question 14 - resistência generalizada ao realizar atividades físicas?, replace with: fôlego ao realizar atividades físicas?; d) question 32 - mover-se ao redor, incluindo arrastar-se, subir e correr?, replace with: movimentar-se, incluindo subir e descer escadas e correr.

In addition, comprehension problems were identified in questions not mentioned by the participants. In question 14 - Part 1, the difficulty was related to completing the answer. The question addresses the work or professional intervention currently received. There is no option to respond if the individual does not receive support or intervention. Moreover, still in this question, even after explaining what ‘case management’ meant, the term was not understood by the participants, who added that referral and follow-up are conducted by the expert physician of the National Institute of Social Security (Brasil, 2018). This result agreed with the assessment of the committee of expert judges, who also claimed not to be aware of this service in Brazil. Table 1 shows a summary of the main changes made to the WORQ considering the analysis of the expert judges’ committee (V1) and the post-test (V2).

Table 1
Main changes to the Brazilian version of the WORQ with considerations by expert judges and users, which originated the V2 version.

Part 1	Original Document	Expression in English	Expression in Brazilian Portuguese	Explanation
Question 4	Work status	Work condition	Situação de trabalho	The judges understood that this question asks about the characteristics, and type of work, which in Brazilian Portuguese, is asked as a “situation” or “condition”.
Question 5	Work situation	Working hours arrangements	Jornada de trabalho	The judges understood that this question referred to the number of hours worked per day. The expression “work situation” had the meaning of having or not having a job and did not correspond to the response options. After reviewing the literature, an option was made for “workday”, according to Brazilian legislation.
Question 5 -	On modified or light duty	Flexible work team	Jornada variável,	Likewise, the expression “modified or light service”, in Brazilian Portuguese, has the meaning of strength, working with a load. The terminology used in the Brazilian labor legislation for people with disabilities was adopted.
			horário flexível
Question 9	e.g., cardboard box manufacturing, road maintenance, retail store, secondary school, dairy farm, municipal government	e.g., in industry, agriculture,	Por exemplo:	In this case, some items in the examples did not correspond to the Brazilian culture. Operating sectors of the Brazilian economy were adopted.
		retail,	Na indústria,
		education,	comércio,
		sports,	educação,
		health care,	esporte,
		public sector	saúde,
			serviço público
Question 10	e.g., driving trucks, operating machines, writing letters, answering telephone calls	e.g., driving a truck, operating machines, computer services, telemarketing	Por exemplo:	The terms “write letters” and “answer phone calls” do not correspond to professions in Brazil. In order not to change the content, the terms were replaced by “computer services” and “telemarketing”, which corresponds to the public service call center.
			dirigindo caminhão,
			operando máquinas,
			serviços de informática, telemarketing
Question 12	e.g., physicians, therapists, etc.	e.g., physical therapy, occupational therapy, speech therapy, psychotherapy, etc.)	Por exemplo: fisioterapia,	The judges understood that the examples needed to be clearer, and the therapeutic modalities were included.
			terapia ocupacional, fonoaudiologia,
			psicoterapia etc.
Question 14	What kind of work or vocational intervention are you receiving now? (List all you know)	Do you receive any kind of support or vocational intervention at the time? (List all that you know)	Você recebe algum tipo de apoio ou intervenção profissional no momento? (liste tudo que você sabe)	In this regard, the respondents had difficulties in the preliminary test. The review clearly showed that the question was exclusively for those under VR. As the WORQ is intended for two populations, it was suggested to include answer options as in the previous alternatives to maintain the standard. Still in this question, the term “process management”, initially translated in a similar way, was modified by the judges to “administrative process”; however, it is not a common term in Brazilian Portuguese, and was not understood in the pre-test. Therefore, it was decided to eliminate it and keep only examples common to Brazil. In the preliminary pre-test, the participants did not understand the expression “general resistance” and suggested changing it to “out of breath”.
	(e.g., physical training, cognitive training, case management, vocational training, workplace adaptation, work evaluation, etc.)	(e.g., physical training, cognitive training, vocational training, the local work adaptation, work evaluation, etc.)	(Por exemplo: treinamento físico, treinamento cognitivo, treinamento profissional, adaptação do local de trabalho, avaliação do trabalho etc.)
	Interventions: general endurance	Yes / No / Not applicable	Sim/Não/Não aplicável
		If yes, please, specify the type of intervention Which do You Get:	Se sim, por favor, especifique que tipo de intervenções você recebe:
		resistance (breathless)	resistência (fôlego)
Part 2	Original Document	Expression in English	Expression in Portuguese	Explanation
Question 26	telecommunication devices	as landline and mobile phones, radio, television	Telefone fixo e móvel,	As the expression is shown in the example, the judges suggested explaining the meaning of telecommunication devices.
			rádio,
			televisão
Question 29	using the hand, fingers, and thumb?	hand and fingers?	Mãos e dedos	In Brazilian Portuguese, the thumb is not considered separately, and its use is implied when referring to the hand and fingers, thus the word “thumb” was deleted.
Question 32	crawling, climbing	Going up and down the stairs	Subir e descer escadas	In Brazilian Portuguese, the word “crawling” was not understood in the preliminary pre-test. Also, in Brazilian Portuguese, it is understood that whoever climbs, climbs something; therefore, “up and down” were added.
Question 40	road (the distance of about 20 meters or 66 feet)?	street (the distance of about 20 meters or 66 feet)?	Da rua (distância de aproximadamente 20 metros)?	For cultural reasons, the word “road” and the unit of measurement “feet” do not apply to the Brazilian Portuguese vocabulary. The judges suggested replacing “road” with street and removing the measurement unit “66 feet”.

The back translation of the final version (BTV2) was performed (stage VII).

1. Submission of the instrument for approval by its authors: The authors agreed with most of the changes and requested clarification on the following questions:

a) Question 5 - Part 1 - the authors mentioned that the expression “On modified or light duty” does not only refer to working time, but also to adapted work tasks, such as not lifting heavy loads, having the opportunity to change work positions, being in a quiet room without distractions, and having breaks at work. The item was revised, and it was concluded that this theme was covered in Question 16 - Part 1, which asks about the support received from a supervisor or boss at work. In addition, this expression does not have the same meaning in the Brazilian culture, and it is associated with the transport of cargo (weight); therefore, the change was rejected.

b) Question 12 - Part 2 - the authors agreed with naming different therapies mentioned in the example; however, they did not understand why the medical specialty was not mentioned. It was explained that the question already explains the medical treatment, which is well-accepted in Brazilian culture. The authors understood and accepted the decision.

c) Question 32 - Part 2 - regarding the exclusion of the word “climbing”, the authors questioned the changes in this question. They mentioned that the expression climbing in the context of Professional Rehabilitation should include not only going up and down the stairs, but also going up and climbing mountains and steep hills. They also mentioned that the expression “crawling” was included to consider construction workers, such as carpenters and plumbers, or women who play with small children on the floor. They requested verification and, if the decision is for exclusion, that this be explicitly declared as a non-relevant skill in the Brazilian population. The terms were evaluated and it was decided to add going up and down stairs, ramps, and slopes, more contextualized to the Brazilian culture. The term “crawling”, on the other hand, could be replaced, but it did not seem to be a significant skill for the studied population, mainly if the types of work destined for People with Disabilities and the characteristics of the sample population are considered. In this study, all participants moved independently.

d) From the authors’ considerations, only Question 5 - Part 1 and a part of Question 32 - Part 2 did not reach a consensus, and an option was made to keep the translation of the Brazilian version (V2). The accepted changes resulted in the final version (V3) of the WORQ.

• PHASE 2:

Evidence of validity - Testing of instrument reliability psychometric measures: At this stage, the study sample consisted of 34 People with Disabilities, purposively selected, of both sexes, with a mean age of 40.3 (SD ±4.2) years, all employees of a higher education institution in the city of São Paulo, hired through the Quotas Law 8.213/91 (Brasil, 1991), who had been working for 10.3 (SD ±7.8) years, on average. Regarding their positions, 11.8% were administrative assistants, 41.2% were administrative attendants, and 47% were administrative technicians. Table 2 presents a summary of the other sociodemographic characteristics.

Table 2
Sociodemographic characteristics of the study sample (n = 34).

Variables	Number of participants	%
Sex
Male	19	55.9
Female	15	44.1
Marital status
Married	19	55.9
Single	13	38.2
Divorced	2	5.9
Disability
Physical	25	73.5
Visual	6	17.6
Hearing	3	8.8
Origin
congenital	14	41.1
acquired	20	58.8
Level of education
Graduate Education	4	11.8
College Degree	10	29.4
High School	19	55.9
Incomplete Elementary School	1	2.9
Work schedule
Part-time	11	32.4
Full-time	21	61.8
Flexible	2	5.9

a) Internal Consistency: The first interview (I1) was held at the workplace on three consecutive days. The Cronbach’s Alpha coefficient was calculated considering Part 2 of the instrument (Main Section), whose scope assesses the respondent’s functionality and assigns a score from 0 to 10, where 0 = no problem and 10 = complete problem, that is, the higher the score assigned, the greater the functional disability. This part of the instrument has 42 questions; however, questions 41 and 42 have a response pattern different from the others and were, therefore, excluded to avoid bias. Thus, the calculation was performed based on 40 items divided into two dimensions: a) body structures and functions - 18 items; b) activities and participation - 22 items. The Cronbach’s Alpha coefficient was calculated considering the total score of the items and the score per dimension, adopting α≥0.80, as shown in Table 3.

Table 3
Internal Consistency of the WORQ - Cronbach’s Alpha coefficient: 1st interview (I1) for the dimensions and total items of the Main Section.

Dimensions (D)	Number of participants	Cronbach’s Alfa coefficient
Body structures and functions	34	0.834
Activities and participation	34	0.883
Total	34	0.915

b) Stability and Equivalence: Fifteen days after the first interview (I1 - test), 20 participants, chosen and divided into two groups (10 participants each) by drawing lots, were invited to participate in the second stage of the study. The first group was reassessed by the same examiner (I1-2 - retest) to analyze the stability of the measurement, whereas the second group was reevaluated by another examiner (I2) to verify the equivalence of the measurement (inter-rater). At this stage, the average interview time was 14 min and 42 s for the I1-2 and 15 min and 30 s for the I2. Stability and equivalence were calculated using the ICC. The mean total score obtained in the 40 questions of the Main Section in the two evaluated conditions was calculated (Alexandre et al., 2013; Miot, 2016). Stability was calculated considering the data collected in the 1st (E1) and 2nd (E1. 2) interviews (Alexandre et al., 2013; Miot, 2016), as shown in Table 4.

Table 4
Stability - I1 vs. I1-2 of the WORQ - Intraclass Correlation Coefficient (ICC) and paired Student’s t-test.

Score	I1 - test	I1-2 - retest	t-Test	Spearman	ICC
Mean	2.71	1.96	p=0.038	r=0.789	0.825
Median	2.62	1.91
Standard deviation	1.42	0.80
Coefficient of variation	52%	41%
Minimum	0.71	0.88			p=0.008
Maximum	5.71	3.05
Number of participants	10	10
Confidence Interval	0.88	0.49

Discussion

The results show that the cross-cultural adaptation of the Work Rehabilitation Questionnaire (WORQ) generated good levels of reliability, considering internal consistency, stability, and equivalence, and satisfactorily met the international assumptions. These findings provide robustness to the cross-cultural adaptation performed, maintaining the functionality concept of the instrument. The WORQ has been culturally adapted for several countries with consistent evidence of validity. The cultural adaptation and reliability analysis of its Brazilian version were demonstrated in this study.

The procedures followed national and international guidelines. Interlocution with the authors was essential to ensure that the translations met the concept of the original version. For the first translation of this scale into Portuguese, it is recommended that two blind translations be carried out by translators with different domains. It is expected that the translator with knowledge about the study and content mastery present a translation with clinical adaptations consistent with the original version (Ramada-Rodilla et al., 2013; Epstein et al., 2015).

The importance of translation synthesis is a consensus in the literature (Guillemin et al., 1993; International Test Commission, 2017). However, there are disagreements regarding the composition of the special committee conducting the synthesis (Reis et al., 2015; Chaves et al., 2017). In this study, the special committee was composed of bilingual experts who were familiar with the objective domains of the instrument, which ensured a consensual translation without discrepancies or changes in content. Therefore, the experts selected were researchers involved in the project (Tsang et al., 2017) but not in the translations, and were responsible for clarifying any doubts between translations (Guillemin et al., 1993; Ramada-Rodilla et al., 2013).

For the back translation, the same procedure used in the first stage of the study for the English-Portuguese translations was adopted. The back translation stage aims to verify whether the translated version ensures the content of the original version, without risk of inappropriate interpretation and with quality assurance for cross-cultural adaptation (Chaves et al., 2017; Ramada-Rodilla et al., 2013).

Internal consistency results were considered good for the domain scores (α>0.8) and very good for the total score (α>0.9). This measure aims to assess the internal consistency of the instrument, verifying whether a set of items or variables is related to a single factor, that is, whether the instrument can consistently assess an attribute (Alexandre et al., 2013; Miot, 2016); thus, the closer to 1, the greater the internal consistency.

In the French development and cross-cultural adaptation of the WORQ, internal consistency was also measured by Cronbach’s Alpha coefficient considering the total number of items (Finger et al., 2014, 2019). The results were similar to those obtained in this study, showing good internal consistency (α=0.883) for the development and very good internal consistency (α=0.968) for the cross-cultural adaptation. Although neither of these studies calculated the mean separation of the dimensions, these data are sufficient to ensure that the reliability measured by internal consistency was maintained in the Brazilian version.

For inter-rater equivalence, the results showed an excellent ICC, which indicates agreement between the scores achieved in the two interviews applied to the same participant at different times, thus demonstrating the ability of the WORQ to measure the same attributes in the same individuals.

In this study, the time interval between the two evaluations was 15 days - very similar to that of the application of the original version (14 days) (Finger et al., 2014). In the WORQ, the respondent answers questions considering a recall period of one week “Overall in the past week, to what extent did you have problems with…”. This condition precludes a time interval shorter than seven days, while a period longer than 15 days could alter the conditions evaluated. As for the time interval, the attribution assessed should be neither too far (to avoid modifications of the observed phenomenon), nor too close (to avoid the learning effect) in time (Ramada-Rodilla et al., 2013).

Some limitations to this study were observed, and may be considered in the continuity of research in this area: the reduced sample size and the impossibility of using blind examiners both in the test and the retest. The continuation of this study provides for testing of construct and criterion validity with other populations.

Conclusion

Health authorities and rehabilitation professionals recognize that adequate assessment includes knowledge about diagnosis and identification of levels of functioning and disability (Brasil, 2011; Finger et al., 2012, 2014; Ziliotto & Berti, 2013; Martins, 2015).

Using the ICF is an important strategy to present a profile of People with Disabilities that goes beyond the clinical diagnosis. The ICF can describe the impact that a diagnosis has on the ability and performance of everyday tasks, in addition to correlating them with environmental barriers and facilitators. Thus, it provides a more comprehensive description, with a unified and multidisciplinary language of health care, and can be used to guide services and plan actions (Biz & Chun, 2020).

In this sense, an instrument based on the ICF biopsychosocial model, such as the Work Rehabilitation Questionnaire (WORQ), offers a functional overview that can ensure longitudinal follow-up of the health conditions of individuals in the Vocational Rehabilitation program. Combined with worker support legislation, this can collaborate in processes to ensure the inclusion, maintenance, and return to work of People with Disabilities.

The WORQ was translated and cross-culturally adapted to Brazilian Portuguese and reliably responded to the ICF domains. The Brazilian version of the WORQ is available at http://www.myworq.org/questionnaire_en.php.

• How to cite: Fernandes, S. M. S., Finger, M., Buchalla, C. M., D’Antino, M. E. F., & Blascovi-Assis, S. M. (2023). Cross-cultural adaptation and reliability analysis of the Brazilian version of the Work Rehabilitation Questionnaire (WORQ). Cadernos Brasileiros de Terapia Ocupacional, 31, e3369. https://doi.org/10.1590/2526-8910.ctoAO257833692

• Funding Source

  This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Referências

• Alexandre, N. M. C., Gallasch, C. H., Lima, M. H. M., & Rodrigues, R. C. M. (2013). A confiabilidade no desenvolvimento e avaliação de instrumentos de medida na área da saúde. Revista Eletrônica de Enfermagem, 15(3), 800-807.
• Anthoine, E., Moret, L., Regnault, A., Sébille, V., & Hardouin, J. B. (2014). Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health and Quality of Life Outcomes, 12(176), 1-10.
• Beaton, D., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2007). Recommendations for the cross-cultural adaptation of the DASH & QuickDASH outcome measures. Institute for Work & Health, 1(1), 1-45.
• Biz, M. C. P., & Chun, R. Y. S. (2020). Operacionalização da Classificação Internacional de Funcionalidade, Incapacidade e Saúde, CIF, em um Centro Especializado em Reabilitação. CoDAS, 32(2), 1-15.
• Borsa, J. C., Damásio, B. F., & Bandeira, D. R. (2012). Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia, 22(53), 423-432.
• Brasil. (2011). Viver sem limite: plano nacional dos direitos da pessoa com deficiência. Brasília: SDH-PR/SNPD.
• Brasil. Casa Civil. (1991, 24 de julho). Lei nº 8.213/91, de 24 de julho de 1991. Dispõe sobre os Planos de Benefícios da Previdência Social e dá outras providências. Diário Oficial [da] República Federativa do Brasil, Brasília.
• Brasil. Conselho Nacional de Saúde. (2012, 12 de dezembro). Resolução nº 466, de 12 de dezembro de 2012. Diário Oficial [da] República Federativa do Brasil, Brasília.
• Brasil. Ministério da Previdência Social. (2018). Manual técnico de procedimentos da área da reabilitação profissional. Brasília: Previdência Social.
• Cassepp-Borges, V., Balbinotti, M. A., & Teodoro, M. L. (2010). Tradução e validação de conteúdo: uma proposta para a adaptação de instrumentos. In L. Pasquali (Ed.), Instrumentação psicológica: fundamentos e práticas (pp. 506-520). Porto Alegre: Artmed.
• Chaves, F. F., Reis, I. A., Pagano, A. S., & Torres, H. D. C. (2017). Tradução, adaptação cultural e validação do Diabetes Empowerment Scale-Short Form. Revista de Saúde Pública, 51(16), 1-9.
• Coluci, M. Z. O., Alexandre, N. M. C., & Milani, D. (2015). Construção de instrumentos de medida na área da saúde. Ciência & Saúde Coletiva, 20(3), 925-936.
• Epstein, J., Santo, R. M., & Guillemin, F. (2015). A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. Journal of Clinical Epidemiology, 68(4), 435-441.
• Finger, M. E., Escorpizo, R., Bostan, C., & De Bie, R. (2014). Work Rehabilitation Questionnaire (WORQ): development and preliminary psychometric evidence of an ICF-based questionnaire for vocational rehabilitation. Journal of Occupational Rehabilitation, 24(3), 498-510.
• Finger, M. E., Escorpizo, R., Glässel, A., Gmünder, H. P., Lückenkemper, M., Chan, C., Fritz, J., Studer, U., Ekholm, J., Kostanjsek, N., Stucki, G., & Cieza, A. (2012). ICF Core Set for vocational rehabilitation: results of an international consensus conference. Disability and Rehabilitation, 34(5), 429-438.
• Finger, M. E., Wicki-Roten, V., Leger, B., & Escorpizo, R. (2019). Cross-Cultural Adaptation of the Work Rehabilitation Questionnaire (WORQ) to French: a valid and reliable instrument to assess work functioning. Journal of Occupational Rehabilitation, 29(2), 350-360.
• Fortes, C. P. D. D., & Araújo, A. P. D. Q. C. (2019). Check list for healthcare questionnaires cross-cultural translation and adaptation. Cadernos Saúde Coletiva, 27(2), 202-209.
• Glässel, A., Finger, M. E., Cieza, A., Treitler, C., Coenen, M., & Escorpizo, R. (2011). Vocational rehabilitation from the client’s perspective using the International Classification of Functioning, Disability and Health (ICF) as a reference. Journal of Occupational Rehabilitation, 21(2), 167-178.
• Guillemin, F., Bombardier, C., & Beaton, D. (1993). Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. Journal of Clinical Epidemiology, 46(12), 1417-1432.
• International Business Machines Corporation - IBM Corp. (2017). IBM SPSS Statistics for Windows, version 25.0 Armonk: IBM Corp.
• International Test Commission - ITC. (2017). The ITC guidelines for translating and adapting tests (2nd ed.). England: ITC. Recuperado em 8 de julho de 2017, de https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf
  » https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf
• Martins, A. C. (2015). Using the International Classification of Functioning, Disability and Health (ICF) to address facilitators and barriers to participation at work. Work, 50(4), 585-593.
• Miot, H. A. (2016). Análise de concordância em estudos clínicos e experimentais. Jornal Vascular Brasileiro, 15(2), 89-92.
• Momsen, A. H., Stapelfeldt, C. M., Rosbjerg, R., Escorpizo, R., Labriola, M., & Bjerrum, M. (2019). International Classification of Functioning, Disability and Health in vocational rehabilitation: a scoping review of the state of the field. Journal of Occupational Rehabilitation, 29(2), 241-273.
• Oliveira, L. C., Cavalli, V. T., & Guidugli, S. T. (2017). Política pública de inclusão das pessoas com deficiência no mercado de trabalho: algumas considerações sobre sua formulação, implementação e avanços de 1991 até 2015. Planejamento e Políticas Públicas, (48), 107-147.
• Padkapayeva, K., Posen, A., Yazdani, A., Buettgen, A., Mahood, Q., & Tompa, E. (2017). Workplace accommodations for persons with physical disabilities: evidence synthesis of the peer-reviewed literature. Disability and Rehabilitation, 39(21), 2134-2147.
• Paiva, J. C. M., & Bendassolli, P. F. (2017). Políticas sociais de inclusão social para pessoas com deficiência. Psicologia em Revista, 23(1), 418-429.
• Pasquali, L. (2003). Teoria da medida. In L. Pasquali (Ed.), Psicometria: teoria dos testes na psicologia e na educação (pp. 23-51). Petrópolis: Vozes.
• Ramada-Rodilla, J. M., Serra-Pujadas, C., & Delclós-Clanchet, G. L. (2013). Adaptación cultural y validación de cuestionarios de salud: revisión y recomendaciones metodológicas. Salud Pública de México, 55(1), 57-66.
• Reis, P. A. M., Carvalho, Z. M. D. F., Tirado Darder, J. J., Oriá, M. O. B., Studart, R. M. B., & Maniva, S. J. C. D. F. (2015). Adaptação transcultural do Quality of Life Index Spinal Cord Injury-Version III. Revista da Escola de Enfermagem da USP, 49(3), 401-408.
• Santos, G. S. H., & Lopes, R. E. (2021). O programa de reabilitação profissional do INSS e a reinserção do trabalhador no mercado de trabalho. Cadernos Brasileiros de Terapia Ocupacional, 29, 1-20.
• Simonelli, A. P., Rodrigues, D. da S., Navas, P. M. G., Soares, L. B. T., & Camarotto, J. A. (2013). Projeto ATO - Ação, Trabalho e Oportunidade: inclusão de pessoas com deficiência no trabalho: relato de experiência. Cadernos Brasileiros de Terapia Ocupacional, 21(1), 119-130.
• Soares, E. B. (2019). Inclusão de profissionais com deficiência no trabalho: desafios para a Gestão de Recursos Humanos. Gestão & Produção, 26(3), 1-11.
• Tolfo, S. D. R., & Piccinini, V. (2007). Sentidos e significados do trabalho: explorando conceitos, variáveis e estudos empíricos brasileiros. Psicologia e Sociedade, 19(1), 38-46.
• Tsang, S., Royse, C. F., & Terkawi, A. S. (2017). Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi Journal of Anaesthesia, 11(Supl. 1), S80-S89.
• Wuellrich, J. P. (2010). The effects of increasing financial incentives for firms to promote employment of disabled workers. Economics Letters, 107(2), 173-176.
• Ziliotto, D. M., & Berti, A. R. (2013). Reabilitação profissional para trabalhadores com deficiência: reflexões a partir do estado da arte. Saúde e Sociedade, 22(3), 736-750.

Edited by

Section editor

Profa. Dra. Iza Faria-Fortini

Publication Dates

• Publication in this collection
  08 May 2023
• Date of issue
  2023

History

• Received
  21 July 2022
• Reviewed
  02 Aug 2022
• Reviewed
  29 Dec 2022
• Accepted
  31 Jan 2023