Cross-cultural adaptation and validation of the World Health Or ganization Health and Work Performance Questionnaire to Brazilian nurses ^*

Abstracts

Study aiming to cross-culturally adapt the instrument Health and Work Performance Questionnaire (HPQ) and evaluate the psychometric properties of the Brazilian version for nurses. The adaptation process followed the Process of Translation of World Health Organization. Data for the evaluation of the psychometric properties were collected in a teaching public hospital of Ribeirão Preto in 2011. The evaluated psychometric properties were: face validity and content (group of experts), reliability by Cronbach’s alpha and test-retest stability. In the evaluation of psychometric properties, the internal consistency of the HPQ adapted version, Cronbach’s alpha was 0.94 for the section A and 0.86 for section B of the instrument. In analyzing the agreement of test-retest stability, the agreements were positive and statistically significant. Thus, the HPQ adapted version proved valid and reliable in the sample studied.

Chronic disease; Nursing; Occupational health; Validation studies; Reproducibility of results

Estudo com objetivo de adaptar transculturalmente o instrumento Health and Work Performance Questionnaire (HPQ ) e avaliar as propriedades psicométricas da versão brasileira para enfermeiros. O processo de adaptação seguiu o Protocolo de Tradução da Organização Mundial da Saúde. Os dados para a avaliação das propriedades psicométricas foram coletados em um hospital público e de ensino de Ribeirão Preto em 2011. As propriedades psicométricas analisadas foram: validade de face e conteúdo (grupo de especialistas), confiabilidade pelo Alfa de Cronbach e estabilidade por teste-reteste. Na avaliação das propriedades psicométricas, a consistência interna da versão adaptada do HPQ, o alfa de cronbach foi de 0,94 para a seção A e 0,86 para a seção B do instrumento. Na análise das concordâncias da estabilidade teste-reteste, as concordâncias foram positivas e estatisticamente significativas. Dessa forma, a versão adaptada do HPQ mostrou-se válida e confiável na amostra estudada.

Doença crônica; Enfermagem; Saúde do trabalhador; Estudos de validação; Reprodutibilidade dos resultados

Los objetivos de este estudio, de delineamiento metodológico, fueron traducir, adaptar culturalmente para Brasil el instrumento Health and Work Performance Questionnaire (HPQ) y evaluar las propiedades psicométricas de la versión brasileña en enfermeros. El proceso de adaptación siguió el Protocolo de Traducción de la Organización Mundial de la Salud Los datos para la evaluación de las propiedades psicométricas fueron colectados en un hospital público y de educación del interior del Estado de Sao Paulo, en 2011. Las propiedades psicométricas analizadas fueron: la validez de forma y contenido (grupo de especialistas), la confiabilidad por el Alfa de Cronbach y la estabilidad por test y retest. En la evaluación de las propiedades psicométricas la consistencia interna de la versión adaptada del HPQ, alfa de cronbach fue de 0,94 para la sección A y 0,86 para la sección B del instrumento. En el análisis de las concordancias de la estabilidad test-retest las concordancias fueron positivas y estadísticamente significativas.

Enfermedad crónica; Enfermería; Salud laboral; Estudios de validación; Reproducibilidad de resultados

INTRODUCTION

The chronic noncommunicable diseases (NCDs) have become the main priority in healthcare in Brazil – 72% of deaths in 2007 were attributed to them, while 10% for infectious and parasitic diseases, and 5% for disorders of mother and child health ^{( 1 )} . This distribution contrasts with the rates found in 1930, when infectious diseases accounted for 46% of deaths in Brazilian state capitals ^{( 2 )} . It occurs in a context of economic and social development marked by important social advances and by resolution of major public health problems prevailing in the country.

Parallel to this transformation, there was a rapid demographic transition in Brazil, which produced an age group pyramid with a higher relative weight for adults and elderly. The rapid and significant changes that characterize the demographic and epidemiological transitions in Brazil ^{( 1 )} constitute an additional barrier to the administration and planning of health resources.

Following the trend of first world countries, the impact of NCDs in many developing countries continually grows and increases socioeconomic difficulties. According to the World Health Organization (WHO), NCDs and poverty create a vicious circle in countries of low and middle income, since they have a negative impact in their macroeconomic development. NCDs are responsible for the greatest economic cost to families, health system and society ^{( 3 )} .

For prevention, it is necessary to identify and deal with its causes – health risks underlying them. Each risk has its own causes and many also have their roots in a complex chain of events over time, consisting of socioeconomic factors, environmental and community conditions and individual behavior ^{( 4 )} .

Despite the epidemiological evidence showing the increased incidence of chronic diseases, few studies highlight the cost of these diseases and there is still no convincing evidence on the economic impact of NCDs.

The diseases ingeneral, particularly the chronic ones, deprive individuals from their health and their productive potential. They reduce life expectancy and therefore economic productivity, thereby depleting the quality and quantity of the labor force of countries and it could result in decreased national income ^{( 4 )} .

Increasingly, NCDs exert pressure on the Brazilian health system and, given estimates of the costs of consultations, hospitalizations and surgeries, the cost reach R$ 10.9 billion per year. Besides that, it has a strong impact on quality of life of affected individuals, since they cause premature death ^{( 5 )} .

The change in the epidemiological profile is not a simple replacement of the communicable diseases to noncommunicable diseases in the statistics, but it is grounded in a complex process that involves numerous biological and socio-economic factors related to currently urbanization and industrialization.

Such conditions have significant social implications because they affect the quality of life of the population, they also have significant impact on productivity and affect the economy of the country, since they generate absenteeism (absence from work), reduced productivity when the worker goes back to work with some health condition (presenteeism), early withdrawal of productive force due to death and disability in young adults, high costs of treatment, hospitalization and rehabilitation, decreased quality of life in general, causing pain, physical limitation and eventual dependence, increasing costs with social security ^{( 6 )} .

In Brazil, the cost of NCDs in the workplace, including heart problems, cancer, diabetes and respiratory diseases, could reach US$ 9.3 billion per year in 2015, according to a study by the World Health Organization. In 2005, such expenditure was estimated at US$ 2.7 billion. In the period 2005-2015, the cumulative loss of income is calculated at US$ 49.2 billion, including reduced productivity and increased costs of illnesses among workers ^{( 7 )} .

The growing interest in measuring the effect of health issues on the productivity of workers has fostered the scientific development of this area and the creation of several instruments that quantify this relationship ^{( 8 )} . In this sense, the Health and Work Performance Questionnaire (HPQ) of the World Health Organization (WHO) was designed asone of the most widely used self-report instruments developed to evaluate the indirect costs of illness in the workplace ^{( 9 )} . The researchers sought to develop a measurement tool to collect data from the worker in relation to their treated or not treated health problems and their job performance.

As in Brazil there are only a few studies that quantify the costs of absenteeism, early retirement and productivity loss related to the occurrence of NCDs, we conducted the research presented here, aiming to cross-cultural adapt the instrument Health and Work Performance Questionnaire ^{( 9 )} (HPQ) and evaluate the psychometric properties of the Brazilian version with nurses.

METHOD

Methodological development study whose purpose was to adapt and validate a questionnaire that assesses health problems and job performance of Brazilian nurses.

A cross-cultural adaptation was performed following the processof translation from the World Health Organization ^{( 10 )} and established by Harvard Medical School consists of the steps presented below.

Forward translation

A Brazilian health professional and translator performed translation of the Original Version of the HPQ. Instructions were given as to the focus of translation, with emphasis on conceptual rather than literal translation, as well as the need to use plain language and support for the general public. This wasthe HPQ version 1.0.

Expert panel

To perform this step, five nurses were selected, who were experts in the methodology of this study and in the area of occupational health, and the translator of the first version of the instrument. The objective was to identify and clarify expressions and concepts of the translation that were inadequate, as any discrepancies between the translation and the existing version of the instrument.

After evaluation by the panel members, we held a review of all notes and we produced the HPQ version 1.1. The result of this process was the productionof the translated version of the instrument in full.

Back-translation

The instrument was back translated into English by an independent translator, whose native language was American English,and who had no knowledge of the instrument. This version, called HPQ version 1.2 was sent to the instrument author for approval. After some suggestions, we obtained HPQ version 1.3, which was the version used for the pre-testing and cognitive interviewing steps.

Pre-testing and cognitive interviewing

During the pre-test, the time nurses dispensed to answer the instrumentwas recorded. Then, we asked them about the understanding of each statement and response of the items, besides suggestions for the instrument to become more understandable and easy to apply. All sample prints during the pre-testing stepwere carefully considered in the preparation of the final version of the HPQ.

We interviewed 10 nurses, aged between 23 and 51 years, nine women (90%) and a man, mostly married (60%). Regarding education, six had completed higher education, and four had post-graduation certificates.

Regarding the application of the instrument, all nurses answered the questionnaire alone (under supervision of the researcher). The time used by the participants to complete the questionnaire ranged from 25 to 55 minutes.

While performing this step, participantswere asked to opine on the instrument in general and on each of its items. Participants were unanimous in considering the questionnaire easy to understand, however, allparticipants had doubts on at least one question.

At the request of the HPQ author, we sent him a report containing the results obtained in the pre-testing step, including the suggestions of participants. The suggestions made during the pre-testing and cognitive interviewing steps were accepted by the author, therefore, we built HPQ version 1.4, which was named HPQ Final Version

Final version

The final version of the instrument was the result of all the steps described above. Each version received a serial number, and a total of five versions of the HPQ were produced until the final version.

Analysis of the psychometric properties of the HPQ adapted version for Portuguese Validation

The face and content validity were verified by consensus obtained by the members of the experts panel that evaluated whether the questionnaire was measuring what it intended to measure (face validity) and also the relevance of each item in the studied construct (content validity).

Reliability

The reliability of the HPQ adapted version was verified by the measure of internal consistency of the instrument, measured by Cronbach’s alpha coefficient. The stabilitywasevaluated by Test-Retest method, in whichthe instrument was applied with a range of application of two weeks.

The study was conducted at the Campus Unity of the Clinics Hospital of the Faculty of Medicine of Ribeirão Preto, Universidade de São Paulo (USP-HCFMRP). The HPQ adapted version was applied to the nurses who met the following inclusion criteria: had a degree in Nursing, agreed to respond the data collection instrument by signing the Consent Form, had an employment bond with institution. We also included nurses away from work and on vacation.

Data collection was conducted from July 1 to August 31, 2011 at their ownworkplace. The instrument was given to the subjects and collected the next day.A total of 100 instruments were returned properly filled. The nurses who were on vacation or away during data collection were later interviewed when they returned to work.

The study was approved by the Ethics Committee in Research of the Clinics Hospital of the Faculty of Medicine of Ribeirão Preto, Universidade de São Paulo (USP-HCFMRP), HCRP No. 10025/2010.

Processing and analysis of data

Data were stored in a database built in Microsoft Office Excel and then were transferred to the software Statistical Package for Social Science (SPSS) version 19.0 STATA SE, version 10.1. Afterwards we conducted the following analysis:

•   Descriptive: with production of frequency tables, measures of central tendency (mean, median, minimum and maximum) and dispersion (standard deviation) for the data of Part C of the HPQ-BV questionnaire, regarding sociodemographics;

•   Cronbach’s alpha coefficient: to verify reliability, ie, the homogeneity of the HPQ-BV questionnaire items. It was established as evidence of satisfactory internal consistency values of Cronbach’s alpha>0.70;

•   Spearman correlation coefficient: used to verify the test-retest and evaluate the validity;

•   Kappa correlation coefficient: used to describe and test the degree of agreement (reliability) of the test-retest. Results were considered significant when α≤0.05.

RESULTS

The sample consisted of 100 nurses who met the inclusion criteria proposed for this study. The data are presented in Table 1 .

Table 1
– Socio-demographic characteristics of nurses – Ribeirão Preto, SP, 2011.

Variables	N	%	Mean (±SD)
Age			33,9 (± 9,78)
Until 29 years	43	43
30 to 49 years	46	46
Over 50 years	10	10
Total	100	100
Gender
Male	11	11
Female	89	89
Total	100	100
Marital status
Single	51	51
Married	44	44
Separated	02	02
Divorced	03	03
Total	100	100
Number of kids
None	76	76
One	11	11
Two	11	11
Three	02	02
Total	100	100
Education
College/university	33	33
Post-graduation	67	67
Total	100	100

Table 2
– Spearman and Kappa correlation coefficients for section A of the HPQ adapted version – Ribeirão Preto, SP, 2011.

Variables	Obs	Mean	±SD	Minimum	Maximum
Spearman	117	0.801	0.193	0.049	1
Kappa	117	0.666	0.250	0	1

Table 3
– Spearman and Kappa correlation coefficients for section B of the HPQ adapted version – Ribeirão Preto, SP, 2011.

Variables	Obs	Mean	±SD	Mínimum	Maximum
Spearman	36	0.666	0.245	-0.107	1
Kappa	36	0.565	0.225	-0.046	1

*p<0,001

Table 4
– Spearman and Kappa correlation coefficients for section C of the HPQ adapted version – Ribeirão Preto, SP, 2011.

Variables	Obs	Mean	±SD	Mínimum	Maximum
Spearman	10	0.971	0.047	0.852	1
Kappa	9	0.920	0.133	0.618	1

*p<0,001

Table 5
– Distribution of Spearman correlation for section A and B of the HPQ adapted version – Ribeirão Preto, SP, 2011.

Correlations Spearman – Section A	Frequency	%	% Accumulated
<0.33	5	4.27	4.27
0.34 a 0.66	17	14.53	18.80
0.67 a 1.00	95	81.20	100.00
Total	117	100
Correlations Spearman – Section B	Frequency	%	% Accumulated
<0.33	2	5.56	5.56
0.34 a 0.66	15	41.67	47.22
0.67 a 1.00	19	52.78	100.00
Total	36	100

Internal Consistency

Cronbach’s alpha for the HPQ section A was 0.94 and for section B was 0.86, indicating high internal consistency. The Cronbach’s alpha of the original questionnaire was 0.74 and 0.81 ^{( 9 )} for sections A and B, respectively. If any of the questionnaire itemswere taken, it would not hadhad major changes over the alpha, which showed variation in section A of 0.0022 and 0.0143 in section B.

Test-Retest reliability

In order to conduct the retest step, we selected 30 subjects who participated in the testing step. The retest was applied approximately 15 days after the first application. To evaluate the HPQstabilitythrough test-retest we used Kappa and Spearman correlation coefficients.

We found high stability;we obtained a Kappa correlation coefficient of 0.666 for section A, 0.565 for section B and 0.920 for section C.

The mean of the Spearman correlation coefficient was r=0.801 for section A, r=0.666 for section B and r=0.971 for section C of the HPQ Brazilian version, revealing a high correlation between the values. Therefore, it means that the HPQ Final Version has high stability. Tables 2 to 4 show the results obtained.

The Spearman correlation coefficient is used to obtain exact significance level for obtained coefficient or to analyze random permutation test to obtain approximate ρ-values, where the number of possible permutations is large.

The Spearman correlations were analyzed for the HPQ adapted version and presented in tables 9 and 10 according to the reference values (0-0.33 low correlation, 0.34-0.66 average correlation and 0.67-1.00 high correlation).

DISCUSSION

The process of cultural adaptation to Portuguese language spoken in Brazil followed the international standards recommended by WHO, respecting the steps of translation, analysis by a expert panel, back-translation and pre-testing and cognitive interviewing ^{( 10 )} . Content validity of the HPQ was conducted during the expert panel. This type of validity determines the representativeness or content relevance of the items of an instrument ^{( 11 )} .

The profile of the studied sample, consisting of nurses, with 89% being female and the mean age 33.9 years, corroborates the profile of the nursing workforce in Brazil, confirmed by other studies ^{( 12 - 14 )} . In addition to their work activities, the large numbers of female workers in the profession have other duties. In addition to the professional activities they perform, many nurses have double or triple work shifts as they also have responsibilities at home and with their family.

Most researches on nurses’ work demonstrates how work-related risk factors cause illnesses in these professionals. However, a small number of studies sought to assess how the nurse worker, carrying a NCD, conducts their work activities.

In one study ^{( 15 )} , which assessed the presence of NCDs in nurses, hypertension was reported by 21.1% of participants, 18.9% reported having a family history of chronic diseases and 14.4% were obese. The majority (82.6%) reported having knowledge about the disease and its complications.

However, research on the presence of NCDs in nurses and other professional groups are still scarce in Brazil and, thereafter, are merely descriptive or focus on only one chronic condition. Specifically with nursing workers, we did not find published studies on the burden of NCDs, their impact on productivity and indirect costs generated, so that they can contribute to the development of effective programs to promote health. With the availability of HPQ-Brazil, such studies could be conducted and allow the data collection on it. Regarding the reliability of the HPQ questionnaire, it was originally recorded by the internal consistency of its items (Cronbach’s alpha) and test-retest stability ^{( 9 )} . In this study we also evaluated the reliability by internal consistency of the HPQ adapted version, and by the measurement of stability(test-retest).

Using Cronbach’s alpha to obtain the reliability of the instrument was important to reveal the degree of covariance of the instrument items in the Portuguese version because after the cross-cultural adaptation to Brazil, HPQ kept optimal values of Cronbach’s alpha. Values above 0.70 are generally acceptable ^{( 16 )} . In evaluating the Cronbach alpha, no major variationswere found. This fact was important to keep all the items of the scale and that further studies can be conducted, new populations may be involved to ratify or not the results achieved here.

One way to assess reliability is through test-retest ^{( 17 )} , a technique that allows us to assess whether similar results are obtained when the instrument is applied under the same methodological conditions, but at different times.

The degree of agreement between assessments is quantified by means of the reliability coefficients, which can be calculated in various ways. The kappa coefficient has been widely used and is considered a proper statistic for reliability of studies techniques and tools that generate categorical variables ^{( 18 )} . It considers the proportion of observed agreement and expected agreement due to chance ^{( 19 )} .

The k values range from -1.0 (agreement lower than chance) to +1.0 (perfect agreement), where k=0 means completely reliable due to chance ^{( 20 )} . The mean Kappa coefficients for the Questionnaire Health and Work Performance – HPQ-Brazil, were 0.666 for section A; 0.565 for section B and 0.920 for section C, showing good agreement of measurement reliability.

The Spearman correlation coefficient was also used to analyze the test - retest reliability. The Spearman coefficient ρ varies between -1 and 1. The closer these ends, the bigger the association between variables is. The negative sign means that the correlation variables vary in the opposite direction (negative or inverse relation), i.e. the categories with higher values of a variable are associated with lower categories of the other.

In retest step (n=30), the results revealed that the HPQ adapted version is stable, with significant correlations, revealing a high correlation agreement between values and therefore, excellent stability.

The results of the evaluation of the psychometric properties of the adapted version of the HPQ-Brazil allow the provision of a valid, reliable instrument for assessment of the profile of NCDs and the economic consequences of these diseases in the workplace and enable the evaluation of these indicators to the creationmore efficient programs of intervention.

With the development of research using the HPQ, managers/employers will have information on the NCDs profile that most affect workers and indirect costs caused by reduced work performance. Thus, they will have a greater understanding of the impact of health problems on performance and productivity at work, to promote investment in programs of health promotion interventions in the workplace.

This topic has been considered promising for the development of new knowledge, with possibilities of great use by managers/employers, from the perspective of evidence-based decisions. It is of great importance not only information about direct costs, but also indirect costs of workers with NCDs in the workplace, because they represent a large impact on economic production.

CONCLUSION

The objectives established for the study were fully met and the Brazilian version adapted from the Health and Work Performance Questionnaire – HPQ was presented after evaluating its psychometric properties, which revealed high reliability of the items.

Futures studies using the Health and Work Performance Questionnaire – HPQ-Brazil will be critical to assess the health problems that imply in higherindirect costs in nurses workforce, if such costs are associated with low rates of treatment (i.e., high costs related to work among non-treated with the diseases), or inadequate treatment (high costs related to work among those treated with disease) or if the indirect costs of work-related diseases in Brazil are comparable to the same costs in other countries.

REFERÊNCIAS

• ¹
  Schmidt MI, Duncan BB, Silva GA, Menezes AM, Monteiro CA, Barreto SM, et al. Chronic non-communicable diseases in Brazil: burden and current challenges. Lancet. 2011;377(9781):1949-61.
• ²
  Silva Junior JB. As doenças transmissíveis no Brasil: tendências e novos desafios para o Sistema Único de Saúde. In: Brasil. Ministério da Saúde. Saúde Brasil 2008: 20 anos de Sistema Único de Saúde (SUS) no Brasil [Internet]. Brasília; 2009 [citado 2013 fev. 22]. Disponível em: http://portal.saude.gov.br/portal/arquivos/pdf/saude_brasil_2008_web_20_11.pdf
  » http://portal.saude.gov.br/portal/arquivos/pdf/saude_brasil_2008_web_20_11.pdf
• ³
  Organização Mundial da Saúde. Prevenção de doenças crônicas: um investimento vital [Internet]. Brasília: OMS; 2005 [citado 2013 fev. 22]. Disponível em: http://www.who.int/chp/chronic_disease_report/part1_port.pdf
  » http://www.who.int/chp/chronic_disease_report/part1_port.pdf
• ⁴
  World Health Organization. An estimation of the economic impact of chronic noncommunicable diseases in selected countries [Internet]. Geneva: WHO; 2009 [cited 2013 Jan 20]. Available from: http://www.who.int/chp/working_paper_growth%20model29may.pdf
  » http://www.who.int/chp/working_paper_growth%20model29may.pdf
• ⁵
  Brasil. Ministério da Saúde; Agência Nacional de Saúde Suplementar. Manual Técnico de Promoção da Saúde e Prevenção de Riscos e Doenças na Saúde Suplementar [Internet]. Brasília; 2007 [citado 2013 jan. 20]. Disponível em: http://www.ans.gov.br/portal/upload/biblioteca/manual_ans.pdf
  » http://www.ans.gov.br/portal/upload/biblioteca/manual_ans.pdf
• ⁶
  Brasil. Ministério da Saúde; Secretaria de Vigilância em Saúde; Organização Pan-Americana de Saúde. A vigilância, o controle e a prevenção das doenças crônicas não-transmissíveis: DCNT no contexto do Sistema Único de Saúde brasileiro [Internet]. Brasília; 2005 [citado 2013 jan. 20]. Disponível em: http://portal.saude.gov.br/portal/arquivos/pdf/doencas_cronicas.pdf
  » http://portal.saude.gov.br/portal/arquivos/pdf/doencas_cronicas.pdf
• ⁷
  World Health Organization. Preventing chronic diseases: a vital investment [Internet]. Geneva: WHO; 2005 [cited 2013 Feb 22]. Available from: http://www.who.int/chp/chronic_disease_report/en/
  » http://www.who.int/chp/chronic_disease_report/en/
• ⁸
  Soarez PC, Kowalski CCG, Ferraz MB, Ciconelli RM. Tradução para português brasileiro e validação de um questionário de avaliação de produtividade. Rev Panam Salud Pública. 2007;22(1):21-8.
• ⁹
  Kessler RC, Barber C, Beck A, Berglund P, Cleary PD, McKenas D, et al. The World Health Organization Health and Work Performance Questionnaire (HPQ). J Occup Environ Med. 2003;45(2):156-74.
• ¹⁰
  World Health Organization. Process of translation and adaptation of instruments [Internet]. Geneva: WHO; 2007 [cited 2013 Jan 20]. Available from: http://www.who.int/substance_abuse/research_tools/translation/en/
  » http://www.who.int/substance_abuse/research_tools/translation/en/
• ¹¹
  Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382-5.
• ¹²
  Silva RM, Beck CLC, Magnago TSBS, Carmagnani MIS, Tavares JP, Prestes FC. Trabalho noturno e a repercussão na saúde dos enfermeiros. Esc Anna Nery Rev Enferm. 2011;15(2):270-6.
• ¹³
  Negeliskii C, Lautert L. Occupational stress and work capacity of nurses of a hospital group. Rev Latino Am Enferm [Internet]. 2011 [cited 2013 Jan 20];19(3):603-13. Available from: http://www.scielo.br/pdf/rlae/v19n3/21.pdf
  » http://www.scielo.br/pdf/rlae/v19n3/21.pdf
• ¹⁴
  Silva RC, Ferreira MA. Características dos enfermeiros de uma unidade tecnológica: implicações para o cuidado de enfermagem. Rev Bras Enferm [Internet]. 2011 [citado 2013 jan. 20];64(1):98-105. Disponível em: http://www.scielo.br/pdf/reben/v64n1/v64n1a15.pdf
  » http://www.scielo.br/pdf/reben/v64n1/v64n1a15.pdf
• ¹⁵
  Teixeira RC, Mantovani MF. Nurses with chronic illness: relations with the illness, prevention and the work process. Rev Esc Enferm USP [Internet]. 2009 [cited 2013 Jan 20];43(2):415-21. Available from: http://www.scielo.br/pdf/reeusp/v43n2/en_a22v43n2.pdf
  » http://www.scielo.br/pdf/reeusp/v43n2/en_a22v43n2.pdf
• ¹⁶
  Hulley SB, Cummings ST, Browner WS, Grady DG, Newman TB. Delineando a pesquisa clínica: uma abordagem epidemiológica. 3ª ed. Porto Alegre: Artmed; 2008.
• ¹⁷
  Ferreira JES, Veiga GV. Confiabilidade (teste-reteste) de um questionário simplificado para triagem de adolescentes com comportamentos de risco para transtornos alimentares em estudos epidemiológicos. Rev Bras Epidemiol [Internet]. 2008 [citado 2013 jan. 20];11(3):393-401. Disponível em: http://www.scielosp.org/pdf/rbepid/v11n3/05.pdf
  » http://www.scielosp.org/pdf/rbepid/v11n3/05.pdf
• ¹⁸
  Raggio LR. Associação estatística em epidemiologia. In: Medronho RA, organizador. Epidemiologia. Rio de Janeiro: Atheneu; 2004. p. 309-34.
• ¹⁹
  Menezes PR, Nascimento AF. Validade e confiabilidade das escalas de avaliação em Psiquiatria. In: Gorenstein C, Andrade LHSG, Zuardi AW. Escalas de avaliação clínica em psiquiatria e psicofarmacologia. São Paulo: Lemos; 2000. p. 23-8.
• ²⁰
  Spitzer RL, Fleiss JL. A re-analysis on the reability of psychiatric diagnosis. Br J Psychiatry. 1974;125:341-7.

• *
  Extracted from thesis “Adaptação transcultural e validação do World Health Organization Health and Work Performance Questionnaire para enfermeiros brasileiros”, Graduate Program in Nursing Fundamentals of School of Nursing at Ribeirão Preto, Universidade de São Paulo, 2011.  

Publication Dates

• Publication in this collection
  01 Dec 2013

History

• Received
  26 Feb 2013
• Accepted
  27 June 2013