Open-access Translation and cultural adaptation of the Perceived Stigmatization Questionnaire for burn victims in Brazil*

Traducción y adaptación transcultural del Perceived Stigmatization Questionnaire para víctimas de quemaduras en el Brasil

Abstracts

This methodological study aimed to describe the process of translation and cultural adaptation of the Perceived stigmatization Questionnaire (PSQ) and analyze the internal consistency of the items in the step of pre-testing. The PSQ was developed to evaluate the perception of stigmatizing behaviors of burn victims. The adaptation process was carried out from August 2012 to February 2013, comprising the steps outlined in the literature. As part of this process, the pre-test with 30 adult burn victims was held. All participants at this step reported to understand the instrument items and the scale of responses. There were no suggestions or changes in the tested version. The value of Cronbach’s alpha at pre-test was 0.87. The contribution of this study is to describe the operation of each of the steps of this methodological process and show the internal consistency of the items in the pre-test.

Burns; Stereotyping; Validation studies; Cross-cultural comparison; Translating


Este estudio tuvo como objetivos describir el proceso de traducción y adaptación cultural del instrumento Perceived Stigmatization Questionnaire (PSQ) y analizar la consistencia interna de los ítems en la etapa del pre-test. El PSQ fue desarrollado para la evaluación de la percepción de los comportamientos estigmatizantes de víctimas de quemaduras. El proceso de adaptación fue realizado de agosto del 2012 a febrero del 2013, incluyendo las etapas preconizadas en la literatura. Como parte de ese proceso, fue realizado el pre-test con 30 adultos víctimas de quemaduras. Todos los participantes de esa etapa informaron comprender los ítems del instrumento y la escala de respuesta. No hubo sugerencias o cambios en la versión probada. El valor del alfa de Cronbach en el pre-test fue de 0,87. La contribución del estudio está en la descripción de la operacionalización de cada una de las etapas de ese proceso metodológico y demuestra la consistencia interna de los ítems en el pre-test.

Quemaduras; Estereotipo; Estudios de validación; Comparación transcultural; Traducción


Este estudo metodológico teve como objetivos descrever o processo de tradução e adaptação cultural do instrumento Perceived Stigmatization Questionnaire (PSQ) e analisar a consistência interna dos itens na etapa do pré-teste. O PSQ foi desenvolvido para avaliação da percepção de comportamentos estigmatizantes de vítimas de queimaduras. O processo de adaptação foi realizado de agosto de 2012 a fevereiro de 2013, compreendendo as etapas preconizadas na literatura. Como parte desse processo, foi realizado o pré-teste com 30 adultos vítimas de queimaduras. Todos os participantes dessa etapa reportaram compreender os itens do instrumento e a escala de respostas. Não houve sugestões ou alterações na versão testada. O valor do alfa de Cronbach no pré-teste foi de 0,87. A contribuição do estudo reside em descrever a operacionalização de cada uma das etapas desse processo metodológico e mostrar a consistência interna dos itens no pré-teste.

Queimaduras; Estereotipagem; Estudos de validação; Comparação transcultural; Tradução


Introduction

Burn is a trauma that affects people's lives and cause changes in body image that can cause psychological damage to the individual(1). The disfigurement is an effect of the change in appearance of the burn trauma, which can be considered outside of the expected social standards and lead to rejection and social isolation, decreased self-esteem, stigmatization, anxiety and depression(2-3).

Stigmatization is the process or the individual's situation that prevents full social acceptance. To be recognized by society as someone who does not have expected patterns, who presents a health problem or other stigmatizing situation, the individual is often labeled, avoided, ridiculed and degraded by people(4).

People burned reported that there are two critical points associated to the perception of changes in their bodies: when they see their own image in the mirror and when they notice how others see them. After hospital discharge, burned individuals with changes in appearance and function of body parts start to suffer the shock from contact with people in social life(5). Thus, we emphasize the importance of assessing individuals when they come in contact with the view of society, outside the context of a Burns Unit.

In recent years, measurement instruments have been developed to assess psychosocial factors for various types of medical conditions and rehabilitation, including burn victims. However, most of these instruments were originally developed in English and therefore, they need to be translated and cross-culturally adapted for use in other countries(6-7).

The cross-cultural adaptation process comprises the translation and cultural adaptation of the instrument to the population where it will be applied and has the purpose of maintaining the semantic equivalence of the original to the adapted version. Thus, we attempted to ensure the psychometric properties of the items through validity and reliability(7). The validity evidence whether what is being measured is what the researcher is intending to measure, and reliability of the instrument assesses the degree of consistency with which the instrument measures the proposed attribute(8-9). However, if any item is not understood by the new population, it must be carefully modified. So, the instrument may change and new statistical tests should be performed after the last translation(7).

The use of instruments has been favoring the communication between healthcare professionals and service users, contributing to the rapid identification of problems and priority needs, as well as for decision making in search of improving care(10).

In the national literature, no specific instruments have been identified for assessment of stigmatization in patients who suffered burns that has been adapted for use in Brazil. In the international literature, we identified Perceived Stigmatization Questionnaire (PSQ), developed in English to assess stigmatizing behaviors commonly reported by children, adolescents and adults who have suffered burns in the United States(11-12), and an adaptation for use with children in Germany(13).

If it is valid for the Brazilian population, it may allow comparison of research findings across countries and effects of interventions in Brazilian patients with burn victims in other countries. It can be used to assess the effects of support systems and other interventions aimed at minimizing the impact with aesthetic procedures such as cosmetics makeup. Thus, it becomes necessary to perform the adaptation process of the instrument to Brazil.

The objectives of the study were to translate and culturally adapt the Perceived stigmatization Questionnaire (PSQ) to the Brazilian Portuguese for use of burn victims and analyze the internal consistency of the items in the pre-test step.

Method

Methodological study performed in a Burn Unit of a teaching hospital located in the countryside of the state of Sao Paulo.

Inclusion criteria

We selected for the pre-test step, individuals who met the following inclusion criteria: age over 18 years, Brazilians, regardless of gender, race and ethnicity, who were discharged between two and 12 months, had an outpatient appointment in the Burn Unit and who were awaiting reconstructive surgeries after hospital discharge.

Individuals with prior psychiatric diagnoses (registered in patients charts) or cognitive difficulties that prevented them from answering the questions of the instruments (assessed by the ability to indicate the address where they lived, day of week and month, and age or date of birth) were excluded, who were serving time in prison in a closed regime or burned in suicide attempt.

Ethical aspects

The study was approved by the Ethics Committee of the Ribeirão Preto College of Nursing, University of São Paulo under protocol No. 243/2012. All participants of the steps of the study, the individuals and experts, received verbal and written information about the study and agreed to participate by signing the Consent Form. Anonymity was preserved by identifying the questionnaire with numbers and the interviews were held in a private environment.

Instruments

The PSQ was developed in the United States with the objective of assessing stigmatizing behaviors of people burned. The questionnaire was based on behavior reported by people with changes in appearance, considering that certain types of emotions give space to specific behaviors that may be offensive or not, i.e., the behavioral manifestation of stigma may differ depending on the type of emotion caused(14). Stigmatizing behaviors that underlie the construction of the questionnaire were identified by the authors in the literature review and were organized into six categories: improper attention and stare, avoidance behavior, confuse behavior, rude and provocative behavior, bullying and external pressure to change their appearance(12).

The PSQ consists of 21 items divided into three domains: Absence of friendly behavior (items 1, 5, 7, 9, 12, 15, 17 and 20), confused/staring behavior (items 3, 4, 6, 10, 13, 14, 19 and 21) and hostile behavior (item 2, 8, 11, 16 and 18). The individual must indicate how often experiences such behavior on a scale with response alternatives 1-5 (never, almost never, sometimes, often, always). The scores of the items of the domain Absence of friendly behavior should be reversed prior to statistical analyzes(12).

The PSQ total score must be calculated by summing the items and dividing by the number of items in the instrument. The same procedure must be performed to calculate the score of each domain, by dividing the number of items in the respective domain. The results of the PSQ are expressed by mean scores on the same metric scale of frequency response (1-5 points) so that the scale scores are comparable and easily interpretable. Higher scores indicate greater perception of stigmatizing behavior. The internal consistency of the original instrument was assessed by Cronbach's alpha and the value of 0.93 was obtained(12).

Together with the PSQ, we used a questionnaire to collect sociodemographic data of subjects in the pre-test, such as gender, marital status, years of education and age.

Procedures for cross-cultural adaptation

The authorization for cross-cultural adaptation of the PSQ in Brazil was granted by the original author via email. The adaptation process was conducted according to the steps outlined in the literature described below(6-7,15-16):

• Instrument translation: performed by two independent translators, Brazilians, with English language and culture expertise and knowledge of the objectives of the study, however, they did not know the original English version of the instrument, resulting in the Portuguese version of each translator (PV1 and PV2).

• Obtaining the first consensus of Portuguese versions: for the synthesis of translations and obtaining the first consensus of the translated version, a meeting between the researchers and translators was performed. The two versions in Portuguese, along with the original English versions were evaluated with the objective to compare them and rank the items that kept the semantic equivalence with the original instrument. This step resulted in the consensus of Portuguese version 1 (CPV1) of PSQ.

• Evaluation by Experts Panel: consensual Portuguese version was submitted for review by a panel consisting of five experts. The experts who were selected to participate in the Panel met at least two of the following criteria: have knowledge about burn individuals, know Portuguese and English languages and research methodology. After selection, a meeting was conducted to evaluate the semantic, idiomatic, cultural and conceptual of items from consensual version of the Portuguese instrument and compared with the original version in English. When any member of the Panel disagreed with any term of the translated version, suggestions were proposed, analyzed and justified in order to achieve better understandability of instruments and equivalence with Brazilian culture. The approval of the changes occurred when 80% of the Panel members agreed with the proposal(16). In this step, we evaluated the face and content validity of the instrument by the Panel members and at its end, we obtained the consensus Portuguese version 2 (CPV2).

• Back-translation: step characterized by the back translation of the original language of the instrument, English. It was performed by two independent translators, foreign, fluent in English, Portuguese and who had knowledge of the Brazilian culture. The translators did not know the research objectives and the original instrument. The objective of this step was to determine whether the Portuguese version reflect the original content. This step resulted in the English version of each translator (EV1 and EV2)

• Obtaining the consensus of versions in English and compared with the original English version: Once the back-translation was completed, we held a meeting between the researchers and translators in order to compare the translated, back-translated and original English version to verify the possible semantic differences in order to reach a consensus on equivalent terms to the original version. Thus, the consensus Portuguese version 3 (CPV3) of the instrument was defined and the consensual English version (CEV).

• Evaluation of the instrument by the original author: performed in order to ensure semantic, idiomatic, cultural and conceptual equivalences between the original and adapted instruments versions. At this step, the consensual English version (CEV) was sent to the original author of the review, resulting in the English version corrected by the original author (PSQ –EVOA).

• Semantic analysis of the items: consensus Portuguese version 3 of the instrument underwent semantic evaluation of four patients, who were selected by convenience, according to the inclusion and exclusion criteria of the study, during the outpatient clinic hospitalization of the Burns Unit, from January 17th to 23rd 2013. The semantic evaluation aims to assess whether all terms of each item of the instrument are understandable to the population which is intended(16). The author suggested this step should be performed with four subjects(16). The step was performed by applying a questionnaire to semantic analysis(17), adapted for use with Brazilian(18) and for this study. The questionnaire is divided into two parts, Part A, related to the general impression of the adapted version of the PSQ and Part B, with specific questions about the importance and difficulties of understanding each PSQ item. An interview with each subject was conducted by reading the instrument and the semantic analysis form.

• Pre-test: we selected a consecutive sample for convenience of 30 patients who were attended at the Burns Unit, from January 24th to February 26th 2013. Patients were invited to participate in the study after evaluation of one of the researchers on the inclusion and exclusion criteria. The adapted version of the PSQ was applied by researcher interview, which conducted the joint reading of each item. The reading of each item could be held up to three times, and after these attempts, if the patient had not understand it, the item would be left blank. The subject was asked about the difficulties and comprehensibility of the adapted version of the PSQ. The number of participants in the pre-test step was defined according to suggested in the literature for this step (30-40 individuals). In this step, the instrument is applied so that we could check their understanding and acceptance of the instrument as a whole, of each item and response scale. Psychometric analyzes are not recommended, such as construct validity and reliability, in the pre-test step(7).

The final result of the methodological process of cultural adaptation of the instrument is shown in Figure 1. Data were analyzed using the Statistical Package for Social Science (SPSS) version 20.0. Descriptive analyzes were performed on all variables. We calculated measures of dispersion (standard deviation, minimum and maximum) and central tendency (mean and median) for continuous variables and single frequency for categorical variables were calculated. To determine the internal consistency of items, Cronbach’s alpha was calculated, considering that values above 0.70 reflect high degree of internal consistency(8). For analysis of the values of the correlation coefficients proposed in the literature, in which correlations with values below 0.30 are considered weak, between 0.30 and 0.50 they are considered moderate and above 0.50 they are considered strong(19).

Figure 1

Results

After translation, CPV1 was obtained, subsequently evaluated by an experts panel, whose members proposed changes in relation to the semantics of the items and the presentation of the scale by inserting the answer choices in the table. These suggested modifications are presented in Chart 1.

Chart 1

At the meeting between researchers and translators who performed the back translation, they obtained consensus among the English versions to compare with the original, being held changes in CPV2 that are united in Chart 2.

Chart 2

The translators have suggested the inclusion of the word bullying in item 16 so that the population may better understand the question, explaining that most Brazilians know the term due to the approach of the media on this issue.

After evaluation of the final version of back-translation and comparison with the original, translators and researchers had the need to assess and amend certain terms of the Portuguese version, although this procedure was not described in the method.

In response scale, the English option Often was translated and unchanged in the step of the expert Panel and subsequently the back-translation, with the final version of the translation for Many times.

The English version of the PSQ was sent to the original author who had always demonstrated availability and responded quickly to emails, suggesting alterations in four items. So we reconvened the experts panel to analyze the author’s suggestions. This step was not initially foreseen in the methodology. Panel expert members agreed with the suggestions of the original author, corrections being made in the English and Portuguese version, resulting in consensual Portuguese version 4 (CPV4 ). The modifications are shown in Chart 3.

Chart 3

The step of semantic analysis was performed on four patients. The mean age of participants was 37.8 years, equally distributed between the genders. Education of four patients evaluated in years had a mean of 7.3 years (range: 3-11, SD = 3.5 years).

In questionnaire of the general impression on instrument, four participants rated the instrument as good and easy questions to be understood. In the specific analysis of the PSQ questionnaire, no participant failed to answer any question. Most of the items were understood except for item 1 The people are friendly to me and Item 2 People call me names which were not understood by 25% of individuals. However, no suggestions to change the items were made. Items 2 People call me names and 8 People laugh at me were not relevant to 50% of individuals. Regarding the presentation of the scale, two individuals suggested the distribution of items and responses in different columns and each item in a row due to the difficulty to view and respond the instrument.

The suggested changes were discussed among researchers and a member of the Expert Panel, who all agreed with the suggestions of the participants, resulting in the final consensus version (PSQ - FCV).

The final step of cultural adaptation consisted of the pre-testing of the final consensus version of PSQ with 30 individuals through interviews. The mean time to answer the instrument was three minutes.

Regarding the characteristics of participants in the pre-test, 53.3% were female and 50% were single. The mean age was 36.2 years (SD = 10.9, range 18.9-58.1) and mean years of education was 8.6 (SD = 4.2, range of 1-22).


Table 1 - Descriptive analysis of the responses to the items, total score and domains of the PSQ in pretest step - Ribeirão Preto, SP, 2013


For the total score, Cronbach’s alpha obtained for 30 participants in the pre-test was 0.87. Cronbach’s alpha was 0.80 for the domain Absence of friendly behavior, 0.79 for confused/staring behavior and 0.84 for hostile behavior domain. Table 2 presents the results of the coefficient of item-total correlation and Cronbach’s alpha if the item was excluded.


Table 2 - Maternal mortality specific for hemorrhage, occurring in the State of Santa Catarina by health macroregion, in the period 1997-2010


As the subjects who participated in the pretest demonstrated understanding and acceptance of Scale, no changes were performed in PCV4 of the instrument, which resulted in the PSQ – final Portuguese version.

Discussion

This study aimed to describe the process of translation, cultural adaptation and pre-testing an instrument that evaluates the perception of stigmatizing behavior for use in Brazil with individuals who have suffered burns. The step of cultural adaptation was performed according to the methodology suggested in the literature(6-7,15-16). These steps were adopted for validation study of instruments of burn patients in Brazil(20).

The review by the Experts Panel was characterized as an important step to identify terms that could not be understood by the target population, allowing them to be changed before the back-translation, as suggested by the authors(15).

The experience of the members of the Expert Panel is considered important on the thematic of the instrument(6), in this case burns, as well as the knowledge of socioeconomic aspects showing the general profile of Brazilian burned patients. This is an important point, because it can influence the understanding of the Brazilian population adapted instruments from another culture. Study conducted in a burn unit identified that the majority of individuals refers to a low level of education, which has proven to be a common feature of patients hospitalized to the burn unit in Brazil(21).

The review conducted by researchers and translators of the Portuguese version of the PSQ, after obtaining the consent of the back-translated version, considering the English expressions that could not have a precise match with Portuguese, helped to maintain the semantic equivalence of versions. The difficulties in translating expressions of the English language and maintenance of semantics according to the cultural context were observed in all steps of translation and review Panel.

One of the strengths of the translation process was sending the instrument after back-translation into English for review by the original author. This step was feasible due to the availability of the author who constantly answered emails promptly. It was possible to perform it as proposed by the authors(7).

The second evaluation by the Expert Panel, to analyze the suggestion of the original author, not initially intended, allowed the assessment and modification of the terms suggested by the original author. This step followed the same procedure of the translation synthesis and was carried out for obtaining consensus between versions translated and then back-translated(7). At this step, there was no involvement of translators, it would be advisable to review the suggestions of the author and translate them.

The analysis of item-total correlation coefficient, most items showed correlation values of moderate to strong, but in item 17 (Strangers are polite with me) we identified a weak and negative correlation. The evaluation was conducted with a small sample and, considering this to be an item that evaluates the perception of stigmatizing behavior by burned patient when they find themselves in front of strangers, we intend to evaluate in the future the psychometric properties of the instrument in a larger sample.

For descriptive analyzes, the values obtained for the total score and for the PSQ domains in this study with 30 subjects, we were close to the original study(12). Likewise, the values obtained in the internal consistency of the full instrument and from the domains were satisfactory even with a small step in the pre-test sample. In the validation of the original version of the PSQ, the author of the instrument obtained Cronbach’s alpha of 0.93 for the total instrument, 0.91 for the domain Absence of friendly behavior, 0.92 for the domain confused/staring behavior and 0.88 for hostile behavior(8-12), similar to data obtained in the present investigation.

Other tests are needed to confirm the reliability and validity of this instrument for use with Brazilians who suffered burns. However, considering the accuracy of the values of Cronbach’s alpha is dependent on the number of subjects in the sample, the values obtained in the pre-test may be underestimated, since there is a very small number of subjects in this study than recommended for analysis of reliability(22).

Therefore, the adapted version of the PSQ will be submitted for construct analysis and reliability in a representative sample of burn victims in Brazil, which may confirm or not whether the psychometric properties of the original version remained in the adapted version.

Conclusion

Methodological studies with the aim of adapting instruments must be designed to keep seeking the idiomatic, semantic and conceptual equivalence of the adapted version in relation to the original. This study showed the importance of the inclusion of a new step of expert panel evaluation on the presence of two translators to undertake the review of the suggestions from the original author.

The cultural adaptation and pre-testing steps are important so that the equivalences are analyzed and acceptance of the items may be initially observed in the target population. Considering that Cronbach’s alpha values depend on the number of items in the instrument and in the number of individuals, the values obtained in the pre-test may be underestimated.

It is noteworthy that the process of validation of the PSQ for burn victims in Brazil is in progress. In the future, after the validation of the PSQ, it will be possible to obtain data to evaluate the perception of stigmatization of the burned patient with regard to the impact of scars on the social interaction and comparison with results from other studies.

This methodological study contributes to teaching and research, it describes in detail each step and its implications for the process of cross-cultural adaptation of instruments to assess subjective constructs.

References

  • 1Carlucci VDS, Rossi LA, Ficher AMFT, Ferreira E, Carvalho EC. A experiência da queimadura na perspectiva do paciente. Rev Esc Enferm USP. 2007;41(1):21-8
  • 2Bernstein NR, Breslau AJ, Graham JA. Coping strategies for burn survivors and their families. New York: Hardcover; 1988
  • 3Lansdown R, Rumsey N, Bradbury E, Carr T, Partridge J. Visibly different: coping with disfigurement. Oxford: Butterworth-Heineman; 1997
  • 4Gofmman E. Estigma: notas sobre a manipulação da identidade deteriorada. 4ª ed. Rio de Janeiro: Jorge Zahar; 1982
  • 5Bergamasco EC, Rossi LA, Amancio AGC, Carvalho EC. Body image of patients with burns sequellae: evaluation through the critical incident technique. Burns. 2002;28(1): 47-52
  • 6Guillemin F, Bombardier C, Beaton DE. Cross-cultural adaptation of health related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417-32
  • 7Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self report measures. Spine. 2000;25(24):3186-91
  • 8Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. Chichester, England: John Wiley & Sons; 2007. Scores and measurements: validity, reliability, sensitivity; p. 43-71
  • 9Pasquali L. Psychometrics. Rev da Esc Enferm USP [Internet]. 2009 [cited 2013 May 16];43(n.spe):992-9. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/en_a02v43ns.pdf
    » http://www.scielo.br/pdf/reeusp/v43nspe/en_a02v43ns.pdf
  • 10Higginson IJ, Carr AJ. Measuring quality of life: using quality of life measures in the clinical setting. BMJ. 2001;322(7297):1297-300
  • 111Lawrence JW, Rosenberg L, Rimmer RB, Thombs BD. Perceived stigmatization and social comfort: validating the constructs and their measurement among pediatric burn survivors. Rehabil Psychol. 2010;55(4):360-71
  • 12Lawrence JW, Fauerbach JA, Heinberg LJ, Doctor M, Thombs BD. The reliability and validity of the Perceived Stigmatization Questionnaire (PSQ) and the Social Comfort Questionnaire (SCQ) among an adult burn survivor sample. Psychol Assess. 2006; 18(1):106-11
  • 13Masnari O, Landolt MA, Roessler J, Weingaertner SK, Neuhaus K, Meuli M, et al. Self-and parent-perceived stigmatisation in children and adolescents with congenital or acquired facial differences. J Plast Reconstr Aesthet Surg. 2012;65(12):1664-70
  • 14Mackie DM, Devos T, Smith ER. Intergroup emotions: explaining offensive action tendencies in an intergroup context. J Pers Soc Psychol. 2000;79(4):602-16
  • 15Ferrer M, Alonso J, Prieto L, Plaza V, Monsó E, Marrades R, et al. Validity and reliability of the St George’s Respiratory Questionnaire after adaptation to a different language and culture: the Spanish example. Eur Respir J. 1996;9(6):1160-6
  • 16Pasquali L. Instrumentos psicológicos: manual prático de elaboração. Brasília: LabPAM/IBAPP; 1999
  • 17DISABKIDS Group. DISABKIDS translation and validation procedure. Guidelines and documentation form. Hamburgo: The Disabkids Group Europe; 2004
  • 18Fegadolli C, Reis RA, Tortelboom S, Bullinger M, Santos SB. Adaptação do módulo genérico DISABKIDS® para crianças e adolescentes brasileiros com condições crônicas. Rev Bras Saúde Matern Infant. 2010;10(1):95-105.
  • 19Ajzen I, Fishben M. Overview: understanding attitudes and predicting social behavior. New Jersey: Prentice Hall; 1998
  • 20Ferreira E, Dantas RAS, Rossi LA, Ciol MA. The cultural adaptation and validation of the Burn Specific Health Scale-Revised (BSHS-R): version for Brazilian burn victims. Burns. 2008;34(7):994-1001
  • 21Rossi LA, Costa MCS, Dantas RAS, Ciofi-Silva CL, Lopes LM. Cultural meaning of quality of life: perspectives of Brazilian burn patients. Disabil Rehabil. 2009;31(9):712-9.
  • 22Charter RA. Study samples are too small to produce sufficiently precise reliability coefficients. J Gen Psychol. 2003;130(2):117-29
  • *
    Project of the Research Group on Rehabilitation and Quality of Life, Ribeirão Preto College of Nursing, University of São Paulo. Ribeirão Preto, SP, Brazil
  • Acknowledgements:
    We thank the State of Sao Paulo Research Foundation (FAPESP, Process 2010 – 10006-8) and National Council for Scientific and Technological Development (CNPq) for research support and doctoral scholarship, respectively. Correspondence addressed

Publication Dates

  • Publication in this collection
    Feb 2014

History

  • Received
    12 Aug 2013
  • Accepted
    31 Oct 2013
location_on
Universidade de São Paulo, Escola de Enfermagem Av. Dr. Enéas de Carvalho Aguiar, 419 , 05403-000 São Paulo - SP/ Brasil, Tel./Fax: (55 11) 3061-7553, - São Paulo - SP - Brazil
E-mail: reeusp@usp.br
rss_feed Acompanhe os números deste periódico no seu leitor de RSS
Acessibilidade / Reportar erro