Abstract
OBJECTIVE: To present the process of transcultural adaptation of the Richmond Compulsive Buying Scale to Brazilian Portuguese. METHODS: For the semantic adaptation step, the scale was translated to Portuguese and then back-translated to English by two professional translators and one psychologist, without any communication between them. The scale was then applied to 20 participants from the general population for language adjustments. For the construct validation step, an exploratory factor analysis was performed, using the scree plot test, principal component analysis for factor extraction, and Varimax rotation. For convergent validity, the correlation matrix was analyzed through Pearson's coefficient. RESULTS: The scale showed easy applicability, satisfactory internal consistency (Cronbach's alpha=.87), and a high correlation with other rating scales for compulsive buying disorder, indicating that it is suitable to be used in the assessment and diagnosis of compulsive buying disorder, as it presents psychometric validity. CONCLUSION: The Brazilian Portuguese version of the Richmond Compulsive Buying Scale has good validity and reliability
Compulsive Behavior; Psychometrics; Psychiatric Status Rating Scales; Diagnosis
ORIGINAL ARTICLE
Cross-cultural adaptation, validation and reliability of the brazilian version of the Richmond Compulsive Buying Scale
Priscilla LeiteI; Bernard RangéII; Monika Kukar-KineyIII; Nancy RidgwayIII; Kent MonroeIII; Rodolfo Ribas JuniorIV; J. Landeira-FernandezV; Antonio Egidio NardiVI Adriana SilvaVII
IInstitute of Psychology, Institute of Psychiatry, Universidade Federal do Rio de Janeiro, Brazil
IIUniversidade Federal do Rio de Janeiro, Graduate Program in Psychology
IIIUniversity of Richmond, United States
IVInstitute of Psychology, Universidade Federal do Rio de Janeiro, Brazil
VInstitute of Psychology, Pontifícia Universidade Católica do Rio de Janeiro, Brazil
VIInstitute of Psychiatry, Universidade Federal do Rio de Janeiro, Brazil
VIILaboratory of Panic and Respiration, Universidade Federal do Rio de Janeiro; Universidade Federal Fluminense, Brazil
Corresponding author Corresponding author: Priscilla Leite. Universidade Federal do Rio de Janeiro, Instituto de Psicologia Av. Pasteur, 250, Praia Vermelha Rio de Janeiro, RJ CEP 22290-902, Brazil E-mail: priscilla.ufrj@gmail.com
ABSTRACT
OBJECTIVE: To present the process of transcultural adaptation of the Richmond Compulsive Buying Scale to Brazilian Portuguese.
METHODS: For the semantic adaptation step, the scale was translated to Portuguese and then back-translated to English by two professional translators and one psychologist, without any communication between them. The scale was then applied to 20 participants from the general population for language adjustments. For the construct validation step, an exploratory factor analysis was performed, using the scree plot test, principal component analysis for factor extraction, and Varimax rotation. For convergent validity, the correlation matrix was analyzed through Pearson's coefficient.
RESULTS: The scale showed easy applicability, satisfactory internal consistency (Cronbach's alpha=.87), and a high correlation with other rating scales for compulsive buying disorder, indicating that it is suitable to be used in the assessment and diagnosis of compulsive buying disorder, as it presents psychometric validity.
CONCLUSION: The Brazilian Portuguese version of the Richmond Compulsive Buying Scale has good validity and reliability
Descriptors: Compulsive Behavior; Psychometrics; Psychiatric Status Rating Scales; Diagnosis.
Introduction
Compulsive buying disorder, also known as oniomania, is characterized by an irresistible and repetitive urge to buy. Although there are different descriptions in the literature, the vast majority of people with this disorder experience excessive worrying and poor impulse control related to spending, chronic purchasing, and repetitive, compulsive buying of unnecessary items.1,2
In 1915, Emil Kraepelin (1856-1926) described oniomania as a disorder characterized by a pathological urge to buy. He emphasized that impulsiveness was a primary factor in this behavior.3 Although some authors admit that there is a strong parallel between the symptoms of obsessive-compulsive disorder (OCD)4,5 and compulsive buying disorder (CBD), the latter is classified as an impulse control disorder,2 which is more commonly observed in women between the ages of 18 and 30 years. The prevalence of this disorder in the general population is approximately 2%, but it may be more prevalent in industrialized countries. For example, estimates from the US indicate prevalence rates of approximately 6%. Research on compulsive buying should be extended for the following two reasons: one, there is evidence that the prevalence of this disorder is increasing and, secondly, this is an underestimated disorder as a result of evaluation problems.6-9
Ridgway et al.9 define compulsive buying as a consumer tendency to worry about the act of purchasing, which is revealed through repetitive buying and a lack of impulse control with regard to buying. According to these authors, most scales that are used to measure this disorder do not examine both the obsessive-compulsive symptoms (e.g., persistent and repetitive worrying) and the impulse control symptoms (e.g., low impulse control to purchase items) during the act of buying. Thus, Ridgway and colleagues designed a new scale to measure compulsive buying that considers both of these dimensions.
Significant harm to the individual, family or close friends10 is necessary for a behavior to be classified as psychiatric or disruptive. For this reason, many existing diagnostic scales include damages in the measures. With regard to compulsive buying, the main loss is debt, which results in extreme financial11,12 and emotional disorders.13,14 An individual may also suffer losses associated with family, social, and professional relationships; however, few studies have focused on these aspects.
Regarding construct validity, compulsive buying and its consequences are different factors that should be evaluated separately.15,16 Ridgway et al.9 created a scale that emphasizes the identification of behavioral tendencies that underlie the disorder. Moreover, the authors argue that public concern for compulsive buying is not confined to patients with psychiatric disorders; there is a relatively widespread belief that individuals who are not diagnosed with this disorder may be compulsive buyers.
Ridgway et al.9 reasoned that the focus on measuring the financial consequences of compulsive buying limits the ability of many of the existing scales to properly identify compulsive buyers. The main contributions of the Richmond Scale to compulsive buying were to include the OCD dimensions and the lack of impulse control in the construct. Thus, the conceptualization of this disorder extended the diagnosis of an inappropriate behavior related to the acquisition of goods to people who did not have a history or a previous diagnosis of compulsive disorder.
Although several scales assess compulsive buying, many of them have deficiencies in measuring and diagnosing the disorder9. The Yale-Brown Obsessive-Compulsive Scale-Shopping Version (YBOCS-SV),17 for example, focuses exclusively on the dimension of OCD whereas the Compulsive Buying Scale (CBS)11,18 only includes items related to impulse control. The Richmond Compulsive Buying Scale (RCBS) overcomes these limitations by assessing these two components simultaneously.
The original version of the RCBS was developed in the following manner: Ridgway et al.9 created a list of 121 potential items for the construction of the scale after reviewing more than 300 scientific articles on compulsive buying. These potential items were examined by judges, who eliminated redundancies and ambiguities among the potential items and evaluated the consequences of compulsive buying. Subsequently, the list was reduced to 15 items, which were then administered to a sample of 352 undergraduate students, with a mean age of 21 years and 54% women. An exploratory factor analysis (EFA) was conducted. In accordance with the research hypotheses, the analysis indicated that two main factors were responsible for 69% of the total variance of the scale. Six of the 15 items did not appear on the factors identified and were excluded from the scale. Following a confirmatory factor analysis (CFA), three items were removed for failure to load on the two factors found. The analysis confirmed that the two dimensions were correlated (r = .77). Thus, the six remaining items showed satisfactory reliability. Cronbach's alphas for the subscales regarding buying concern and impulsive buying were .77 and .78, respectively.
The scale was also distributed to 555 participants to determine convergent validity. The six items of the scale were measured using a 7-point Likert scale. The authors of the study also distributed the CBS, as well as scales for measuring financial consequences, materialism index, stress, depression, and anxiety disorders.
Discriminant validity verified the relationship between the tendency to compulsively buy and OCD; both have a compulsive component, and thus, they may be positively correlated. The scale was also distributed to individuals who either did or did not have a diagnosis of being compulsive buyers. A cutoff of 24 points was set such that values above 24 points were considered to be indicators of the presence of CBD.
All of these tests indicated that the RCBS is a reliable and sensitive measure that can be applied to the general population, including individuals who have not been diagnosed as compulsive buyers. Being a short, easy to complete scale (it can be used via internet), it is also of simple interpretation.
Despite its importance, only a few Brazilian studies focus on CBD.19,20 Moreover, no scales have been validated in Brazil to measure this disorder. This study was conducted to reduce this knowledge gap, by means of the adaptation and validation of the RCBS7 to a Brazilian version
Methods
Participants
This study included 254 adult participants, who were older than 18 years and who were able to understand and sign an informed consent. The inclusion criterion was the diagnosis of compulsive buying, with or without comorbid disorders, such as depression and anxiety, confirmed by clinical interview with the Mini International Neuropsychiatric Interview (MINI), version 5.0. The exclusion criteria included the diagnosis of any personality disorder, bipolar disorder, severe depression with suicidal ideation, schizophrenia, reading difficulty, age less than 18 years, and disagreement with the informed consent.
Participants were divided into four groups. The non-clinical (NC) group included 202 participants, and the three other groups comprised the remaining 52 participants. These 52 participants were referred to participate in this study by clinical psychologists or call centers and were screened as positive for the following mental disorders: OCD (15), Impulse Control Disorder (ICD) (15), and CBD (22).
Instruments
Consent forms and the socio-demographic questionnaire were distributed beforehand to assess age, gender, marital status, occupation, and education level. The following scales were used: MINI21 5.0, YBOCS-SV,17 adapted to Portuguese by Tavares22 to diagnose CBD, the Compulsive Buying Scale (CBS)11,18 adapted to Portuguese by Leite,23 and the RCBS.9 To evaluate levels of depression and anxiety, the Beck Depression Inventory (BDI) and Beck Anxiety Inventory (BAI), originally created by Beck24,25 and colleagues and adapted to Portuguese by Cunha,26 were used.
Adaptation
The semantic adaptation process for the RCBS was primarily composed of five steps, which were based on suggestions made by Skevington.27 A translator, two psychologists and one bilingual mental health expert performed the initial translation from the scale's source language to Portuguese. The Portuguese version was retranslated into English (i.e., back-translated) by two translators and a psychologist. This version was then sent to the authors of the original scale to evaluate and approve the adjustments that had been made to translate the instrument into Portuguese. Then, five psychology experts assessed the clarity of language, its theoretical relevance, and the dimensions evaluated by the scale.28,29 As a pre-test, the instrument was administered to 20 participants from the general population who had different education levels. The final version was translated and adapted into Portuguese and is shown in Appendix I Appendix I .
Procedure
All participants signed the consent forms after being informed of the goals and procedures of the study, issues related to confidentiality and the voluntary nature of their participation. Procedures were adopted to administer the questionnaires online, over the Internet, in accordance with application specific needs. Thus, patients were assessed through the MINI 5.0 and diagnosed by mental health professionals, who also recruited clinical subjects to participate in the study. NC subjects volunteered after a widespread dissemination of the research, performed directly by the main researcher among groups of mental health professionals and support groups for mental disorders that were part of the composition of our clinical sample. Participants were invited via e-mail and all subjects who signed the consent, were summoned to answer the online version of the questionnaire. Participants were requested to access the website that was hosting the questionnaire. Each participant could accept or deny participation; in order to participate, they had to fill out a protocol with seven instruments used in this research. Participants responded to questions regarding their consumption habits and completed Y-BOCS-SV, CBS, RCBS, BDI, and BAI.
Characteristics of the study sample were descriptively presented with mean and standard deviation. Chi-square test and analysis of variance (ANOVA) were performed to assess differences between the groups. Values of p < .05 were considered significant. The scales' reliability was assessed using Cronbach's alpha.30 An exploratory factor analysis (EFA) with Varimax rotation was performed to define the construct validity.31 The index of adequacy of the Kaiser-Meyer-Olkin test of sphericity and Bartlett's were used to evaluate the factored data. A number of factors with high-value > 1 was estimated. For criterion-related validity31,32 Pearson's correlation matrix is presented. We compared the results of the RCBS with those obtained in the Y-BOCS-SV and in the CBS. To quantify and discriminate the cutoff for the scale, a ROC curve33,34 was computed.
For differentiating non-clinical, compulsive buying, OCD and ICD groups, a one-way ANOVA with post-hoc test of least significant difference was performed, using p < .05 as the criterion for statistical significance.
Results
Table 1 presents the characteristics of the study sample. The mean age of participants was 32.0 years (SD = 11.8). Significant differences were found between groups with regard to age, BDI and BAI. A post-hoc analysis (Bonferroni) revealed that the group with OCD was significantly older than the NC group (p < .0001). Groups with OCD and ICD had higher BAI and BDI scores than the non-patients (p < .001 and p < .0002, respectively). There were no significant differences with regard to gender, marital status and education level. The differences observed in the BAI and the BDI scores are compatible with the literature.
Internal consistency of the scales
The scales' reliability (i.e., internal consistency) was assessed using Cronbach's alpha. The original version of the RCBS reported an alpha of .89, and the alpha of our version of the scale was .87. This result is satisfactory and demonstrates that this short scale has good reliability.
The alphas of the clinical group indicate how much the scale was able to differentiate CBD patients (.59) from individuals with OCD (.90) and ICD (.78).
To verify data adequacy to perform the factor analysis, the Kaiser-Meyer-Olkin (KMO) index and Bartlett's sphericity were used. The results were satisfactory: KMO = .87 and Bartlett = 781.94, p < .01. An analysis of the factorial structure of the Portuguese version of the RCBS was conducted with EFA, Varimax rotation, and numerical and graphical methods (screenplot) as criteria for extracting factors. Figure 1 presents the scree plot method as a criterion for extraction of factors and Varimax rotation method. Corroborating previous literature, the results clearly indicated the presence of two main factors. The first factor accounted for 60.7% of the total variance, whereas the second factor, also with a significant effect, accounted for 17.8% of the variance. Table 2 presents the levels of each item loading on the two identified factors. The factor "concern regarding purchasing" consisted of the following: RC1 ("There are unopened shopping bags in my closet"), RC2 ("Other people think I buy too much"), RC3 ("Much of my life is based on purchasing"), and RC4 ("I consider myself an impulse buyer; I don't think of the consequences"). The factor "impulse to buy" consisted of RC5 ("I buy things that I don't need") and RC6 ("I buy things that I did not plan to buy").
Criterion validity
For concurrent validity, Table 3 presents the correlation matrix of Pearson and the variables investigated, which is compatible with the available literature, showing a strong correlation between OCD and ICD with compulsive buying. Compulsive buying disorder has been associated with obsessive-compulsive disorder. As noted by Ridgway et al.,9 since compulsive buying disorder has an obsessive-compulsive component, it is possible to understand the significant correlations between the scores on the RCBS and the scores produced by the Y-BOCS-SV. Considering that the RCBS and the CBS evaluate similar constructs, significant correlations were expected between these variables. We observed a high correlation (r = -.76) between these measures, which share 58% of the variance. The indication of an inverse correlation reflects the reversed scoring system on the CBS.
As mentioned, compulsive buying disorder affects individuals' emotional state and is associated with other disorders. Thus, the correlations of the RCBS with the BDI and BAI were also expected.
Prevalence, sensitivity and specificity
Ridgway et al.9 reported that a cutoff of 24 points would indicate the presence of CBD. Our study shows that 21 of the 22 participants (95.4%) who were identified with CBD had scores equal to or greater than 24 points. In the NC group, 22 of the 202 participants (10.9%) had scores equal to or greater than 24. To verify the variation of the sensibility and specificity of the scale, a ROC curve was constructed, as proposed by Altman and Bland33 and Jaeschke et al.34 The entered values were: 22 absent and 21 present for test positive and 180 absent and 1 present for test negative. The results (Table 4) indicate that, for the cutoff of 24 points, the Brazilian version of the RCBS has a sensitivity of 95.4% and a specificity of 89.1%. Based on these results, it is possible to conclude that the prevalence of the disorder was 9.8% (95%CI: 6.4-14.7) in the analyzed sample.
Comparison between different clinical groups
The RCBS was applied to subjects from different groups according to clinical status. Average scores were the following: non-clinical: 14.5; OCD: 13.2; ICD: 14.4; and CBD: 33.4. In order to compare and detect statistically significant differences between these four groups, an ANOVA was performed. Peer-to-peer comparisons indicated that the compulsive buying group was significantly different from the other three groups (p < .01 in each of the three comparisons). There was no statistically significant difference between nonclinical and OCD groups (p = .49); nonclinical and ICD (p = .99), and OCD and ICD (p = .62).
Discussion
The Brazilian version of the RCBS is an instrument suitable for the national reality. Statistical analyses revealed that the scale has satisfactory psychometric properties. According to Pasquali,29 Cronbach's alphas above 0.80 are satisfactory, and the RCBS showed an alpha of .87, indicating high reliability.
These results are consistent with the factor structure originally presented by Ridgway et al.9 and support the conclusion that the Brazilian version of the RCBS has similar structure to the original scale and reflects the true dimensions of compulsive buying disorder.
Nunnaly and Bernstein35 support the use of Varimax rotation for the extraction of factors (i.e., main components), given that is has fast calculation rates and is suitable for validation. The Richmond validation study found two factors in the structure of the instrument. The first factor is related to "concern regarding purchasing" and the second factor is "impulse to buy". The factors found in our study are compatible with previous studies. All item loadings were at or above .50 and were comparable in magnitude to those achieved in the original study of Richmond's compulsive buying scale.
The coefficients observed in the correlation matrix were largely consistent with previous studies regarding compulsive buying disorder. Significant correlations between the scores on the scale and other measures were identified, as the urge to buy, the severity and change (after treatment) in compulsive buying, as well as the scores found in other scales, such as the Questionnaire About Buying Behavior12 and Y-BOCS-SV17 which indicates consistency among variables.
When comparing the average scores of the scales for compulsive buying, Y-BOCS-SV, CBS and RCBS with the averages of the BDI and BAI, the non-clinical group and ICD group did not differ significantly. However, when compared with the group of participants with OCD, the results showed a satisfactory correlation, indicating that the symptoms of depression and anxiety, when present, may be comorbid with OCD and compulsive buying disorder.
The comparison between the scores of subjects belonging to groups with distinctive clinical characteristics (NC, OCD, ICD and CBD) showed a statistically significant difference between the CBD group and all others in peer-to-peer comparisons. This difference was not observed when comparing other pairs of groups. Thus, the scale was able to differentiate this particular group.
An important reason to conduct this study was the lack of validated scales that assessed compulsive buying in Brazil. Given the satisfactory results obtained here, we now have a scale for the measurement and diagnosis of this disorder in Portuguese. This scale can be used in other studies examining compulsive buying, as well as in epidemiological surveys.
Despite its significant contribution to the study of compulsive buying, the present study has some limitations: although there was a good sample of participants from the general population, the clinical sample can be considered small (n = 22). Data collection was carried out partially by an electronic, non-presential, means, which may have introduced some bias in the selection of participants (not including those who are deprived of Internet access). Thus, future studies that focus on this clinical population and also allow greater access to participation are recommended. Another point to note is that the present study aimed to assess the reliability and internal consistency of the instrument, and has not investigated its temporal stability. Thus, future studies are needed to evaluate, through test-retest reliability, this particular question.
The authors intend to continue investigating these issues through new studies based on etiology, epidemiology and methods for the treatment and prevention of CBD. More consistent studies addressing this topic are required given that compulsive buying is a disorder with an increasing incidence. Results obtained in this study can offer insight and shed light on new research about ways to prevent and treat compulsive buying.
Received on September 06, 2011; accepted on October 15, 2012
References
- 1. Lejoyeux M, Haberman N, Solomon J, Adès J. Comparison of buying behavior in depressed patients presenting with or without compulsive buying. Compr Psychiatry. 1999;40,1:51-6.
- 2. Black M. A review of compulsive buying disorder. World Psychiatry. 2007;6(1):14-8.
- 3. Filomensky TZ, Tavares H, Cordas TA. Comprar compulsivo. In: Abreu CN, Tavares H, Cordas TA, et al. Manual Clínico dos Transtornos do Controle dos Impulsos. Porto Alegre: Artmed. 2008; p. 121-36.
- 4. Frost RO, Krause MS, Steketee G. Hoarding and obsessive compulsive symptoms. Behav Modif, 1996;20;16-132.
- 5. Frost RO, Steketee G. Cognitive Approaches to Obsessions and Compulsions: Theory, Assessment, and Treatment. Massachussets: Pergamon, 2002.
- 6. McElroy SL, Keck Jr. P, Philips K. Kleptomania, compulsive buying and binge-eating disorder. J Clin Psychiatry. 1995;56:14-26.
- 7. McElroy SL, Keck Jr. P, Pope Jr. H, Smith JM, Strakowski S. Compulsive buying: a report of 20 cases. J Clin Psychiatry. 1994;55-56:242-8.
- 8. Mitchell JE, Burgard M, Faber R, Crosby RD, de Zwaan M. Cognitive behavioral therapy for compulsive buying disorder. Behav Res Ther. 2006;44:1859-65.
- 9. Ridgway NM, Kukar-Kinney M, Monroe KB. An expanded conceptualization and a new measure of compulsive buying. J Consum Res. 2008;35:622-39.
- 10. American Psychiatric Association.Diagnostic and Statistical Manual of Mental Disorders, Washington, DC: American Psychiatric Association. 2000:4.
- 11. Faber RJ, Christenson GA. In the mood to buy: Differences in the mood states experienced by compulsive buyers and other consumers. Psychol Mark. 1996;13:803-20.
- 12. Lejoyeux M, Tassain V, Solomon J, Adès J. Study of compulsive buying in depressed patients. J Clin Psychiatry. 1997;58:169-73.
- 13. Edwards EA. Development of a new scale for measuring compulsive buying behavior. Fin Couns Plan. 1993;4:67-84.
- 14. Valence G, D'astous A, Fortier L. Compulsive buying: conceptand measurement. J Consum Policy. 1988;11:419-33.
- 15. DeVillis RF. Scale Development: Theory and Applications. 2 ed. Thousand Oaks: Sage; 2003.
- 16. Tian KT, Bearden WO, Hunter GL. Consumers' need for uniqueness: scale development and validation. J Consum Res. 2001;28:50-66.
- 17. Monahan P, Black DW, Gabel J. Reliability and validity of a scale to measure change in persons with compulsive buying. Psychiatry Res. 1996;64:59-6.
- 18. Faber RJ, O'Guinn TC. Classifying compulsive consumers: advances in the development of a diagnostic tool. Adv Consum Res. 1989;16;738-44.
- 19. Guzman CS, Filomensky T, Tavares H. Compulsive buying treatment with topiramate, a case report. Rev Bras Psiquiatr. 2007;29(4):383-4.
- 20.Hermano T, Sabbatini LS, Fuentes D, Black DW. Compras compulsivas: uma revisão e um relato de caso. Rev Bras Psiquiatr. 2008;30(suppl.1):S16-S23.
- 21. Amorim P. Mini International Neuropsychiatric Interview (MINI): validation of a short structured diagnostic psychiatric interview. Rev Bras Psiquiatr. 2000;22(3):106-15.
- 22. Filomensky TZ, Tavares H. Cognitive restructuring for compulsive buying. Rev Bras Psiquiatr. 2009;31(suppl.1):77-8.
- 23. Leite P. Adaptação Transcultural e Validação das Escalas "The Richmond Compulsive Buying Scale" e "Compulsive Buying Scale" [dissertation]. Rio de Janeiro (RJ). Universidade Federal do Rio de Janeiro; 2011.
- 24. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh G. An Inventory for Measuring Depression. Arch Gen Psychiatry. 1961;4:53-63.
- 25. Beck AT, Steer RA. Beck Depression Inventory - Manual. Psychological Corporation, San Antonio, 1993.
- 26. Cunha JA. Beck Depression Inventory (BDI) - Manual da Versão em Português das Escalas Beck. São Paulo: Casa do Psicólogo, 2001.
- 27. Skevington SM, Bradshaw J, Saxena S. Selecting national items for the WHOQOL: conceptual and psychometric considerations. Soc Sci Med. 1999;48:473-87.
- 28. Pasquali L. Escalas psicométricas. In: Pasquali, L. & Cols. Instrumentação Psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010. p. 116-35.
- 29. Pasquali L (Org.). Técnicas de exame psicológico - TEP. Manual. Vol. I: Fundamentos das técnicas psicológicas. São Paulo: Casa do Psicólogo; 2001.
- 30. Carmines EG, Zeller RA. Reliability and Validity Assessment. Newbury Park, CA: Sage, 1979.
- 31. Gorsuch RL. Factor Analysis. 2Ed. Psychology Press; 1983.
- 32. Streiner DL, Norman GR. Health Measurements Scales: a practival guide to their development and use. 4. ed. New York: Oxford Press; 2008.
- 33. Altman DG, Bland JM. Diagnostic testes 1: Sensitivity and Specificity. BMJ. 1994;308:1552.
- 34. Jaeschke R, Guyatt G, Sackett DL. How to Use an Article About a Diagnostic Test. Users' Guides to Medical Literature. JAMA. 1994;271(5):389-91.
- 35. Nunnaly JC, Bernstein IJ. Teoria Psicométrica.Buenos Aires: McGraw Hill; 1995.
Appendix I
Publication Dates
-
Publication in this collection
04 Apr 2013 -
Date of issue
Mar 2013
History
-
Received
06 Sept 2011 -
Accepted
15 Oct 2012