Accuracy of the Composite International Diagnostic Interview (CIDI 2.1) for diagnosis of post-traumatic stress disorder according to DSM-IV criteria

Validade do diagnóstico de transtorno de estresse pós-traumático do Composite International Diagnostic Interview (CIDI 2.1) de acordo com os critérios diagnósticos da DSM-IV

ARTICLE ARTIGO

Accuracy of the Composite International Diagnostic Interview (CIDI 2.1) for diagnosis of post-traumatic stress disorder according to DSM-IV criteria

Validade do diagnóstico de transtorno de estresse pós-traumático do Composite International Diagnostic Interview (CIDI 2.1) de acordo com os critérios diagnósticos da DSM-IV

Maria Inês Quintana; Jair de Jesus Mari; Wagner Silva Ribeiro; Miguel Roberto Jorge; Sergio Baxter Andreoli

Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brasil

^{Correspondence}

ABSTRACT

The objective was to study the accuracy of the post-traumatic stress disorder (PTSD) section of the Composite International Diagnostic Interview (CIDI 2.1) DSM-IV diagnosis, using the Structured Clinical Interview (SCID) as gold standard, and compare the ICD-10 and DSM IV classifications for PTSD. The CIDI was applied by trained lay interviewers and the SCID by a psychologist. The subjects were selected from a community and an outpatient program. A total of 67 subjects completed both assessments. Kappa coefficients for the ICD-10 and the DSM IV compared to the SCID diagnosis were 0.67 and 0.46 respectively. Validity for the DSM IV diagnosis was: sensitivity (51.5%), specificity (94.1%), positive predictive value (9.5%), negative predictive value (66.7%), misclassification rate (26.9%). The CIDI 2.1 demonstrated low validity coefficients for the diagnosis of PTSD using DSM IV criteria when compared to the SCID. The main source of discordance in this study was found to be the high probability of false-negative cases with regards to distress and impairment as well as to avoidance symptoms.

Post-Traumatic Stress Disorders; Mental Disorders; Diagnosis

RESUMO

O objetivo deste artigo foi estudar a validade concorrente da seção de transtorno de estresse pós-traumático do CIDI 2.1 critérios DSM IV, utilizando o Structured Clinical Interview (SCID) como padrão-ouro, e comparar o diagnóstico de TEPT entre CID-10 e DSM IV. O CIDI foi aplicado por entrevistadores leigos treinados e o SCID por uma psicóloga. A amostra foi composta por sujeitos da comunidade e de um ambulatório de especialidade psiquiátrica. Sessenta e sete sujeitos completaram ambos os questionários. O coeficiente kappa foi de 0.46 ao comparar DSM IV com a SCID. A validade diagnóstica usando critérios do DSM IV foi de: sensibilidade = 51.5%, especificidade = 94.1%, valor preditivo positivo = 89.5%, valor preditivo negativo = 66.7%, taxa de classificação incorreta = 26.9%. O CIDI 2.1 apresentou valores baixos para os coeficientes de validação de TEPT usando os critérios do DSM IV ao comparar com o SCID. A principal causa de discordância foi o grande número de casos falsos negativos devido aos sintomas de significância clínica e sintomas de evitação.

Transtornos de Estresse Pós-Traumáticos; Transtornos Mentais; Diagnóstico

Introduction

The Composite International Diagnostic Interview (CIDI) is a fully standardized, structured interview that provides a psychiatric diagnosis through computerized algorithms ^1,2 in accordance with the International Classification of Diseases, 10^th edition (ICD-10) ³ and the American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders, 4^th edition (DSM-IV) ⁴. It comprises 11 diagnostic sections, which may be administered independently, covering substance use (tobacco, alcohol, and drug use), phobias and anxiety disorders, depressive disorders, mania, anorexia nervosa, obsessive-compulsive disorder, schizophrenia and other psychoses. A Portuguese version of the CIDI has been developed in Brazil ^5,6.

A post-traumatic stress disorder (PTSD) section was included in the latest version of the CIDI ^7,8, and has since been widely used in epidemiological studies ^9,10,11,12. A number of studies have been conducted to assess the ability of the CIDI to diagnose PTSD accurately, using either ICD-10 or DSM-IV criteria ^13,14,15 however results are conflicting due to the divergent criteria of these classification systems (Table 1). In order to meet DSM-IV criteria for a diagnosis of PTSD, patients must fulfill one additional criterion (F: requires distress and impairment), two additional symptoms on criterion C (regarding avoidance) and a symptom duration of at least one month. Memory loss is assigned a different weight in each classification: in the DSM-IV criteria, it is part of a list of symptoms, up to three of which must be reported for diagnosis; in the ICD-10 classification, it is a key symptom that confirms diagnosis if present.

The PTSD section of the Brazilian version of the CIDI has not yet been validated. Validation and assessment of its performance in accordance with the adopted classification criteria is essential for use of the CIDI 2.1 in epidemiological studies conducted within Brazil. Therefore, the objectives of this investigation were to study the concurrent validity of the PTSD section of the Brazilian CIDI and to investigate the possible sources of discordance between the two diagnostic classification systems (ICD-10 and DSM-IV) (Table 1).

Method

This study assessed the concurrent validity of the PTSD section of the CIDI 2.1, using the Structured Clinical Interview for DSM disorders (SCID) as the gold standard.

The total sample comprised 67 subjects: 28 referred from a specialized outpatient unit [the Program for Victims of Violence - PROVE of the São Paulo Federal University (Universidade Fe-deral de São Paulo - UNIFESP)], due to a psychiatric diagnosis of PTSD on DSM-IV criteria; and 39 volunteers from the local community, including residents, trainees, psychology students, occupational therapists, social workers, nurses and their family members, who had experienced at least one traumatic event during their lives. This strategy was meant to enroll a convenience sample that presented a broad spectrum of symptoms, so as to enable measurement of the items listed in the CIDI.

The PTSD section of the CIDI is composed of a list of traumatic events (11 events: direct combat experience in a war; life-threatening accident; natural disaster; witnessed someone being badly injured or killed; rape; sexual molestation; serious physical attack or assault; threatened with a weapon, held captive, or kidnapped; torture or terrorism; any other extremely stressful or upsetting event; great shock because one of the events on the list happened to someone close) which was adapted in its Brazilian Portuguese version to include 23 new events related to common episodes of violence in Brazil (organized crime, childhood violence, urban violence, and death of or presence of severe chronic diseases in close relatives). Subjects who confirmed exposure to at least one such traumatic event underwent a specific diagnostic investigation of lifetime symptoms of PTSD. The CIDI 2.1 was administered by a psychologist who had previously received standard training in use of the instrument, in accordance with WHO guidelines ².

The SCID ^16,17 is a semi-structured interview designed to be administered by a clinician or trained mental health professional. The SCID uses the DSM-IV diagnostic criteria and has been used as a gold standard for diagnosis ¹⁸.

The PTSD section of the SCID starts with an open-ended question about the occurrence of a traumatic event, citing a few examples. The interviewee considers whether the event has ever occurred in his or her life and whether it was traumatic. The SCID questions follow the same format as the DSM-IV questions used in the CIDI 2.1 interview, namely, yes/no answers. Interviewers also compiled data from the medical records of the individuals under treatment. The SCID was administered by an experienced and duly trained PROVE staff psychologist.

Subjects were initially interviewed using the SCID and, on the same day or within 24 hours, were asked to fill out the Lifetime Post-Traumatic Stress Disorder Section (K) of the CIDI 2.1. The interviewer was blind to the SCID diagnosis. All subjects provided written informed consent. This study was approved by the Ethics Committee of UNIFESP.

The concurrent validity of the Lifetime Post-Traumatic Stress Disorder Section (K) of the CIDI 2.1 was compared to the SCID-based diagnosis by estimating validity coefficients and the kappa statistic. The validity coefficients used were sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and misclassification ¹⁹. 95% confidence intervals (95%CI) were computed using an exact binomial distribution ²⁰. Discordance between the two diagnostic classification systems (ICD-10 and DSM-IV) was analyzed by the kappa coefficient. The kappa statistic is defined as a chance-corrected measurement of inter-rater agreement ²¹. Kappa values were interpreted according to the scale recommended by Fleiss ²², whereby: Κ < 0.40 indicates poor agreement; 0.40-0.64, satisfactory agreement; 0.61-0.75, good agreement; and > 0.75, excellent agreement.

Results

Of the 67 participants, 28 (42%) were recruited from the outpatient unit and 39 (58%) were drawn from the community. 64% of the sample was composed of women. The mean age was 39 years (SD: 12.18; range: 17-67); 42% of the participants were single, 40% were married, 12% were separated or divorced and 6% were widowed. 63% percent of the participants were actively employed. The mean educational achievement was 12.8 years of formal schooling (SD: 5.64; range: 0-30).

Overall, 390 traumatic events were reported by the 67 respondents. The most common events were, in decreasing order of frequency: death of a loved one (10.51%); seeing dead bodies or witnessing atrocities or massacres (9.23%); being a victim of gang warfare (8.2%); physical assault or robbery without a weapon (6.92%); witnessing a shooting (6.15%); and violence during childhood (5.9%). Other events had a frequency below 5% each.

Cross-tables comparing CIDI scores (using DSM-IV) criteria against SCID interviews are displayed in Table 2. The SCID identified 33 cases of PTSD, whereas the DSM-IV criteria diagnosed only 19. The CIDI 2.1 validity coefficients for diagnosis according to DSM-IV criteria were as follows: sensitivity, 51.5% (95%CI: 33.5-69.2); specificity, 94.1% (95%CI: 80.3-99.2); PPV, 89.5% (95%CI: 80.3-99.3); NPV, 66.7% (95%CI: 51.6-79.6); and misclassification rate, 26.9%. The kappa coefficient was 0.459 (standard error - SE = 0.099; 95%CI: 0.26-0.65) for the DSM-IV criteria.

When applying DSM-IV criteria, the CIDI exhibited positive agreement in 17 cases and negative agreement in 32 cases, leading to the misclassification of 18 cases (16 false negatives and 2 false positives). In most false negatives (13 cases), there was no agreement on criterion F which

requires that the disturbance cause significant psychological distress or impairment in functioning. The requirement of a duration of symptoms of at least one month led to 5 false-negatives, according to DSM-IV criteria, and a further 6 cases were due to a lack of the minimum of three symptoms related to numbing and detachment responses. The two false positives were due to negative responses to criterion C (avoidance) of the SCID interview. In both cases, the interviewer reported that the traumatic event was assault.

The kappa coefficient of agreement between the two diagnostic classifications systems, ICD-10 and DSM-IV, was 0.50 (SE = 0.09; 95%CI: 0.310-0.681). The systems diverged on 17 cases. In the single ICD-negative but DSM-positive case, the discordant criterion was ICD-10 criterion C, which requires "one symptom of actual or preferred avoidance". In 16 cases, patients met ICD-10 criteria for diagnosis of PTSD but did not meet DSM-IV criteria; disagreement was most often (in 12 cases) due to criterion F, which assesses distress and impairment-symptoms not provided for in the ICD-10 criteria. The remaining criteria also played a role in disagreement, but to a lesser extent: criterion C was involved in 7 cases, criterion E was implicated in 3 cases and criterion A, in 2.

Discussion

The ability of the CIDI 2.1 to accurately identify PTSD cases, using the SCID interview as the gold standard, was fairly low when the DSM-IV criteria were employed: sensitivity was 51.5% and the kappa coefficient was 0.46. It is worth noting that false negatives were mostly related to the requirement of distress and impairment, as well as the need for exhibiting three or more criterion C symptoms, for fulfilling the DSM-IV criteria for PTSD.

In the present study, if DSM-IV criterion F (which requires distress and impairment) were excluded, the sensitivity would increase to 78.8% and the kappa coefficient would reach 0.64, a level of discordance similar to that reported by Peter et al. ¹⁵. Breslau & Alvorado ²³ claim that the prevalence of PTSD in the Detroit Area Survey ¹³ and in a sample of Mid-Atlantic urban youths ²⁴ was reduced from 10.8% to 7.8% and from 14% to 8.8%, respectively, because of the inclusion of criterion F.

A possible source of discordance was the inclusion of traumatic events which, despite involving situations of high psychological impact, do not constitute an immediate threat to life or physical integrity. In this study, 9 out of 16 false-negative cases, in accordance with DSM-IV criteria, were associated with one such event, examples of which include "seeing dead bodies", "serious illness or injury experienced by a family member or a close friend" and "sudden death of a loved one".

Breslau et al. ¹³ assessed the impact of new traumatic events recently included in the CIDI, such as "serious injury or illness experienced by a family member" or "sudden death". They found that individuals presented a lower mean duration of PTSD symptoms (12.1 months) than when symptoms originated from other events (48.1 months). Breslau & Kessler ²⁵ claim that the inclusion of these events may generate a diagnosis of PTSD though these subjects would exhibit milder disturbances than those who actually experienced life-threatening situations. These findings point out the problem of including these events without adapting the CIDI questions to provide for these specific traumas.

It bears noting that the misclassification of cases by the CIDI occurred when avoidance symptoms and impairment were identified in the presence of non-life-threatening stressors. The CIDI questions that elicit these symptoms were devised in its original version, when the instrument included only catastrophic and/or life-threatening events, and have not been adapted to account for the inclusion of new traumatic events. These events may be associated with PTSD symptoms, such as persistent remembering, hyper-arousal, avoidance, and emotional numbing, though with less intensity ²⁶. In six SCID-negative but CIDI-positive cases, patients denied experiencing avoidance symptoms. One possible explanation concerns the sequence in which symptoms are investigated in the two instruments (CIDI and SCID). The SCID assesses avoidance symptoms before intrusive thoughts. In the absence of avoidance symptoms, the interview is stopped and the presence of intrusive thoughts is not investigated at all, leading to misclassification.

For said cases, it would be advisable to reorganize the CIDI questions directed at avoidance/numbing and impairment. For instance, if the traumatic event concerns sudden death of a close friend, relative or other loved one, the CIDI question is: "avoid places or people or activities that might have reminded you of sudden death of a loved one". In this case, "avoiding places or people or activities" may not be the most adequate way of investigating avoidance symptoms. The same applies to questions directed at impairment, which need to be clearer; the criteria could also be more flexible. This flexibility has been studied by some authors with the concept of partial, subsyndromal or subthreshold PTSD (PPTSD) ^27,28,29,30. In general, a diagnosis of PPTSD requires the presence of at least one symptom for each criterion of PTSD; criteria E (time) and F (impairment and distress) are maintained ³⁰. Moreover, Stein et al. ³⁰ have reported that partial PTSD subjects presented social and occupational impairments as severe as those with full-blown PTSD. Mylle & Maes ²⁹ propose that "subsyndromal PTSD" is a syndrome where at least one symptom of each criterion is required, whereas "partial PTSD" requires the presence of criterion F and does not necessarily require any other criteria. Regarding the type of traumatic event, Breslau et al. ³¹ found differences between full-blown PTSD and PPTSD, with the former presenting more often with "high magnitude events", similar to the findings of this study.

Agreement between the ICD-10 and DSM-IV classification was merely satisfactory (kappa, 0.50), as previously reported in the literature ¹⁵.

The main source of discordance was related to the requirement of distress and impairment (DSM-IV criterion F). If this criterion were excluded, agreement would reach a kappa of 0.64. This discrepancy is in line with that reported by Peter et al. ¹⁵, where this criterion accounted for 48% of the discordance between ICD-10 and DSM-IV diagnoses.

Several limitations of this study warrant mention: (a) we were unable to verify ICD-10 diagnoses of PTSD, as the SCID follows the DSM-IV criteria; (b) co-morbidities where not fully evaluated, which may have produced some noise when comparing impairment caused by full-blown versus partial PTSD; (c) the evaluation of impairment relied only on CIDI 2.1 questions, that is, there was no measurement of the economic and social impact of diseases, as proposed by Kessler & Frank ³²; (d) we were unable to evaluate current diagnoses of PTSD due to the small number of cases. The conclusions presented are preliminary and further studies should be carried out.

In summary, this study showed that the CIDI 2.1 exhibited low accuracy for identification of PTSD cases in accordance with the DSM-IV criteria. A number of suggestions were raised for reorganization of the eliciting symptoms in the CIDI instrument, particularly those regarding DSM-IV criteria C and F. Special attention should be paid in cases where the traumatic event was not life-threatening. We suggest that the order of inquiry on symptoms be adapted in cases where traumatic events may have caused a significant psychological impact though were not life-threatening. Use of this version of the CIDI in epidemiological studies may require adjustment of the diagnostic algorithm and redesigning of the ratings assigned to DSM-IV criteria C and F in the CIDI.

Contributors

M. I. Quintana participated in the training process to conduct the CIDI 2.1, field work supervision, organization of the data bank, statistical analysis, project conception, data analysis and interpretation, drafting and critical revision of the intellectual content of the article and approval of the final version for publication. J. J. Mari and M. R. Jorge collaborated with project conception, data analysis and interpretation and in the drafting and critical revision of the intellectual content of the article and approval of the final version for publication. W. S. Ribeiro contributed towards field work supervision, project conception and approval of the final version of the article for publication. S. B. Andreoli participated in the general coordination of the study, project conception, data analysis and interpretation and the critical revision of the intellectual content of the article and approval of the final version for publication.

Acknowledgments

This study was supported by FAPESP through grant 2004/15039-0.

Submitted on 23/Nov/2010

Final version resubmitted on 09/Nov/2011

Approved on 06/Feb/2012

References

• 1. World Health Organization. CIDI bibliography. Geneva: World Health Organization; 1994.
• 2. World Health Organization. The Composite International Diagnostic Interview (CIDI). Geneva: World Health Organization; 1997.
• ³ World Health Organization. The ICD-10 classification of mental and behavioural disorders: diagnostic criteria for research. Geneva: World Health Organization; 1993.
• 4. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4^th Ed. Washington DC: American Psychiatric Association; 1994.
• 5. Quintana MI, Andreoli SB, Jorge MR, Gastal FL, Miranda CT. The reliability of the Brazilian version of the Composite International Diagnostic Interview (CIDI 2.1). Braz J Med Biol Res 2004; 37:1739-45.
• 6. Quintana MI, Gastal FL, Jorge MR, Miranda CT, Andreoli SB. Validity and limitations of the Brazilian version of the Composite International Diagnostic Interview (CIDI2.1). Rev Bras Psiquiatr 2007; 29:18-22.
• 7. Andrews G, Peters L. The psychometric properties of the Composite International Diagnostic Interview. Soc Psychiatry Psychiatr Epidemiol 1998; 33:80-8.
• 8. Wittchen HU. Reliability and validity studies of the WHO-Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res 1994; 28:57-84.
• 9. de Jong JT, Komproe IH, Van Ommeren M, El Masri M, Araya M, Khaled N, et al. Lifetime events and posttraumatic stress disorder in 4 postconflict settings. JAMA 2001; 286:555-62.
• 10. Icaza MH-M, Borges-Guimarães G, Ramos-Lira L, Zambrano J, Fleiz-Bautista C. Prevalencia de sucesos violentos y de transtono por estrés postraumático en la población mexicana. Salud Pública Méx 2005; 47:8-22.
• 11. Norris FH, Murphy AD, Baker CK, Perilla JL, Rodriguez FG, Rodriguez JJ. Epidemiology of trauma and posttraumatic stress disorder in Mexico. J Abnorm Psychol 2003; 112:646-56.
• 12. Zlotnick C, Johnson J, Kohn R, Vicente B, Saldivia S. Epidemiology of trauma, post-traumatic stress disorder (PTSD) and co-morbid disorders in Chile. Psychol Med 2006; 36:1523-33.
• 13. Breslau N, Kessler RC, Chilcoat HD, Schultz LR, Davis GC, Andreski P. Trauma and posttraumatic stress disorder in the community: the 1996 Detroit Area Survey of Trauma. Arch Gen Psychiatry 1998; 55:626-32.
• 14. Komiti AA, Jackson HJ, Judd FK, Cockram AM, Kyrios M, Yeatman R, et al. A comparison of the Composite International Diagnostic Interview (CIDI-Auto) with clinical assessment in diagnosing mood and anxiety disorders. Aust N Z J Psychiatry 2001; 35:224-30.
• 15. Peters L, Slade T, Andrews G. A comparison of ICD10 and DSM-IV criteria for posttraumatic stress disorder. J Trauma Stress 1999; 12:335-43.
• 16. Spitzer RL. Psychiatric diagnosis: are clinicians still necessary? Compr Psychiatry 1983; 24:399-411.
• 17. Spitzer RL, Williams JB, Gibbon M, First MB. The Structured Clinical Interview for DSM-III-R (SCID). I: History, rationale, and description. Arch Gen Psychiatry 1992; 49:624-9.
• 18. Steiner JL, Tebes JK, Sledge WH, Walker ML. A comparison of the structured clinical interview for DSM-III-R and clinical diagnoses. J Nerv Ment Dis 1995; 183:365-9.
• 19. Fletcher RH, Fletcher SW, Wagner EH. Epidemiologia clínica. 3Ş Ed. Porto Alegre: Editora Artmed; 1991.
• 20. Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934; 26:404-13.
• 21. Spitzer RL, Cohen J, Fleiss JL, Endicott J. Quantification of agreement in psychiatric diagnosis: a new approach. Arch Gen Psychiatry 1967; 17:83-7.
• 22. Fleiss JL. Statistical methods for rates and proportions. 2^nd Ed. New York: John Wiley & Sons; 1981.
• 23. Breslau N, Alvarado G. The clinical significance criterion in DSM-IV post-traumatic stress disorder. Psychol Med 2007; 37:1437-44.
• 24. Breslau N, Wilcox HC, Storr CL, Lucia VC, Anthony JC. Trauma exposure and posttraumatic stress disorder: a study of youths in urban America. J Urban Health 2004; 81:530-44.
• 25. Breslau N, Kessler RC. The stressor criterion in DSM-IV posttraumatic stress disorder: an empirical investigation. Biol Psychiatry 2001; 50:699-704.
• 26. Figueira I, Mendlowicz M. Diagnóstico do transtorno de estresse pós-traumático. Rev Bras Psiquiatr 2003; 25 Suppl 1:12-6.
• 27. Carlier IV, Gersons BP. Partial posttraumatic stress disorder (PTSD): the issue of psychological scars and the occurrence of PTSD symptoms. J Nerv Ment Dis 1995; 183:107-9.
• 28. Kapczinski F, Margis R. Transtorno de estresse pós-traumático: critérios diagnósticos. Rev Bras Psiquiatr 2003; 25 Suppl 1:3-7.
• 29. Mylle J, Maes M. Partial posttraumatic stress disorder revisited. J Affect Disord 2004; 78:37-48.
• 30. Stein MB, Walker JR, Hazen AL, Forde DR. Full and partial posttraumatic stress disorder: findings from a community survey. Am J Psychiatry 1997; 154:1114-9.
• 31. Breslau N, Lucia VC, Davis GC. Partial PTSD versus full PTSD: an empirical examination of associated impairment. Psychol Med 2004; 34:1205-14.
• 32. Kessler RC, Frank RG. The impact of psychiatric disorders on work loss days. Psychol Med 1997; 27:861-73.

Publication Dates

• Publication in this collection
  18 June 2012
• Date of issue
  July 2012

History

• Received
  23 Nov 2010
• Accepted
  06 Feb 2012
• Reviewed
  09 Nov 2011