Clinical judgment to estimate pretest probability in the diagnosis of Cushing's syndrome under a Bayesian perspective

Contribuição do julgamento clínico na estimativa da probabilidade pré-teste do diagnóstico da síndrome de Cushing sob a perspectiva Bayesiana

ORIGINAL ARTICLE

Clinical judgment to estimate pretest probability in the diagnosis of Cushing's syndrome under a Bayesian perspective

Contribuição do julgamento clínico na estimativa da probabilidade pré-teste do diagnóstico da síndrome de Cushing sob a perspectiva Bayesiana

Daniel E. Cipoli^I; Edson Z. Martinez^II; Margaret de Castro^I; Ayrton C. Moreira^I

^IDivision of Endocrinology, Department of Medicine, School of Medicine of Ribeirao Preto, University of Sao Paulo (FMRP-USP), Ribeirao Preto, SP, Brazil

^IIDivision of Statistics, Department of Social Medicine, FMRP-USP, Ribeirao Preto, SP, Brazil

^{Correspondence}

ABSTRACT

OBJECTIVE: To estimate the pretest probability of Cushing's syndrome (CS) diagnosis by a Bayesian approach using intuitive clinical judgment.

MATERIALS AND METHODS: Physicians were requested, in seven endocrinology meetings, to answer three questions: "Based on your personal expertise, after obtaining clinical history and physical examination, without using laboratorial tests, what is your probability of diagnosing Cushing's Syndrome?"; "For how long have you been practicing Endocrinology?"; and "Where do you work?". A Bayesian beta regression, using the WinBugs software was employed.

RESULTS: We obtained 294 questionnaires. The mean pretest probability of CS diagnosis was 51.6% (95%CI: 48.7-54.3). The probability was directly related to experience in endocrinology, but not with the place of work.

CONCLUSION: Pretest probability of CS diagnosis was estimated using a Bayesian methodology. Although pretest likelihood can be context-dependent, experience based on years of practice may help the practitioner to diagnosis CS. Arq Bras Endocrinol Metab. 2012;56(9):633-7

Keywords: Cushing's syndrome; Bayes theorem; pretest probability

RESUMO

OBJETIVO: Estimar a probabilidade pré-teste do diagnóstico de síndrome de Cushing (SC) por meio de julgamento clínico utilizando abordagem Bayesiana.

MATERIAIS E MÉTODOS: Médicos res­ponderam a três perguntas, em sete congressos de endocrinologia. Após obtenção da história clínica/exame físico, sem exames laboratoriais, apenas com base em sua experiência pessoal, qual a probabilidade de diagnosticar SC?; Há quanto tempo você pratica endocrinologia?; Onde você trabalha? Uma regressão beta Bayesiana, utilizando o software WinBugs, foi empregada.

RESULTADOS: Foram obtidos 294 questionários. A estimativa Bayesiana da probabilidade média de diagnosticar SC foi 51,6% (IC 95%: 48,7-54,3). Houve relação direta entre probabilidade de diagnosticar SC e experiência da prática endócrina, porém não com o local de trabalho.

CONCLUSÃO: A probabilidade pré-teste do diagnóstico de SC foi estimada utilizando uma metodologia Bayesiana. Embora a probabilidade pré-teste possa ser dependente do contexto, a experiência de anos de prática pode auxiliar no diagnóstico intuitivo da CS. Arq Bras Endocrinol Metab. 2012;56(9):633-7

Descritores: Síndrome de Cushing; teorema de Bayes; probabilidade pré-teste

INTRODUCTION

The diagnosis of Cushing's syndrome (CS) still presents challenges. Once the clinical practitioner or endocrinologist suspects CS diagnosis, endogenous hypercortisolism must be confirmed using biochemical tests (1-3). These tests are based on the demonstration of excessive cortisol secretion, loss of the circadian rhythm, and abnormal feedback regulation of the hypothalamic-pituitary-adrenal (HPA) axis. Tests currently used to screen for CS are measurement of free cortisol in 24-hour urine (UFC), assessment of cortisol circadian rhythm using late-night serum and salivary cortisol le­vels (LNSF), and cortisol suppressibility by low doses of dexamethasone (1 mg-DST) (4). Recently, Elamin and cols. conducted a systematic review and meta-analysis to summarize the accuracy of these tests in CS diagnosis (5). UFC, LNSF, and 1 mg-DST showed similarly high accuracy when analyzed by classical statistical methods.

The Bayesian approach provides a kind of probabilistic reasoning where the probability of occurrence of an event is obtained conditionally on the knowledge of the occurrence of an earlier event (6,7). Thus, the Bayesian approach basically consists of three steps: (a) determining a prior estimate of the parameter of interest based on former knowledge and previous experien­ces; (b) finding an appropriate likelihood function or a mathematical expression that describes the distribution of the observed data; and (c) combining the prior distribution with the likelihood function using the Bayes' theorem, providing the posterior distribution of the parameter. The likelihood ratio (LR) has been used as part of a Fagan nomogram that offers a graphic solution to the Bayes' rules (8). This nomogram integrates the pre- with posttest probabilities, enabling clinicians to estimate the posttest probability of a disease.

The difficulties in estimating pretest probabilities of CS are derived from the rarity of endogenous CS, and from the overlap in clinical findings between CS and other common disorders in the population, classified as pseudo-Cushing. The availability of clinical data and the subjective clinical judgment information, which mean the pretest probability, may enable us to estimate the posttest probability of CS diagnosis (9). There are few descriptions of pretest probability in the diagnosis of CS (10-12). According to the Bayesian approach, which considers that previous knowledge regarding any event may constitute the prior probability of occurrence, in the present study, we used clinical judgment and endocrine expertise of physicians in an attempt to estimate the pretest probability of CS diagnosis.

MATERIALS AND METHODS

Physicians were requested to answer a questionnaire in a voluntary and anonymous basis, during seven different endocrinology meetings in the following countries: Brazil (Post-Endo-2008, Salvador, BA, in 2008; V Endo-Amazon, São Luis, MA, in 2008; VIII Meeting on Endocrinology, Franca, SP, in 2009; Brazilian Society of Endocrinology and Metabolism Meeting, Belo Horizonte, MG, in 2009); Colombia (10^th Colombian Congress of Endocrinology, Bucaramanga, Colombia, in 2009); Italy (1^st ENEA Workshop: Novel Insights in the management of Cushing's syndrome, Napoli, Italy, in 2009), and USA (Symposium: The challenges of Cushing's disease diagnosis and treatment: The Latin America perspective, San Diego, CA, in 2010).

The questionnaire with three questions was availa­ble in Portuguese, Spanish or English according to the official language of the meetings, as shown in English in figure 1. This study was approved by the Ethics Committee for Research of the University Hospital of School of Medicine Ribeirao Preto.

A Bayesian beta regression (13) was used to investigate the relationship between the probability of diagnosing CS and a set of independent variables, such as experience in endocrinology, evaluated by the number of years of practice, professional activity, evaluated by the place of work in the Healthcare system, and the city of the meeting. Beta regression is a Bayesian hie­rarchical model applied to continuous data with sample space restricted to the interval (0, 1). Data were mode­led using beta distributions with mean probability of CS diagnosis that depended on covariates by means of a logit link function. The analysis was carried out in the software WinBugs (version 1.4.3, MRC, Cambridge, UK) based on Markov chain Monte Carlo (MCMC) estimation. Summaries for each parameter of interest are presented as the mean of the respective simulated chain and its 95% credible intervals (95% CI).

RESULTS

A total of 294 complete questionnaires were obtained. Bayesian estimates of the mean probability of diagnosing CS and its 95% credible intervals (95% CI) in each meeting are shown in table 1. The overall mean was estimated as 0.5161 (95% CI: 0.4878-0.5436; Figure 2).

The curves presented in figure 3 describe the relationship between the years of clinical experience and the probability of diagnosing CS, according to each endocrinology meeting. These curves were obtained by a beta regression model with interaction terms, considering the changing values of CS diagnosis probabilities as a function of the time of experience in endocrinology and represented by a logit function. The curves obtained by this approach have, theoretically, a sigmoidal shape. However, in figure 3 we show only part of these curves corresponding to the interval from zero to forty years of endocrinology practice. Thus, in this interval, these curves are linear and represent a direct relationship between the probabilities of CS diagnosis with the experience in endocrinology, evaluated by the number of years of physician practice, independent of the meeting. It may also be observed that a shorter time of experience in endocrinology led to greater differences between the curves obtained during the different meetings, but these differences decreased with longer experience. In addition, there were no differences among physicians working in primary, secondary or tertiary healthcare, or in private or public hospitals.

DISCUSSION

In the present study, we applied a Bayesian approach to estimate the probability of CS diagnosis derived from self-reported clinical expertise. Considering all centers, the overall results show that clinicians are able to recognize CS in half of the patients on clinical grounds before the confirmation with biochemical tests. A linear increase in the probability of detecting CS was observed with the years of practice of the endocrinology physician.

The three first-line tests commonly used for CS screening are based on different physiopathological characteristics of the HPA axis (3-5). They are complementary because none of the tests has proven to be fully capable of distinguishing all cases of CS from pseudo-Cushing states. Elamin and cols. (5) summarized the sensitivities and specificities for these tests using the likelihood ratio (LR) of these tests in a Fagan nomogram. Although the LRs offer some advantages, they derive from sensitivity and specificity, which are operating characteristics of the test. The most useful advantage of LRs is their easy application to estimate the posttest probability based on the pretest probability of CS. In addition, the rates of false-negative and false-positive test results depend not only on sensitivity and specificity of the test, but also on the prior probability that the subject has CS. Without any additional information, the pretest probability is equal to the prevalence of the disease (14,15), and this may be true for a disease with high prevalence. However, it is not applicable in the case of rare endogenous CS, which besides the overlapping of some clinical findings with more prevalent disorders, has little accurate epidemiological information. Depending on the studied population, CS estimated incidence is about 0.7-2.4 per million inhabitants per year (16,17). Therefore, in the case of rare diseases, in which pretest probability is uncertain, the Bayesian approach may be useful.

In the last five decades, three previous reports (10-12) calculated the posttest probability of CS using Bayes' theorem; two of them assumed the prevalence of CS as 5 to 7 per million to elicit pretest odds of the disease (10,11). Nugent and cols. (12), using the incidence of clinical signs of CS and non-steroid laboratory data, concluded that a trustworthy diagnosis could be carried out in half of the patients. Coincidently, in the present study, applying a Bayesian approach to clinical judgment, the overall mean of pretest probability of CS diagnosis was 51.6% (48.8-54.4%). However, different from previous reports, our data are more intuitive and based on self-reported clinical experience. Of note, in the Bayesian approach, the veracity of the results is affected by the goodness of the prior distribution. This, in turn, is dependent on the previous experience of one or different researchers, who can easily reach quite distinct conclusions, according to their accumulated knowledge and practices. Indeed, the choice of pretest probabilities in clinical settings may be inaccurate and vary widely (18-20). However, this uncertainty in pretest probability represents no obstacle for clinicians, once they use their intuition based on their experience, as they are natural Bayesians (6,14).

Our data showed a positive relationship between experience in clinical endocrinology and the probabili­ty of diagnosing CS, independent of the meeting. In addition, we observed that a short time of experience in endocrinology led to greater differences between the curves obtained during the different meetings, but these differences decreased with longer experience. In our opinion, these observations provide evidence that it is possible to incorporate prior clinical judgment, a consequence of previous experience in endocrinology, to provide pretest inferences on CS diagnosis. Therefore, Bayesian reasoning may numerically express this intuitive and probabilistic mechanism of thought. In addition, prior clinical suspicion can be incorporated on pretest assessment to minimize the uncertainty of pretest estimation of CS diagnosis, thus facilitating the interpretation of laboratory tests. In addition, our results may not necessarily apply to general clinical practice, once this is a selective study in endocrinology meetings, some of them attended by seasoned endocrinologists. Finally, pretest probability varies, when focusing on an individual patient, concerning the severity of clinical features of the same disease and the intuitive experience of the same physician. Thus, the physician may have a range of pretest probabilities, instead of a single one.

In conclusion, our study, using a formal Bayesian methodology, presented the numeric expression of intuitive clinical estimation of pretest probability of CS diagnosis, whose mean is estimated in 51%. In addition, there is a direct relationship between the probabilities of the CS diagnosis and the experience in endocrinology, evaluated by years of practice. Therefore, pretest likelihoods may be context-dependent.

Acknowledgements: this study was supported by Sao Paulo Research Foundation (Fapesp – 2007/58365-3) and Brazilian National Research Council (CNPq – 314279/2009-1 and 313241/2009-0).

Disclosure: no potential conflict of interest relevant to this article was reported.

Received on May/14/2012

Accepted on Sept/29/2012

References

• 1. Alexandraki KI, Grossman AB. Novel insights in the diagnosis of Cushing's syndrome. Neuroendocrinology. 2010;92:35-43.
• 2. Bertagna X, Guignat L, Groussin L, Bertherat J. Cushing's disease. Best Pract Res Clin Endocrinol Metab. 2009;23:607-23.
• 3. Castro M, Moreira AC. Screening and diagnosis of Cushing's syndrome. Arq Bras Endocrinol Metabol. 2007;51:1191-8.
• 4. Nieman LK, Biller BM, Findling JW, Newell-Price J, Savage MO, Stewart PM, et al. The diagnosis of Cushing's syndrome: an Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab. 2008;93:1526-40.
• 5. Elamin MB, Murad MH, Mullan R, Erickson D, Harris K, Nadeem S, et al. Accuracy of diagnostic tests for Cushing's syndrome: a systematic review and metaanalyses. J Clin Endocrinol Metab. 2008;93:1553-62.
• 6. Gill CJ, Sabin L, Schmid CH. Why clinicians are natural bayesians. BMJ. 2005;330:1080-3.
• 7. Lipsky AM, Lewis RJ. Placing the Bayesian network approach to patient diagnosis in perspective. Ann Emerg Med. 2005;45:291-4.
• 8. Fagan TJ. Letter: Nomogram for Bayes theorem. N Engl J Med. 1975;293:257.
• 9. Aron DC. Cushing's syndrome: why is diagnosis so difficult? Rev Endocr Metab Disord. 2010;11:105-16.
• 10. Danese RD, Aron DC. Principles of clinical epidemiology and their application to diagnosis of Cushing's syndrome: Rev Bayes Meets Dr. Cushing. Endocrinologist. 1994;5:339-46.
• 11. Kreisberg R. Clinical problem-solving. Half a loaf. N Engl J Med. 1994;330:1295-9.
• 12. Nugent CA, Warner HR, Dunn JT, Tyler FH. Probability theory in the diagnosis of Cushing's syndrome. J Clin Endocrinol Metab. 1964;24:621-7.
• 13. Branscum AJ, Johnson WO, Thurmond MC. Bayesian beta regression: application to household expenditure data and genetic distance between foot-and-mouth disease viruses. Aust N Z J Stat. 2007;49:287-301.
• 14. Bianchi MT, Alexander BM, Cash SS. Incorporating uncertainty into medical decision making: an approach to unexpected test results. Med Decis Making. 2009;29:116-24.
• 15. Motulsky H. Intuitive biostatistics. New York: Oxford University Press; 1995.
• 16. Etxabe J, Vazquez JA. Morbidity and mortality in Cushing's disease: an epidemiological approach. Clin Endocrinol (Oxf). 1994;40:479-84.
• 17. Lindholm J, Juul S, Jorgensen JO, Astrup J, Bjerre P, Feldt-Rasmussen U, et al. Incidence and late prognosis of Cushing's syndrome: a population-based study. J Clin Endocrinol Metab. 2001;86:117-23.
• 18. Redelmeier DA, Ferris LE, Tu JV, Hux JE, Schull MJ. Problems for clinical judgement: introducing cognitive psychology as one more basic science. CMAJ. 2001;164:358-60.
• 19. Myers KA, Farquhar DR. The rational clinical examination. Does this patient have clubbing? JAMA. 2001;286:341-7.
• 20. Kulkarni UP. The Bayesian clinician. J Postgrad Med. 2007;53:267-9.

Publication Dates

• Publication in this collection
  15 Jan 2013
• Date of issue
  Dec 2012

History

• Received
  14 May 2012
• Accepted
  29 Sept 2012