Ceiling effects in the "Effectiveness of adjunctive antidepressant treatment for bipolar depression" study: was the sky the limit?

Efeitos do teto no estudo "Eficácia do tratamento antidepressivo adjuvante para depressão bipolar": era o céu o limite?

LETTERS TO THE EDITORS

Ceiling effects in the "Effectiveness of adjunctive antidepressant treatment for bipolar depression" study: was the sky the limit?

Efeitos do teto no estudo "Eficácia do tratamento antidepressivo adjuvante para depressão bipolar": era o céu o limite?

The widely discussed study by Sachs et al., namely the Effectiveness of adjunctive antidepressant treatment for bipolar depression is based on a pragmatic trial that randomly assigned bipolar depression patients to receive mood stabilizer plus placebo or mood stabilizer plus antidepressant. It achieved similar results in terms of durable recovery and other secondary outcomes. Considering that the aforementioned study employed a new methodological approach, I believe that it would be worthwhile exploring its advantages and disadvantages so as to be better positioned to interpret current studies, as well as to design new ones. Based on the aspects listed below, I would like to discuss whether or not the study contains a "ceiling effect" i.e., a false negative finding resulting from performance and selection biases.

1) Refractoriness - In the study by Sachs et al., 60% of the patients presented more than 10 previous manic and depressive episodes,¹ thus suggesting that this group may constitute a refractory sample. Refractoriness is associated with poor treatment outcomes and, therefore, to ceiling effects.² For instance, in the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study, the cumulative remission rates after one, two, three or four antidepressant trials were 33%, 57%, 63% and 67%, respectively.³ Sachs et al. aimed at detecting an absolute difference of 15% between groups, a goal that could hardly be achieved had the subjects presented at least a moderate rate of refractoriness.

2) Strict primary outcomes - the primary outcome of the Sachs et al. study was "durable recovery", defined as eight consecutive weeks of euthymia, with no more than two manic or depression symptoms. This is a more orthodox criterion when compared to the criterion generally applied to most antidepressant trials, which tend to define remission as a mood score <10 points, which, in turn, can be translated as the presence of 4 to 5 mild or 2 to 3 moderate symptoms.⁴ When stricter outcome criteria are adopted, both the experimental and the control interventions are more likely to yield similar response rates, thus leading to a ceiling effect.

3) Excessive "noise" - clinical trials are ultimately designed to detect whether treatment effects (the "signal") surpass non-specific effects (the "noise"). In the Sachs et al. study, patients were allowed to increase the dose of the mood stabilizer and enhance the use of pharmacotherapy, psychotherapy, etc. Such factors could have increased the trial "noise" by increasing non-specific effects at the expense of decreasing the "signal" of the experimental treatment.

4) Design issues - Sachs et al. compared mood stabilizer + placebo against mood stabilizer + antidepressant drug. In such type of design, ceiling effects are important since the effect of the combined intervention is weaker compared to the effect of each intervention tested separately.² For example, when two treatments whose individual remission rate is set at 50% are combined, the combined treatment is not likely to lead to full remission (100%). To assess an antidepressant's efficacy, which was probably lower in the Sachs et al. study, the study's design should have included a pure placebo arm. The study, however, followed the ethical principle of equipoise, which states that should be genuine uncertainty on the preferred treatment when designing interventional studies.⁵ Therefore, the use of pure placebo arms in pragmatic, phase IV studies is unfeasible due to ethical concerns involving the offering of an inferior treatment for a prolonged period of observation.

In conclusion, given that the number of pragmatic trials in Psychiatry has increased over time, we are now allowed to test the effectiveness of interventions in the "real world". As exemplified by the Sachs et al. study, such trials, however, have certain design characteristics that can lead to ceiling effects. Therefore, when translating results into clinical practice, clinicians should be aware of the possible caveats to be found in pragmatic studies.

Andre Russowsky Brunoni

Center for Clinical Research, University Hospital,

Universidade de São Paulo (USP), São Paulo, SP, Brazil

Department of Neurosciences and Behavior,

Institute of Psychology, Universidade de São Paulo (USP),

São Paulo, SP, Brazil

Click to enlarge

• 1. Sachs GS, Nierenberg AA, Calabrese JR, Marangell LB, Wisniewski SR, Gyulai L, Friedman ES, Bowden CL, Fossey MD, Ostacher MJ, Ketter TA, Patel J, Hauser P, Rapport D, Martinez JM, Allen MH, Miklowitz DJ, Otto MW, Dennehy EB, Thase ME. Effectiveness of adjunctive antidepressant treatment for bipolar depression. N Engl J Med 2007;356(17):1711-22.
• 2. Jindal RD, Thase ME. Integrating psychotherapy and pharmacotherapy to improve outcomes among patients with mood disorders. Psychiatr Serv 2003;54(11):1484-90.
• 3. Gaynes BN, Rush AJ, Trivedi MH, Wisniewski SR, Spencer D, Fava M. The STAR*D study: treating depression in the real world. Cleve Clin J Med 2008;75(1):57-66.
• 4. Gelenberg AJ, Thase ME, Meyer RE, Goodwin FK, Katz MM, Kraemer HC, Potter WZ, Shelton RC, Fava M, Khan A, Trivedi MH, Ninan PT, Mann JJ, Bergeson S, Endicott J, Kocsis JH, Leon AC, Manji HK, Rosenbaum JF. The history and current state of antidepressant clinical trial design: a call to action for proof-of-concept studies. J Clin Psychiatry 2008;69(10):1513-28.
• 5. Freedman B. Equipoise and the ethics of clinical research. N Engl J Med 1987;317(3):141-5.

Publication Dates

• Publication in this collection
  26 Apr 2011
• Date of issue
  Mar 2011