Randomized controlled trials: advantages and pitfalls when studying causality

PRACTICAL SCENARIO

A pharmaceutical company has developed a new drug to improve asthma control and they are asking a respected team of investigators to design a study to compare “betteraline” (new drug) with “normalraline” (usual care). The investigators believe that the best design should be a randomized controlled trial (RCT) comparing both drugs and measuring the improvement in FEV₁ after three months of treatment as the main outcome. However, they are worried about costs, time commitment, and the need for an organized team to minimize follow-up losses, as well as about the logistics to measure the primary outcome. They wonder what pros and cons of performing an RCT are in this case.

In clinical and epidemiological research, analytical studies aim to assess the potential cause-effect association between an intervention and an outcome to ensure that causation is the best possible explanation among all available options.

To establish causation, the research question we would like to answer is: what would the outcome be if patients received an experimental intervention (factual scenario) compared with what would have happened if the same patients had received a control treatment, at the same moment of their lives, under identical conditions (counterfactual scenario)? Because we cannot test that in real life, the best substitute is to randomly select “similar” patients to receive either the intervention or the control and to compare outcomes. The outcome of the control group is the counterfactual scenario.¹ Although not perfect, this model served as the central concept inspiring the inception of randomized experiments and their statistical inference by Ronald Fisher circa 1920.

ADVANTAGES OF RCTS

RCT is a robust design because participants are randomly assigned to receive the intervention or control, which ensures that both known and unknown potential confounders are balanced at baseline in the two (or more) study groups. This process is achieved in two steps. First, the generation of a random list; second, allocation concealment, which is a procedure to prevent investigators from knowing to which group the next patient will be assigned. There are a few ways to do this, such as using sealed opaque envelopes or using digital automated response systems accessed by phone or over the Internet.

Any attempt to manipulate the process disrupts the balance that we are trying to achieve. Another advantage of RCTs is that measurement of variables during the study is prospective and ensures that all participants have measurements taken in the same manner throughout the study, avoiding information bias, minimizing missing data, and increasing internal validity.

Masking, when possible, is another advantage of RCTs. The participants, the researchers who follow the patients during the study, the researchers who are responsible for defining whether or not the participants experienced the outcome, and/or the statistician who analyzes the data may be prevented from knowing the assignment of each participant in order to minimize bias.

Performing an RCT requires a lot of preparation, with a carefully designed study protocol, a manual of procedures (for example, specific instructions to perform spirometry), a team, and an experienced leader. That takes time and money; therefore, a realistic schedule and budget are essential.

IMPORTANT CONSIDERATIONS AND PITFALLS

Participants in an RCT are not selected at random from the population of interest. They are usually referred by their doctors or self-referred by seeing advertisements or receiving recommendations from other patients, which might affect generalizability. In addition, the wonders of randomization are at the heart of RCTs, but like any vital organ, it can be affected by certain conditions:

• Crossover: patients who are assigned to one of the study arms but, due to unexpected reasons, receive the treatment of the other study arm. For instance, participants assigned to the intervention group obtain inhalers containing “normalraline” at a pharmacy.

• Nonadherence: some participants may not adhere to the assigned treatment. In our example, a patient may decide to stop using his/her asthma inhalers. If this proportion is high, or if it occurs more frequently in one arm than in another, it becomes a potential bias.

• Loss to follow-up: if a participant drops out of the study and cannot be contacted, it cannot be determined whether they experienced the study outcome or not, affecting the interpretation of the results.²

• Co-interventions: when participants receive interventions other than the main intervention, it may be difficult to know whether to attribute the benefit to the study intervention or to the co-intervention. In our example, the addition of corticosteroids to achieve asthma control is a co-intervention.

The investigators have decided to perform an RCT to test if “betteraline” is superior to usual care to treat asthma, because RCT is the most robust design to determine causality if all premises are met. To obtain valid results, the study will need careful planning, time, resources, and a dedicated team.

Figure 1
Framework and potential pitfalls in randomized clinical trials. Loss to follow up, dropout, co-interventions and crossover can happen in either of the study arms. R: randomization.

REFERENCES

• ¹
  Höfler M. Causal inference based on counterfactuals. BMC Med Res Methodol. 2005;5:28. https://doi.org/10.1186/1471-2288-5-28
  » https://doi.org/10.1186/1471-2288-5-28
• ²
  Bell ML, Kenward MG, Fairclough DL, Horton NJ. Differential dropout and bias in randomised controlled trials: when it matters and when it may not. BMJ. 2013;346:e8668. https://doi.org/10.1136/bmj.e8668
  » https://doi.org/10.1136/bmj.e8668

Publication Dates

• Publication in this collection
  22 Mar 2024
• Date of issue
  2024