The Inherent Problems With Randomized Controlled Trials

By |August 15, 2011|Editorial, Research|

The Inherent Problems With Randomized Controlled Trials (RCTs)

The Chiro.Org Blog

SOURCE:   J Manip Physiol Ther 2003 (Sep); 26 (7): 460–467

Anthony Rosner, PhD

Foundation for Chiropractic Research and Education,
1330 Beacon Street, Suite 315,
Brookline, MA 02446, USA.

For 50 years, the accepted standard by which the usefulness of a therapeutic treatment is judged has been the randomized controlled trial (RCT), building from Hippocrates’ premise 2000 years ago that experience combined with reason was the therapy of choice for patients; that is, any treatment plan should both seem reasonable in theory and then be tested experimentally. Assuming that threats to both internal and external validity could be ruled out, the RCT became what is commonly regarded as the highest quality of clinical outcome study that could be mounted to allow inferences about cause and effect relationships to be drawn. The thinking was that the more rigorous and fastidious the design, the more credibility could be attached to the conclusions drawn from the outcomes of the study and the more likely the intervention was thought to have brought about those outcomes. [1] One of the strongest proponents of the RCT through the 1950s and 1960s was the British epidemiologist Archie Cochrane, who held that this type of experimental approach was essential for upgrading the quality of medical evidence. [2] In common hierarchical schemes of clinical experimental design, the RCT has been ranked the highest in rigor, as shown in Table 1.3 Even greater rigor has been presumed to occur with the statistical combination and weighting of the results of multiple RCTs in a meta-analysis to generate a more conclusive estimate of effect size. [4-5]

Table 1.   Hierarchy of Experimental Designs [3]

  1. Control group outcomes study (including RCTs).
  2. Single-subject experiment, replicated single-subject experiments.
  3. Single-group outcome study.
  4. Systematic case study.
  5. Anecdotal case report.

Designs are presented in descending order of rigor.