Externally controlled trials (ECTs) are used when randomized controlled trials (RCTs) are impractical or unethical, particularly in rare diseases and oncology. But new analysis reveals that many ECTs fall short on methodological rigor, threatening the reliability of their findings. Unless the field embraces standardized practices, ECTs risk undermining the very evidence base they aim to strengthen.
RCTs remain the benchmark for clinical evidence, but in settings where they are infeasible, ECTs have emerged as a surrogate framework. By leveraging historical or real-world comparators, ECTs attempt to provide efficacy and safety data without randomization. Regulatory agencies, including the FDA, have signaled conditional openness to such designs. However, this latest meta-epidemiological review of 180 ECTs published between 2010 and 2023 exposes systemic shortcomings that jeopardize evidentiary validity.
Only 16.1% of ECTs prespecified the use of external controls in their protocols, creating opportunities for post hoc selection bias. While 54.4% of external controls were sourced from real-world clinical data and 37.2% from trial-derived datasets, a staggering 90% relied on non-contemporaneous data, raising the specter of temporal confounding. Even more concerning, just 7.8% conducted feasibility assessments of external control datasets, leaving major gaps in comparability, data completeness, and sample adequacy. Transparency was also lacking. Although 91.1% of studies reported important covariates, fewer than one-quarter described how they were selected, and only 4.9% prespecified covariates in the protocol. Such opacity introduces a high risk of selective reporting and threatens causal interpretability.
The study highlighted an overdependence on univariate analyses, with 75.8% of ECTs estimating treatment effects without multivariable adjustments. Among the minority employing statistical techniques, propensity score weighting dominated, but methods like entropy balancing and disease risk scores—central to modern causal inference—were rarely applied. Worse still, even after matching, 43.8% of studies retained covariate imbalances, underscoring methodological fragility. Sensitivity analyses, essential for robustness checks, were conducted in only 17.8% of studies. Even fewer—just 1.1%—applied quantitative bias analyses (QBA), a critical tool for modeling uncontrolled confounding. This near-total absence of QBA reflects a profound methodological blind spot, leaving many ECT conclusions vulnerable to hidden biases.
Improving externally controlled trials requires a comprehensive focus on design, conduct, and reporting. Researchers should begin by clearly justifying why external controls are needed and prespecifying selection criteria, data sources, and analytic strategies in study protocols to avoid bias. External controls should, whenever possible, be contemporaneous with the treatment arm, and feasibility assessments must verify the accuracy, completeness, and comparability of data. Key covariates should be predefined based on clinical relevance, and advanced statistical techniques such as propensity score methods, entropy balancing, or disease risk scores should be applied to mitigate confounding. Balance between arms should be assessed using standardized mean differences rather than p-values. Treatment effects must be estimated with multivariable models, supported by sensitivity analyses and QBA to evaluate robustness. Finally, adopting standardized reporting guidelines, such as extensions of STROBE, will enhance transparency, reproducibility, and regulatory acceptance of ECT evidence.
ECTs have undeniable utility in drug development pipelines, health technology assessments, and rare-disease research. But without methodological guardrails, they risk producing unreliable evidence that misguides regulators, clinicians, and patients. For ECTs to fulfill their promise, transparency, standardization, and advanced statistical rigor must become non-negotiable. In short, the future credibility of ECTs hinges not on their growing adoption, but on how rigorously they are designed, analyzed, and reported.
