A lot has been written about the potential of Real World Data (RWD) and Real World Evidence (RWE) over the last decade. In the latest guidance from a regulatory agency, the European Medicines Agency (EMA) released a reflection paper on using RWD derived from non-interventional studies (NIS) to generate RWE for regulatory purposes. It emphasizes methodological considerations crucial for the design, conduct, and analysis of such studies to ensure the reliability and validity of RWE.
The primary value of RWE is its potential to address critical gaps in knowledge that traditional Randomized Clinical Trials (RCTs) often cannot fully resolve. RCTs, while the gold standard for efficacy assessment, are often conducted in highly selected patient populations under controlled conditions, limiting their generalizability to the broader clinical landscape. RWD/RWE offers a complementary approach, providing insights into the heterogeneity of Treatment Effects across diverse patient subgroups, reflecting the variability encountered in routine clinical practice. RWE allows the evaluation of factors like comorbidities, age, ethnicity, and socioeconomic status that influence treatment outcomes. RWD facilitates the assessment of long-term safety and effectiveness, real-world treatment patterns, including adherence to prescribed regimens, switching between therapies, and the use of concomitant medications. RWD also informs clinical decision-making by providing evidence on the relative benefits and risks of different treatment options and can be used to characterize the natural history of diseases.
Generating reliable RWE requires careful attention to methodological rigor. Key considerations include:
- Study Design and Target Trial Emulation (TTE): The choice of study design should be driven by the research question and the nature of the available data. For causal inference, the TTE framework provides a structured approach to emulate the design and analysis of a hypothetical RCT, minimizing bias and improving the interpretability of results. This involves clearly defining the target population, intervention, comparator, outcome, and time horizon.
- Bias Mitigation: Addressing potential biases is paramount. Strategies include
-
Selection Bias: Careful definition of inclusion/exclusion criteria, consideration of prevalent vs. new-user designs, and addressing differential loss to follow-up.
-
Information Bias: Validation of data sources, standardization of data extraction procedures, and sensitivity analyses to assess the impact of misclassification.
-
Time-Related Bias: Proper alignment of study entry, treatment initiation, and follow-up periods, and consideration of appropriate risk windows.
- Confounding Control: Confounding remains a significant challenge in NIS. Strategies include:
-
Propensity Score Methods: Use of propensity score matching, weighting, or adjustment to balance baseline characteristics between treatment groups
-
Instrumental Variable Analysis: Application of instrumental variable methods to address unmeasured confounding.
-
Negative and Positive Controls: Inclusion of negative and positive control exposures/outcomes to assess the presence of residual confounding.
- Data Quality Assessment: A comprehensive assessment of data quality is essential. This includes evaluating the reliability (accuracy, completeness, consistency) and relevance (availability of key variables, representativeness of the study population) of the data.
- Statistical Analysis: The statistical analysis plan should be pre-specified and include sensitivity analyses to assess the robustness of findings. Appropriate methods for handling missing data and addressing heterogeneity across data sources should be employed.
The EMA paper offers the following tips for successful RWD/RWE use in regulatory decisions.
- Early Engagement with Regulatory Agencies: Proactive communication with regulatory agencies (e.g., EMA) is crucial to discuss the suitability of RWD/RWE for specific regulatory purposes and to align on methodological approaches.
- Comprehensive Feasibility Assessments: Thorough feasibility assessments should be conducted to evaluate the availability, quality, and relevance of RWD for addressing the research question.
- Transparent Reporting: Study protocols and results should be reported transparently, including detailed descriptions of data sources, study design, analytical methods, and limitations.
- Data Source Characterization: Comprehensive characterization of data sources is essential, including information on data collection procedures, data quality controls, and potential biases.
- Collaboration and Data Sharing: Collaboration between researchers, data holders, and regulatory agencies is encouraged to facilitate data sharing and promote the development of best practices for RWE generation.
- Continuous Improvement: Ongoing efforts are needed to improve data quality, develop new analytical methods, and refine regulatory guidelines for the use of RWD/RWE.
By adhering to these recommendations, clinicians and researchers can contribute to the generation of robust and reliable RWE that informs clinical decision-making and supports regulatory evaluations.
