Real World Data Studies of Antineoplastic Drugs: How Can They Be Improved to Steer Everyday Use in the Clinic?

Gold Standard – A Constantly Evolving Perceptive In Research

The translation of real world data to real world evidence has been a hot topic for several years now. However, randomised controlled trials remain the “gold standard” especially in the drug development phase of antineoplastic or anticancer drugs that include cytotoxic drugs, hormones, and signal transduction inhibitors.¹ Whereas efficacy and safety assessments of most antineoplastic drugs rely heavily on randomised controlled trials, there is a growing interest in the use of observational data or real world evidence owing to the low costs and shorter length of time required by the latter in comparison to the traditional drug approval processes.²

Although randomised controlled trials remain the gold standard during the drug development process, complementing randomised controlled trials with real world evidence especially in the regulatory phase can vastly increase the external validity and generate evidence on treatment effects observed in the real world.²

There is a need to reconsider what is defined as the gold standard in all phases of drug development. We need to evolve our perceptions and educate ourselves to apply real world evidence into practice as many (especially within the clinical community) still do not feel confident in real world evidence outcomes other than results from registry data.^3–5 Decision-making in the drug development phase needs to translate real world data into real world evidence which should be integrated with randomised controlled trials.² This should be defined as the new “gold” standard that we aspire towards and the gold standard in this context should be a constantly evolving perspective across both the research and clinical communities. The aim of this review will be to discuss and explore how real world data may be integrated with randomised controlled trials.

Big Data In Healthcare Research

The term “big data” has been used exponentially over the past few decades. Big data is not only relevant in the healthcare setting but in other industries including finance, advertising, and retail. There exist numerous definitions of big data, however, essentially, it refers to the collection of a large volume of data, which, in the healthcare setting, can be used to facilitate decision-making on the diagnosis and treatment of patients. In theory, big data is the future of medicine as it can inform artificial intelligence (AI) models thus saving clinician time and ensuring more accurate diagnosis and treatment decisions, particularly in the field of cancer. However, in practice, the innate heterogeneity in data poses a significant issue.

The collection of healthcare data takes many forms from electronic healthcare records (EHRs), claims data and registries. Within each of these methods of data collection, the variables themselves can differ substantially and the way in which the variables are coded can also differ. Even when using the data, or in randomised controlled trials, studies themselves are at times poorly designed and can have ill-defined endpoints leading to heterogenous outcomes which are not comparable to other studies.^6,7 To help overcome these issues, core outcome sets (COS) and common data models (CDM) may be applied. COS are an agreed standardised set of outcomes which should be reported as a minimum within a clinical trial.^8,9 Initiatives like core outcome measures in effectiveness trials (COMET) are paving the way to the development and application of COS in the real world.¹⁰ Moreover, other cancer specific initiatives like PIONEER, which is part of the innovative medicine initiative’s (IMI’s) “Big Data for Better Outcomes” (BD4BO), have built the development of COS into their projects. It is encouraging to see that the COS within PIONEER are endorsed by the European Association of Urology, a reflection of the importance of COS in such projects.¹¹ COS are therefore a useful tool to ensure that the correct data is collected. However, the coding of such data can differ enormously from dataset to dataset.

CDMs offer a possible solution to standardise the structure of observational data to aid in the analysis of multiple large-scale datasets. CDMs like the one developed by The observational health data sciences and informatics (OHDSI), called the observational medical outcomes partnership (OMOP) CDM are just one example of such an idea.¹² The idea behind the OMOP CDM is to facilitate the analysis of data from datasets originally built for different purposes, eg, claims data and EHR data, for one common purpose of generating evidence for clinical research.

Artificial Intelligence – Is It Worth The Hype?

Although big data can provide the platform for more standardised and structured data, heterogeneity and ethical considerations are still an issue when it comes to real world data. AI techniques have been emerging to combat the challenges of traditional methods for analysis of biological data, as it can often result in overfitting in clinical datasets. AI using machine learning approaches in biomedical research can include genetic algorithms to calibrate complex models of clinical data while preserving biological heterogeneity.¹³ The enormity of the multi-omics and biological sequencing data has opened AI to a field of genomic and proteomic data analysis. The Basic Local Alignment Search Tool (BLAST) is an example of a robust algorithm for initial analysis of each DNA sequence, identifying areas of similarity, and mapping sequences to specific sites in a reference genome. The application even calculates the statistical significance by comparing nucleotide or protein sequences to sequence databases.¹⁴ Other such examples include the expressed sequence tag (EST)¹⁵ which is comprised of raw transcriptome data and the Sequence Read Archive (SRA)¹⁶ database which holds raw DNA, complementary DNA, and RNA sequencing data along with alignment data.

However, AI has so far failed in risk prediction models due to its struggle to distinguish between real and random data patterns when data sets are modest in size, its inability to incorporate subtleties of medical data such as the effects of informative missingness, censoring by competing risks, or confounding factors, and by its limited ability to quantify the reliability of its predictions. Moreover, extensive algorithm refinement on a small sample of data (particularly in the real world) can result in a sample-specific refined algorithm with limited generalisability to larger, diverse datasets.¹⁷ An AI algorithm developed in this manner would, therefore, memorise the noise and statistical variations of the restricted sample leading to overfitting. Overfitting occurs when the number of parameters is too large compared to the number of data points available to determine clinical outcome predictions.

Modern cancer data is high-dimensional and multi-modal which requires complex mathematical models that specifically target clinical outcome prediction. Coolen et al suggests replica analysis for undoing the effects of overfitting in studies that focus on risk and outcome prediction, including (in ongoing work) for competing risk and time-to-event analyses.^18–20 When studying the risk of developing a disease, Bayesian models that allow for latent heterogeneity enable one to identify classes of individuals that differ in their susceptibility to a particular disease, how they react to a specific risk factor, or how they respond to a particular therapy. Further investigation of these individuals can therefore help to uncover indicators for either preventative or therapeutic measures.²¹ Therefore, replica analysis and further AI techniques may be useful for prediction models in the real world settings.

Efficacy And Effectiveness In Real World Data

Whilst randomised controlled trials are the gold standard in healthcare research, particularly when looking at new antineoplastic treatments, they are conducted in a highly controlled setting with strict inclusion and exclusion criteria. Once a drug or new treatment has been deemed beneficial through sufficient randomised controlled trials, the performance of that drug can differ greatly in the real world. This is because, patients who are otherwise not eligible to take part in or are under-represented within clinical trials are suddenly given a treatment which has not been tested on their demographic. Patients of ethnic minorities, older patients or patients with disabilities are just some of the examples of patients not sufficiently represented within clinical trials, which has also been acknowledged by the Food and Drug Administration in its draft guidance to enrol more patients from underrepresented racial and ethnic populations.^22,23 This is why we must consider both the efficacy and effectiveness of any drug or new treatment. The efficacy of a treatment relates to how well the treatment performs when the ideal conditions are met ie, in a randomised controlled trial.^24,25 Effectiveness, however, is how well the treatment performs in a real-world setting. Pragmatic trials offer a way to look at the effectiveness of new treatments, without losing all the benefits of randomised controlled trials.

Need for Pragmatic Trials – Trials Within Cohort (TWICs)

The need for pragmatic trials arose because many of the randomised controlled trials failed to adequately inform routine clinical practice since they were optimised to assess efficacy. A pragmatic trial aims to determine the ideal care option as opposed to an explanatory trial where causal hypotheses are investigated. Whereas explanatory trials focus on homogeneity to reduce errors and biases, pragmatic trials aim to maximise heterogeneity in all aspects. A well-designed pragmatic trial would have the following characteristics: high external validity, large sample size, simple design, diverse settings and would include mostly Phase IV trials.²⁶ Both pragmatic and explanatory aspects exist in most randomised clinical trials. This review will explore two such methodologies for conducting pragmatic trials – pragmatic-explanatory continuum indicator summary (PRECIS) and trials within cohorts (TWICs). The PRECIS tool with 10 domains from eligibility criteria to primary outcomes was developed by Thorpe et al to help researchers design trials that take both aspects into account.²⁷

TWICs evolved from the need for pragmatic trials to better inform decision-making in healthcare. TWICs was first introduced as “cohort multiple randomised controlled trials” designed by Clare Relton et al in 2010.²⁸ TWICs make use of cohorts to conduct multiple randomised controlled trials without compromising the precision of randomisation while retaining the characteristics of real world data in routine clinical practice. It was developed to address shortcomings relating to recruitment, treatment comparisons and ethics associated with existing randomised controlled trials design.²⁸

The Graham Roberts Study was the first single-centre TWICs study in bladder cancer that created an observational cohort of bladder cancer patients with longitudinal measurements of cancer characteristics and outcomes recorded.²⁹ Following consent to the study, patients were given validated self-administered questionnaires which aimed to collect patient-reported outcome measures including quality of life, fatigue, anxiety and depression, physical activity, and dietary habits. This information was collected at baseline and every 12 months thereafter for at least 10 years.

The Utrecht cohort for Multiple BREast cancer intervention studies and Long-term evaLuAtion (UMBRELLA) Fit trial within the UMBRELLA breast cancer cohort also used the TWICs framework. At cohort entry, breast cancer patients provided consent for randomisation in future trials. For the UMBRELLA Fit trial, inactive patients were randomised 12–18 months after cohort enrolment. Whereas the intervention group (n = 130) were offered a 12-week exercise intervention, the control group (n = 130) received usual care.³⁰ Although the offer of an exercise intervention resulted in no improvement on quality of life, it gave more of an insight into exercise intervention in this group as 52% of inactive women with breast cancer accepted the intervention. A modest reduction in fatigue levels was also noted in the intervention group.

Although the TWICs design enhances study recruitment logistics, there are some limitations due to non-compliance owing to refusal of an intervention when offered one at the start of a study, which attenuates the estimated effect of the intervention under investigation.³¹ Moreover, if an intervention measurement coincides or overlaps with a cohort measurement as was the case in the UMBRELLA study,³⁰ self-reported measurements may be completed with trial participation in mind thus diluting the effect size of the intervention. To employ cohort measurements as pre- and post-intervention measurements, thorough intervention planning is therefore necessary in between cohort assessments.

Quest For The Perfect Study Design

One of the biggest challenges that affect real world evidence are biases including unmeasured confounding and confounding by indication.³² Confounding by indication or channeling bias is a major concern when physician prescription behaviour is influenced by patient characteristics rather than random allocation. Moreover, choosing the wrong controls in a case-control study and accounting for the wrong factors or confounders in a study can result in reporting non-existent associations.³³ A randomised controlled trial can account for these biases to a certain extent by using methods such as randomisation, blinding and concealment. Identification and investigation of treatment effects and its extent are feasible in a randomised controlled trial without knowing the risk factors associated with the treatment. However, randomised controlled trials are not without their own limitations.

Randomised controlled trials and real world evidence must complement one another. However, the biggest challenge with this is that findings of randomised controlled trials and real world evidence vary significantly in some cases. Whereas real world data showed a lower risk of developing cardiovascular disease in men with prostate cancer on long-term androgen deprivation therapy, no such associations were found in randomised controlled trials.^34,35 Such contradictory findings largely depend on the varying study populations and metrics used in the two designs. The Food and Drug Administration Demonstration Project therefore seeks to gain insights on factors that define clinical questions that can be evaluated using real world data, which will lay the foundation for future substitutions of randomised controlled trials with real world evidence.³⁶

Conclusions

Real world data is often limited as it is affected by incomplete data, selection bias and residual or unmeasured confounding due to a lack of randomisation, which means they are far from being classed as substitutes for randomised controlled trials in studies of antineoplastic drugs. However, large, well-established patient cohort studies with consistent methodological rigour and AI methods designed to accommodate disease risk prediction models can complement pragmatically designed randomised controlled trials and can ultimately help bridge the gap between randomised controlled trials and clinical practice.