Understanding the evidence used in drug product withdrawals

In the ever-evolving landscape of regulatory decisions, the quest for safeguarding public health remains paramount. Decisions to remove medicines from market are not taken lightly and are often based upon all evidence available to inform a benefit/risk balance. This evidence can come from a range of sources, such as randomised controlled trials (RCTs), observational studies, and spontaneous reporting. But over time, trends in what evidence is most available and therefore most used, change.

Over the past few years, we have researched the types of evidence used in regulatory decisions for drug product withdrawals due to safety concerns in Europe. Our work unveiled a mosaic of sources, shedding light on the diverse range of evidence that supports such critical decisions.

Among the various forms of evidence, spontaneous reports and published case studies emerged as the bedrock, underpinning a significant majority of regulatory actions.

The value of epidemiological and observational research as evidence

However, there has been a noteworthy transformation with a growing reliance on epidemiological and observational research, which involve research where the investigators have no role in assigning which patients are exposed or unexposed, rather investigators observe patients’ experiences in the real-world clinical practice setting instead of under artificial conditions of a clinical trial.^1,2

This signals a promising trend towards a more comprehensive and diversified approach in regulatory decision-making.

Our research showed that in Europe between 1999 and 2016, 35 products were withdrawn. Ninety-one percent of withdrawals were supported by spontaneous reports and/or published case reports or case series, 60 percent were supported by RCTs, 49 percent were supported by non-randomised trials, and 40 percent were supported by observational studies. Meta-analyses contributed least frequently (31.4 percent).¹

The use of observational studies to support product withdrawals has increased considerably during the study period. Between 1999 and 2001, it supported 17 percent of withdrawals, between 2002 and 2011, it supported 26 percent of withdrawals and by 2012 to 2016, 80 percent of withdrawals were supported by observational studies (cohort, case-control, or other epidemiologic design).¹

The shift in regulatory decision-making

Using evidence from observational studies

The findings, reflecting a crucial shift in decision-making paradigms, offer insightful perspectives on the interplay between evidence, regulatory measures, and their direct influence on public health.

Evidence from observational studies is key to understanding the effectiveness and safety³ of medicines and vaccines in the real world. This may come as a surprise because RCTs are seen as the gold standard of drug development, and they certainly have their place. But from a safety point of view, observational studies can be more representative of the population the drug will serve. These studies are larger in numbers and take place over a longer period. They also include many of the common “exclusions” in RCTs, such as pregnant women, children, the elderly and patients on multiple medications.

Observational studies should not be seen as competition to RCTs but as complementary to them, and as a sector we need to stop working in silos and develop an integrated and more comprehensive approach to evidence-based decision-making in safeguarding public health. 

The UK is well placed to be a leader in the development of observational studies. Our NHS system allows for longitudinal follow-ups that simply are not possible in many other countries. As of October 2023, 62.93 million patients were registered with GP practices in England, which hold health information about each person “from the cradle to the grave”.^4,5 Even if a patient moves to a different GP practice their medical record moves with them, thus data continues to be updated throughout their life and therefore provides a more complete picture for researchers to study.

A rich resource

These health records can be a rich text for researchers. They enable a range of observational study designs to be used where researchers study a period of exposure to a particular medicine and compare it with a control period of non-exposure in the same patient to establish any changes in the risk of adverse side effects.

The detail of our medical records is also an advantage for observational researchers. Information on demographics, prescribed medications, vaccine history, health screening, diagnosed conditions, and potential confounders, such as body mass index and ethnicity, are all logged in detail in our records. Disease epidemiology studies can be conducted using these data looking at risk factors for disease and disease progression for example, as well as assessing drug utilisation patterns amongst exposed patients, including how and why drugs are prescribed and to whom.

All this information is available to researchers, anonymised, of course, and subject to the right approvals. Initiatives in the UK such as the Clinical Practice Research Datalink (CPRD), The Health Improvement Network (THIN) and OpenSAFELY support UK, European, and international studies. It is also possible to link to other databases, including hospital admissions and ONS death data.

Creating effective pharmacovigilance legislation

Despite these benefits, observational studies are still seen as second best to RCTs, however that opinion is changing. In 2012, an update to pharmacovigilance legislation aimed to produce more rapid regulatory decisions based on more robust data,³ and our research supports this aim. To continue this trend into the future, it is important that infrastructure is developed to allow for background data, including background incidence of events in the population, to be available as they are needed.

Part of this infrastructure may be technological advancements, or international networks of databases with planned structure and processes in place to allow for rapid production and earlier delivery of epidemiological results.

Additionally, methods to analyse and synthesise multiple sources of data should be progressed, including statistical techniques for observational studies, and the use of ‘living’ designs to continually update the evidence produced by studies such as systematic review or benefit/risk assessments.

Finally, the use of interim reports at periodic and planned points during the study process should be built in when designing a study, ensuring that results are shared with key stakeholders in a timely manner.

Through our years of research on the types of evidence used in regulatory decisions to remove products, it is clear a more holistic approach to evidence utilisation promises a future where decisions are based on a web of evidence which complements each other instead of being competitive, which ultimately leads to better-informed decisions for the benefit of everyone.

1. Lane S, Lynn E, Shakir S. Investigation assessing the publicly available evidence supporting postmarketing withdrawals, revocations and suspensions of marketing authorisations in the EU since 2012. BMJ Open. 2018; 8(1).
2. The European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). Guide on Methodological Standards in Pharmacoepidemiology (Revision 11). EMA/95098/2010. [Internet] European Medicines Agency. 2010. [cited Dec2023]. Available from: https://www.encepp.eu/standards_and_guidances/documents/01.ENCePPMethodsGuideRev.11.pdf
3. Implementation of the pharmacovigilance legislation 2011 [updated 18 May 2015]. [Internet] European Medicines Agency. 2011. [cited Dec2023]. Available from: https://www.ema.europa.eu/en/human-regulatory-overview/pharmacovigilance-overview/legal-framework-pharmacovigilance/implementation-pharmacovigilance-legislation#:~:text=The percent20pharmacovigilance percent20legislation percent2C percent20which percent20came,the percent20European percent20Union percent20(EU).
4. Pressures in general practice data analysis 2023 [updated 30 November 2023]. [Internet] British Medical Association. [cited Dec2023]. Available from: https://www.bma.org.uk/advice-and-support/nhs-delivery-and-workforce/pressures/pressures-in-general-practice-data-analysis#:~:text=September percent202015 percent20to percent20September percent202023,-0 percent2010 percent2C000 percent208 percent2C000&text=Whilst percent20the percent20GP percent20workforce percent20is,with percent20GP percent20practices percent20in percent20England.
5. Cradle to grave. [Internet] Available from: British Medical Association. 2022. [cited Dec2023]. Available from: https://www.bma.org.uk/news-and-opinion/cradle-to-grave.