About this blog

CRIT is no longer adding posts or updating the CRITical Thinking blog.

CRITical Thinking is a blog written by staff, directors, and friends of the Collaboration for Research Integrity and Transparency (CRIT), a joint program of Yale Law School, Yale School of Public Health, and Yale School of Medicine. CRIT's mission is to promote health by improving the integrity and transparency of biomedical and clinical research.

This blog is published by and reflects the personal views of the individual authors, in their individual capacities. It does not purport to represent Yale University's institutional views, if any. No representation is made about the accuracy of the information, which solely constitutes the authors’ personal views on issues discussed. The information contained in this blog is provided only as general information and personal opinions, and blog topics may be updated after being initially posted.


Evaluating the evidence used by the US Food and Drug Administration during the drug approval process

May 18, 2018

Over the past few years, the US Food and Drug Administration (FDA) has faced continual pressure to accelerate the review and approval of new drugs, in order to promote innovation and provide rapid access to new treatments for patients with serious life-threatening conditions. As expedited review and approval programs have increasingly been adopted for certain drugs, there have been concerns that drugs are being approved on the basis smaller, shorter, and fewer trials. Our study published recently in BMC Medicine aimed to evaluate one component of clinical trial evidence that informs FDA approval decisions for new drugs: the study outcomes.

FDA’s approval process: Pivotal trials, patient-relevant endpoints, and surrogate markers

In the US, the FDA is responsible for determining whether a new drug is safe and effective. Generally, drug manufacturers must provide data from at least two “adequate and well-controlled” trials—often referred to as pivotal trials—that demonstrate independent evidence of safety and efficacy.

The FDA’s approval decisions are guided by the primary outcomes evaluated in pivotal trials. These primary outcomes measure the impact that new treatments have on preventing diseases or symptoms. Ideally, clinical trials should use patient-relevant endpoints (often referred to as final endpoints or clinical endpoints), which reflect how patients feel, function, or survive.

Although patient-relevant outcomes are most likely to inform clinical practice, trials that evaluate these may not always be ethical, realistic, or practical. For instance, if a drug is being evaluated in a population with early-stage cardiovascular disease, thousands or tens of thousands of patients would need to be studied for an extensive period of time in order to determine differences between the intervention and a control therapy on the basis of patient-relevant outcomes, such as mortality, stroke, or myocardial infarction.

In certain instances, it may be more realistic to use intermediate endpoints that are meant to predict and replace patient-relevant outcomes. In particular, surrogate markers of disease as trial endpoints (i.e. surrogate endpoints, biomarkers, or intermediate endpoints) can be used to decrease the size, duration, and cost of a clinical trial. For example, systolic blood pressure—a surrogate marker that can be measured multiple times throughout a trial—is considered to be a reliable predictor of major cardiovascular events in patients with hypertension, such as mortality, stroke, or myocardial infarction.

Certain surrogate markers that have been repeatedly shown to correlate with patient-relevant outcomes (i.e. they have been formally evaluated and “validated”) can be used as the primary outcomes in pivotal trials. Moreover, certain drugs that target serious or life-threatening conditions can be evaluated through a special expedited FDA review pathway (the Accelerated Approval pathway), and can be approved on the basis of surrogate markers that are only “reasonably likely” to predict patient-relevant outcomes. However, after approval has been granted, drug sponsors are required to perform additional trials to confirm efficacy.

The curious case of surrogate markers

Prior research has shown that trials using surrogate endpoints as their primary outcome formed the exclusive basis of approval for nearly half of the new FDA-approved indications between 2005 and 2012. However, multiple studies have shown that commonly used surrogate markers are often poor proxies for patient-relevant outcomes. Evidence also suggest that surrogate markers may actually overestimate the benefit associated with a drug. Primary outcomes based on surrogate markers can allow for shorter and smaller trials, which are two characteristics that have been associated with potentially exaggerated trial results.

Comparing the outcomes from pivotal and postapproval trials

In our new BMC Medicine article, we aimed to evaluate the reliability and consistency of surrogate markers when used in premarket trials as compared to postapproval trials. In particular, we compared the magnitudes of the primary outcomes (i.e. treatment effects) among pivotal trials supporting FDA approval of novel therapeutics, on the basis of surrogate markers of disease, with those observed among trials for the same uses (i.e. indications) completed after drug approval (i.e. postapproval trials)

One of our most surprising findings was that we were often unable to locate postapproval trials that evaluated the same drugs for the same indications with either the same surrogate markers as pivotal trials or appropriate patient-relevant endpoints.

Among our sample of 88 drugs for 90 different indications approved on the basis of one or more pivotal trials using surrogate markers, only three drugs for three indications had postapproval trials that evaluated patient-relevant endpoints. We also found matching pivotal and postapproval trials for 27 drugs for 27 indications that evaluated the same surrogate markers as endpoints.

When we compared the treatment effects among pivotal trials with surrogate markers to those observed among postapproval trials for the same drugs, indications, and surrogate markers as endpoints, we found that treatment effects based on non-continuous surrogate markers were often significantly larger than those observed among postapproval trials. However, among continuous surrogate markers (such as the differences between the treatment and control group in the change in diastolic blood pressure), we found no evidence of a difference between pivotal and postapproval trials.

In our article, we outline potential reasons that could explain the inconsistencies between the treatment effects observed among pivotal and postapproval trials. For instance, we found that postapproval trials were significantly smaller and less likely to be double blinded, which may indicate lower methodological quality and potentially exaggerated treatment effects among postapproval studies.

Our study has certain limitations that are worth mentioning here. To ensure an adequate number of matched pairs, we required only that pivotal and postapproval trials evaluated the same drug for the same indication with the same endpoint based on a surrogate marker. Even though the majority of our sample fulfilled all matching criteria, we acknowledge that most trials will have different eligibility criteria and methodological characteristics, which could influence the observed effects. For instance, for four of the matched pairs, the pivotal trial had a placebo comparator and the postapproval trial had an active comparator.

This between-study heterogeneity could influence the observed treatment effects that were then compared. But it is also important to note that, for a number of pivotal trials, we found no postapproval trials that evaluated the same drug for the same indication with the same comparators.

That said, this study may suggest that the treatment effects from pivotal trials supporting FDA approval of novel therapeutics on the basis of surrogate markers of disease are often larger than the treatment effects observed among postapproval trials using the same surrogate markers, specifically for non-continuous endpoints. Overall, our study indicates that there is a lot of uncertainty when it comes to surrogate markers. Considering that there are new proposals for increased reliance on smaller and shorter trials with wider use of surrogate markers in the US, policy makers, doctors, and patients should interpret treatment effects based on surrogate markers of disease as primary endpoints with caution.


Dr. Joshua D. Wallach is a research fellow at Yale University, working within the Collaboration for Research Integrity and Transparency (CRIT) and the Center for Outcomes Research and Evaluation (CORE).

Dr. Joseph S. Ross is an Associate Professor of Medicine (General Medicine) and Public Health (Health Policy and Management)

This post originally appeared on BioMed Central Blogs Network