Dr. Scott Gottlieb—Donald Trump’s pick to lead the Food and Drug Administration (FDA)—has been praised and criticized for having worn many hats during his career. He is not only a physician, but also a venture capitalist with close ties to industry and a former FDA deputy commissioner during the George W. Bush administration. Last week, the Senate Committee on Health, Education, Labor, and Pensions (HELP) voted 14-9 to advance Dr. Gottlieb’s nomination to the full Senate, where he will likely be confirmed. What remains unclear, however, is how his many hats will influence his interpretation of the FDA’s gold standard of safety and efficacy for medical products.
During Dr. Scott Gottlieb’s hearing with the Senate HELP Committee on April 5, Senator Aaron Burr (R-NC) asked the follow question:
Senator Burr: “In 2017, should there still be double blind studies where the doctor doesn’t even know whether the patient is getting a placebo or not?”
Ideally, Dr. Gottlieb’s response could have been something along the lines of: “Yes, of course we need double blind studies.” However, Dr. Gottlieb’s actual response suggested a shift away from the traditional understanding of “gold standard”:
Dr. Gottlieb: “… I believe that there are opportunities to modernize how we do clinical trials in ways that aren’t going to sacrifice on the gold standard of safety and effectiveness but perhaps could think of clinical trial constructs that don’t require the tight randomization that current clinical trials do, and Congress believes this as well because it’s directed the agency to look at alternative clinical trial designs as a component of the review process.”
When it comes to clinical study design, rigorous and adequately powered double-blind, randomized controlled trials (RCT) are often considered the “gold standard.” There are hundreds of biases in biomedical research that can compromise a study’s validity. “Blinding” and “randomizing” are two important trial features that help to minimize some of these biases. While Dr. Gottlieb notes that the gold standard of safety and effectiveness should not be sacrificed, he also wants to “modernize” the design of clinical trials in ways that could decrease the validity of research findings.
Blinding
Blinding minimizes the influence that trial participants’ or investigators’ beliefs, actions, or expectations can have on study results. There are three main types of blinding when it comes to RCTs: single blinding (where the patient is unaware of treatment allocation), double blinding (where both the patient and investigator—i.e. “doctor” as stated by Senator Burr—are unaware of treatment allocation), and triple blinding (where the patient, investigator, and data collectors/organizers are unaware of the treatment allocation).
Contrary to Senator Burr’s doubts, double-blinding is critical for scientific integrity as researchers recording and assessing the outcome(s) of trial may unconsciously or subtly influence study findings if they are aware of treatment allocation. For example, an investigator eager to prove that a new treatment is effective may provide a higher quality of overall care to those in the treatment group.
Evidence suggests that when investigators are not blinded, they may make over-optimistic estimates of intervention effects for subjectively assessed outcomes. According to the FDA Draft Guidance for Industry on Adaptive Design Clinical Trials for Drugs and Biologics, “shielding the investigators as much as possible from knowledge of the chosen adaptive changes is important because knowledge of the interim unblinded data used to make the adaptation decision, or even knowledge only of the specific adaptive choice, has the potential to introduce operation bias into the treatment-effect estimates.”
The whole point of designing clinical studies “where the doctor doesn’t even know whether the patient is getting a placebo or not” is to ensure valid (unbiased) estimates of safety and efficacy. Given the importance of double-blind studies, it raises concerns that Dr. Gottlieb does not directly speak to whether they are still necessary.
Randomized Controlled Trials
“Randomizing” and “controlling” studies, like blinding, are critical components of trial design that minimize various biases and allow investigators to estimate the differences in treatment efficacy and safety between the two (or more) groups. In a typical two-armed randomized controlled trial, participants are randomly allocated to receive either the novel intervention or a placebo.
Proper randomization helps to ensure that the baseline characteristics of participants (i.e., age, sex, and race), which can influence the way that individuals respond to a certain intervention, are as similar as possible across the different treatment groups being compared. Randomization minimizes systematic differences between treatment groups and allows for an evaluation of the possible treatment effects that are due to the actual interventions instead of patient characteristics.
In his answer, when Dr. Gottlieb envisions clinical trial designs that do not require “tight randomization,” it appears as if Dr. Gottlieb may be talking about adaptive clinical trials (ACT). Adaptive trials allow researchers to modify characteristics related to study design or analyses based on interim examinations of the accumulated data. According to the FDA Draft Guidance for Industry Adaptive Design Clinical Trials for Drugs and Biologics, there is “great interest in the possibility that clinical trials can be designed with adaptive features.” ACTs are particularly popular among those eager to decrease the number of study participants required per study, lower the costs of research, and shorten the duration of individual trials.
However, one of the main problems with ACTs is that they are extremely complex. At a minimum, they require greater statistical expertise during the design, conduct, and analyses phases of a clinical trial than in a traditionally-designed clinical trial. According to a recent survey of researchers’ attitudes towards adaptive trials, the attitudes and beliefs about ACTs among trial community representatives are inconsistent. In particular, biostatisticians are not as optimistic about the validity and conclusions of adaptive trials. “Modernized” trials that allow for changing arms and randomization rules are likely to lead to more biases. Unfortunately, Dr. Gottlieb only speaks to adopting new clinical trial designs without discussing their limitations.
Moving forward, the research community will need to ensure that there is not an erosion in the design of clinical trials which would lower the evidentiary standards at the FDA.
“Risk aversion within the culture of the FDA”
After asking about the standard for drug approval, Senator Burr goes on to ask Dr. Gottlieb about the “culture” of the FDA.
Senator Burr: “In fact, Congress in 2012 passed breakthrough therapy legislation on drugs. We followed it up in the Cures Act with a breakthrough therapy pathway for devices. That was an acknowledgement by Congress that there was a risk aversion within the culture of the FDA. So, if confirmed, what are you going to do to change the culture of the FDA?”
It’s puzzling to hear Senator Burr criticize the FDA for being risk-averse when the very mission of the agency—the impetus for its creation and its modern evolution—is to protect consumers from harms. The FDA’s job is to scrutinize the risks presented by therapeutics and ensure that those risks are outweighed by proven clinical benefits. Thanks to the risk aversion of Frances Kelsey, we dodged thalidomide in the 60s, and Congress created a regulatory system that ensures that medical products are safe and effective before they are widely marketed.
However, Senator Burr insists that Congress now wants to fix the “culture” of the FDA and asks Dr. Gottlieb how he plans to do that. The preface of Senator Burr’s question is that Congress’ creation of expedited approval pathways for “breakthrough” drugs and devices was a message that the FDA is too cautious.
Senator Burr’s logic is flawed. While it’s true that Congress has carved out various expedited pathways to speed drug approval, these pathways were intended to be the exception, not the rule – exceptions reserved for truly promising innovations where there are compelling scientific justifications and unmet clinical need. They were not intended to change the “culture” of the FDA so that all medical products would be approved faster with less reliable evidence.
Unfortunately, Dr. Gottlieb seems to agree with the overall spirit of Senator Burr’s question.
Dr. Gottlieb: “… I want to find ways to look at what’s working well within the agency … and try to bring those best practices to the parts of FDA that might not be adopting the tools … We’ve seen, for example, … uneven adoption of the principles embedded in the breakthrough therapy pathway. I think there are ways to inspire more consistency and more broad adoption of those ethos.”
It is troubling to hear Dr. Gottlieb imply that expedited pathways, rather than selectively used when scientifically appropriate, should inspire a new norm. When it comes to the “principles embedded in the breakthrough therapy pathway,” he wants to go from “uneven adoption” to “more broad adoption.”
What both Senator Burr and Dr. Gottlieb overlook is the scientific literature that warns us of the problems raised by approving drugs faster and with less evidence. Studies have shown that drugs billed as “breakthrough” often have not been clinically effective. Moreover, the term “breakthrough”—being vague—can be used by companies as a “marketing gimmick,” has been misinterpreted by patients and providers, and has led to overly optimistic views about the effectiveness of drugs or the strength of the evidence supporting their uses.
Dr. Gottlieb does not address these concerns but goes on to explain how he will use “benchmarks” to measure the FDA’s performance in adopting new, alternate ways of approving medical products.
Dr. Gottlieb: … That’s something we were able to do when I was at the FDA the last time in part by challenging the agency to examine its own processes to see how it was performing against these kinds of benchmarks … and I would hope to bring back those same tools … with respect to these new set of authorities that Congress has given FDA.
Earlier in the hearing, Dr. Gottlieb urged us to “reject the false dichotomy” of speed vs. safety and stated that he hopes to promote speed without sacrificing the standard of safety and effectiveness that has been “enshrined in the law many times.” But such statements aren’t reassuring when he uses the word “benchmarks” with respect to the “new set of authorities that Congress has given FDA”—i.e. the provisions in the 21st Century Cures Act that pressure the FDA to approve drugs faster by relying on surrogates, biomarkers, real world evidence, data summaries, etc.
All of these tools aren’t bad in themselves. Flexible standards can lower the cost and time for drug approval, which can be a good thing. But taking on greater risks at the front end of market approval makes it all the more imperative to follow up with post-market studies. And Dr. Gottlieb gives us no answers as to how he will ensure that the standard of safety and effectiveness won’t be sacrificed in pursuit of speed.
In 2017, we can only hope that among Dr. Gottlieb’s many hats, the one that says “Make the FDA great again!” is aligned with research and regulatory integrity.
Jeanie Kim is a research fellow at Yale Law School and the Collaboration for Research integrity and Transparency [CRIT].
Dr. Joshua D. Wallach is a research fellow at Yale University, working within the Collaboration for Research Integrity and Transparency [CRIT] and the Center for Outcomes Research and Evaluation [CORE].