In June 2017, Yale CRIT hosted an international conference titled “Ensuring Safety, Efficacy, and Access to Medical Products in the Age of Global Deregulation.” The following blogpost is the seventh installment of a blog series with commentaries from the conference participants. The views and opinions expressed in this blogpost are those of the author and do not necessarily reflect the position of Yale CRIT. For more blogposts related to this series, see here or click the tag “YaleCRIT17” below.

There is great interest currently in innovative methods for clinical trials.  Everyone involved in the trials process would like to find ways to make trials more efficient, both in time and resources, without reducing their reliability.  Most methods being talked about today are the same as, or minor variations of, designs that have been used (or at least proposed) in the past.  Trial designs that simultaneously evaluate multiple treatments with or without a control arm (“platform” trials), with or without options of bringing in new treatments as they come out of basic dosage and safety testing in phase 1, have been discussed and used for years in cancer trials.

“Pragmatic” trial—that is, trials that are intended to show the “real-world effectiveness” of an intervention in the broad target population (as opposed to explanatory trials that are designed to confirm a scientific hypothesis in a homogenous population)—have been around for a long time.  Probably the most pragmatic trial ever done was the trial of the Salk polio vaccine in 1954, in which nearly half a million children were randomized to vaccine or placebo with follow-up done of suspected cases of polio reported to departments of public health—no formal follow-up of individual enrollees.  Almost as pragmatic were the International Studies of Infarct Survival conducted by the Oxford group in the 1980s to evaluate treatments of myocardial infarction.  These trials implemented no eligibility criteria other than agreement by doctor and patient that they were uncertain as to whether the treatment would help, very limited collection of baseline information (one-half page), and follow-up for survival via national health records.

The new wrinkle, applicable primarily to cancer trials, is the desire to individualize treatment according to patient characteristics.  The recognition that oncologic drugs could be developed to target tumors with certain genomic characteristics has led to “basket” trials in which the entry criteria are based on those characteristics rather than the site of the tumor in the human body. 

Platform designs and multi-stage designs do have the potential to increase the efficiency of identifying effective treatments, if not the efficiency of any particular 2-arm comparison, by utilizing a single control group and avoiding the time and administrative burden of opening multiple trials.  At a time when new agents are rapidly being developed and coming out of phase 1, platform trials will indeed be an efficient way to prioritize agents for entry into confirmatory efficacy trials.

Some have argued for more openness to basing drug approvals solely on uncontrolled trials, without the requirement for randomization between the new agent and an appropriate control.  Such an approach has been taken on rare occasions when early uncontrolled trials showed outcomes so much better than prior experience that the data provided persuasive proof of efficacy.  An example is the 1978 approval of cisplatin for treatment of testicular cancer.  But this option must be limited to the very few “penicillins” out there. 

All drugs going into phase 3 (confirmatory) trials show promise in uncontrolled studies—otherwise they would not make it to phase 3.  But only half of drugs that enter phase 3 evaluation are ultimately approved. 

If drugs that appear promising in uncontrolled trials are routinely approved without requiring evidence from randomized trials with appropriate control groups, what will we face in the near future?  More drugs to choose from, to be sure; but little in the way to guide us to the ones that are truly beneficial.  Many patients will end up being treated with ineffective or even harmful drugs. 

We cannot count on experience over time to sort out the good from the bad.  First of all, that ignores all those treated with ineffective drugs while we are waiting for “experience.”  Second, there is a long history of ineffective and actually harmful treatments being used for years until randomized trials showed that they were not beneficial.  Consider the case of antiarrhythmic drugs that were widely used to prevent severe arrhythmias until a randomized clinical trial showed that these agents were associated with a tripling in the rate of sudden death!  No one noticed.  Observational experience cannot be counted on to sort out drugs that benefit from drugs that do nothing from drugs that harm.

Susan S. Ellenberg, Ph.D. is a Professor of Biostatistics at the University of Pennsylvania School of Medicine.  She served as the first director of the Division of Biostatistics and Epidemiology at the U.S. Food and Drug Administration, a position she held from 1993 to 2004.