About this blog

CRIT is no longer adding posts or updating the CRITical Thinking blog.

CRITical Thinking is a blog written by staff, directors, and friends of the Collaboration for Research Integrity and Transparency (CRIT), a joint program of Yale Law School, Yale School of Public Health, and Yale School of Medicine. CRIT's mission is to promote health by improving the integrity and transparency of biomedical and clinical research.

This blog is published by and reflects the personal views of the individual authors, in their individual capacities. It does not purport to represent Yale University's institutional views, if any. No representation is made about the accuracy of the information, which solely constitutes the authors’ personal views on issues discussed. The information contained in this blog is provided only as general information and personal opinions, and blog topics may be updated after being initially posted.


Improvement in publication bias following the 2007 FDA Amendment Act

July 26, 2017

Publication bias plagues the medical literature. Positive trials are more likely to be published than negative trials, and negative trials are often published with a more positive spin than warranted by the results. This distorts the medical literature, and leaves clinicians and patients with a less than accurate understanding of treatment effectiveness.

One way to study publication bias is to look directly at trial results, and compare these to the results published in the medical literature. However, trial sponsors rarely make their results directly available to the public. Alternatively, we could use the results of trials, as described by medical officers of the Food and Drug Administration (FDA), who do have direct access to trial results. This allows us to compare those trials to the same ones described in the medical literature and see if publication bias exists for the drug in question.

An example of this is a paper from 2007 by Turner and colleagues that looked at antidepressants and found that trials described and interpreted by FDA medical officers as ‘positive’ were more likely to be published than ‘negative’ trials. While only half of all trials submitted to the FDA were positive, more than 90% of trials in the published literature were positive.

This is confusing. How often should we expect an antidepressant to work? Half of the time or nearly every time? We would arrive at different answers depending on whether we were reviewing the trials submitted to the FDA or the trials published in the medical literature.

Mitigating publication bias

Policies have been written by government and journal editors to mitigate publication bias. In 2007 the FDA Amendment Act (FDAAA) mandated clinical trial registration and reporting of results for most FDA-regulated products on ClinicalTrials.gov, an NIH-funded, publicly-accessible clinical trials registry.

This would theoretically increase transparency of clinical trial methods and results. What we didn’t know was whether this would improve transparent trial publication in the medical literature.

With this question in mind, we looked at trials supporting newly approved drugs in cardiovascular disease and diabetes – two areas of robust clinical evidence generation – completed before and after FDAAA took effect to see how frequently they were registered on ClinicalTrials.gov, published in the medical literature, and published with results that were in agreement with the FDA’s interpretation.

What we found was encouraging: after FDAAA took effect, trials were more likely to be registered (100% vs. 70%), published (97% vs. 89%), and published with findings that were in agreement with the FDA’s interpretation (97% vs. 84%). Put simply, FDAAA made the clinical research supporting approval of new drugs by the FDA more transparent and, when published, less biased.

This suggests that government policy aimed at making clinical trials more transparent, by requiring registration and reporting of methods and results on public clinical trial registries such a ClinicalTrials.gov, also creates an environment that encourages publication of both positive and negative trials, and allows negative trials to be published without a more positive spin.

Beyond transparency of clinic trial results, major patient-level data sharing efforts are underway. These efforts will further improve our understanding of how well, and for whom, our medical interventions can be expected to work.

Adam T. Phillips, MD, is a Fellow in the Division of Cardiovascular Disease at the Beth Israel Deaconess Medical Center in Boston, Massachusetts.

Joseph S. Ross, MD, MHS, is an Associate Professor of Medicine and of Public Health at the Yale School of Medicine. His work uses health policy research methods to examine the use and delivery of higher quality care and to better understand issues related to pharmaceutical and medical device regulation, evidence development, postmarket surveillance, and clinical adoption.

This blog was originally posted via the BioMed Central Blogs Network