The Importance of Replication Studies

 

By guest contributor and TEDMED 2015 Speaker Elizabeth Iorns, PhD

Every year, billions of dollars are spent funding biomedical research, resulting in more than one million new publications presenting promising new results. This research is the foundation upon which new therapies will be developed to enhance health, lengthen life, and reduce the burdens of illness and disability.

In order to build upon this foundational research, these results must be reproducible. Simply put, this means that when an experiment is repeated, similar results are observed. Over the last five years, multiple groups have raised concerns over the reproducibility of biomedical studies, with some estimates indicating only ~20% of published results may be reproducible (Scott et al. 2008, Gordon et al. 2007, Prinz et al. 2011, Steward et al. 2012, Begley and Ellis 2012). The National Institutes of Health (NIH), the largest public funder of biomedical research, has stated, “There remains a troubling frequency of published reports that claim a significant result, but fail to be reproducible. As a funding agency, the NIH is deeply concerned about this problem”.

shutterstock_261331172Despite the growing concern over lack of reproducibility, funding for replication studies, the only way to determine reproducibility, is still absent. With no funding systematically allocated to such studies, scientists almost never conduct replication studies. It would be interesting to obtain the exact numbers, but it appears that last year the NIH allocated $0 to funding replication studies, out of a $30B+ budget. In the absence of replication studies, scientists end up wasting precious time and resources trying to build on a vast, unreliable body of knowledge.

It is easy to see why funders might shy away from funding replication studies. Funders want to demonstrate their “impact,” and it is tempting for them to solely focus on funding novel exploratory findings that can more easily be published in high profile journals. This is a mistake. Funders should instead focus on how to truly achieve their stated goals of enhancing health, lengthening life, and reducing the burdens of illness and disability. Although allocating a portion of funding towards replication studies would divert funds from new discoveries, it would enable scientists to efficiently determine which discoveries were robust and reproducible and which were not. This would allow more rapid advancements by allowing scientists to build upon the most promising findings and avoid wasting their time and funding pursuing non-robust results.

Some researchers find the idea of replicating previous studies unnecessary or even offensive. However, it is the responsibility of the scientific community, including funders, to work as quickly and cost effectively as possible to make progress. Introducing replication studies as part of the process provides an effective way to enable this.

If you would like to see funding specifically allocated for replication studies, please register your support here. We will share this information with funders in the hope that it will encourage them to establish funding programs specifically for replication studies to improve the speed and efficiency of progress in biomedical research.


Illustration_0007_ElizabethIorns.tif

In her TEDMED talk, cancer biology scholar and CEO of Science Exchange highlights the importance of funding reproducibility studies to advance scientific progress while maintaining its integrity.