Reproducibility has plagued the scientific community for many, many years. Shouldn’t the results of one research group be identical to another if the same procedure is followed? While this would be the ideal case, there is often variability observed due to deviations in methodologies, changes in reagent manufacturer or lot, or data misinterpretations. This problem has been in the spotlight recently, due to significant resources and time being spent attempting (often unsuccessfully) to reproduce results which have implicated the discovery of important work such as biomarker identification, disease treatment, and drug discovery.
Scientists frustrated with reproducibility (via Flickr)
So, what is the scientific community doing to combat this issue? The Reproducibility Initiative, a collaboration between PLOS ONE, Science Exchange and figshare, is working to replicate 50 top-tier cancer biology papers from 2010-2012. Another group, the Global Biological Standards Institute has identified a key issue which centers on a lack of documented standards. Their report, “The Case for Standards” includes recommendations on conducting and reporting research. Very recently, a new nonprofit group, dubbed the National Biomarker Development Alliance (NBDA), was formed to develop standards for the many steps involved during biomarker development and validation. And, journals such as Science are looking for ways that they can raise their publication standards to combat this issue. They’re encouraging their authors to include additional details for reproducing methods, looking for ways to allow transparent data accessibility, and plan to make better use of scientists trained in high-level statistics to evaluate data analysis methodologies.
And what can do you do as a scientist in your own lab to ensure that your results can be reproduced and utilized in future studies?
- Utilize quality control checks to ensure good quality samples (remember, junk in = junk out)
- Include biological replicates (for statistics, see below)
- Use vetted approaches: what does the literature in this field recommend?
- Document your methodologies: include all the details (well, sort of)
- Make use of quality statistics during your analysis: p-values, coefficient of variation, t-tests, etc.
- Partner with an independent service provider to ensure quality, non-biased results
If you’re shopping around for a service provider for your next-generation DNA/RNA sequencing needs, be sure to critically evaluate what each service provider is offering, and decide what works best for you. Don’t hesitate to ask questions, and compare deliverables side-by-side like we do. And don’t worry, we provide you with a full set of Materials and Methods that you can incorporate into your publication or document in your R&D workflow.
Still overwhelmed? Contact our Project Scientist team, and we’ll walk you through the details.
Related articles:
Taking on the Problem of Reproducibility . CENews Article, Volume 92 Issue 3 p. 11, January 20, 2014.
PLOS ONE Launches Reproducibility Initiative. PLOS Blogs. August 14, 2012.
Reproducibility (Editorial) Science. Volume 343. Number 6168. Page 229. January 17, 2014
