Reproducible Research › Module 1: The problem

The reproducibility crisis in numbers

Lesson 1 of 3 · 2 min

In 2016, the journal Nature surveyed about 1,500 researchers. The results became famous:

  • More than 70% had tried and failed to reproduce another scientist's experiments.
  • More than 50% had failed to reproduce their own.
  • About 52% said there was a "significant crisis"; another 38% said a "slight crisis".

Replication projects

In 2015, the Open Science Collaboration repeated 100 published psychology studies. 97% of the originals had reported significant results; only about 36% of the repeats did. Similar projects in cancer biology and economics found many results were hard to repeat, often because key details, data or code were missing.

Why it happens

  • Small samples, flexible analyses and publication pressure (you met these in Statistics for Researchers).
  • Missing materials: data not shared, code lost, steps done by hand and never written down.
  • Software drift: the same code gives different results with different library versions.

The good news

The second group of causes is largely fixable by habits: organised projects, version control, recorded environments and shared code. That is what this course teaches.

RememberMuch irreproducible research isn't fraud; it's lost files, unrecorded steps and changed software. Those you can fix.