Reproducible Research › Module 1: The problem
Reproducible vs replicable
Lesson 2 of 3 · 2 min
People use these words in different ways. A 2019 report from the US National Academies of Sciences, Engineering, and Medicine gave clear definitions that many now follow:
- Reproducible: using the same data and the same code, you get the same results.
- Replicable: running a new study, with new data, you get consistent results.
Why the difference matters
| Same data | New data | |
|---|---|---|
| Same analysis | Reproducibility | Replicability |
| Different analysis | Robustness | Generalisability |
- Same analysis
- Same data: Reproducibility
- New data: Replicability
- Different analysis
- Same data: Robustness
- New data: Generalisability
Reproducibility is the minimum. If even the original authors can't get the same numbers from their own data and code, nothing else can be checked. Replicability is the deeper test: does the finding hold in the world, not just in one dataset?
A computational check anyone can do
For any ML or data paper with released code: download it, run it, compare the numbers. If they match, the work is reproducible. If you then collect new data and the effect holds, it has replicated.
Your goal in this course
Make every project reproducible by default, so that others can spend their effort on replication and extension, not on guessing what you did.
RememberReproducible: same data, same code, same result. Replicable: new data, same finding. Reproducible comes first.