Learn

Designing Experiments

Design experiments that give answers you can trust: controls, randomisation, blinding, fair comparisons, ablations, A/B tests, natural experiments and a written plan before you start.

18 video lessons · about 32 min · free certificate

Your progress is saved in this browser. Sign in when you start the project or quiz.

  1. Watch · module 1Why design matters3 lessons
  2. Watch · module 2Variables and bias3 lessons
  3. Watch · module 3Fair comparisons in ML3 lessons
  4. Watch · module 4Controls and checks3 lessons
  5. Watch · module 5Online and field experiments3 lessons
  6. Watch · module 6Planning3 lessons

What you will learn

  • Explain why controls and randomisation make results trustworthy
  • Spot confounders, and use blinding to remove hidden bias
  • Compare ML methods fairly: equal tuning, many seeds, no leakage
  • Design ablations, positive and negative controls, and sanity checks
  • Run A/B tests without peeking, and find natural experiments when you can't randomise
  • Write a pre-registered test plan and run a pilot first

Who it is for: Students about to run a study, a user test or an ML comparison. Statistics for Researchers helps but is not required.