Research Foundations › Module 2: Asking a good question
Hypotheses and variables
Lesson 3 of 3 · 2 min
A variable is anything that can take different values: hours of sleep, model size, accuracy, age.
Three kinds you must name
- Independent variable. What you change or compare. Example: the learning rate.
- Dependent variable. What you measure to see the effect. Example: validation accuracy.
- Controlled variables. What you keep the same so it cannot explain the result. Example: the dataset, the model, the number of training steps, the random seed.
If two things change at once, you cannot tell which one caused the result. Changing one thing at a time is the most useful habit in experimental work.
Operational definitions
Words like "better", "faster" or "fair" mean nothing until you say how you will measure them. This is an operational definition.
| Vague | Operational |
|---|---|
| The model is better | Macro F1 score on the held-out test set is higher |
| Training is faster | Wall-clock time to reach 90% validation accuracy on the same GPU is lower |
| Students learn more | Score on the same 20-question test, taken one week later, is higher |
- The model is better
- Operational: Macro F1 score on the held-out test set is higher
- Training is faster
- Operational: Wall-clock time to reach 90% validation accuracy on the same GPU is lower
- Students learn more
- Operational: Score on the same 20-question test, taken one week later, is higher
The null hypothesis
In many studies you write two hypotheses:
- Null hypothesis (H0): there is no effect. "The learning rate does not change final accuracy."
- Alternative hypothesis (H1): there is an effect. "A smaller learning rate gives higher final accuracy."
Statistics then asks: if H0 were true, how surprising would my data be? You will learn this properly in a statistics course. For now, the key idea is that you try to reject "no effect", rather than set out to prove your idea right.
RememberBefore any experiment, write one line each: what I change, what I measure, what I keep the same.
ExampleA student compares two models, but trains one for 10 epochs and the other for 50. The second wins. Was it the model or the training time? Nobody can tell. Training time should have been a controlled variable.