Variables and confounders

Every experiment has three kinds of variables:

• Independent variable: what you change on purpose (the treatment, the method, the learning rate).
• Dependent variable: what you measure (recovery, test score, accuracy).
• Controlled variables: what you hold fixed so they can't explain the result (dataset, hardware, time of day).

The confounder

A confounder is a third factor that affects both the treatment and the outcome, creating a link that isn't causal.

ExampleStudies once found that coffee drinkers had more lung cancer. But coffee drinkers were also more likely to smoke. Smoking was the confounder: it drove both the coffee and the cancer.

How design removes confounders

• Randomisation spreads confounders evenly between groups, including ones you never thought of.
• Holding things fixed (same dataset, same hardware, same budget) removes known ones.
• Measuring likely confounders lets you check balance and adjust.

In ML

A new model trained on more data, for longer, on better hardware, is compared with an old model. Is the gain from the model, or from the data, time and hardware? Those are confounders. Change one thing at a time.

RememberKnow what you change, what you measure and what you hold fixed. Every uncontrolled difference is a possible explanation.