Correlation is not causation

Two things are correlated when they rise and fall together. Correlation is easy to find. Cause is much harder to show.

Three reasons two things move together

1. A causes B. More study time leads to better marks.
2. B causes A (reverse causation). People who are already fit enjoy exercise more, so "exercise enjoyment" and "fitness" are linked, but fitness may come first.
3. Something else causes both (a confounder). Ice-cream sales and drowning both rise in summer. Ice cream does not cause drowning: hot weather drives both.

There is also a fourth reason: chance. If you test enough pairs of variables, some will look linked purely by luck. Tyler Vigen's "Spurious Correlations" project shows many real datasets that match closely by coincidence, like the number of films an actor appeared in and unrelated national statistics.

The same trap in machine learning

Models learn correlations, not causes. A well-known study by Zech and colleagues (2018) found that a model for detecting pneumonia from chest X-rays could also identify which hospital an X-ray came from. Because hospitals had different rates of pneumonia, the model could use hospital-specific signs instead of signs of disease, and it did worse on X-rays from new hospitals.

How to be careful

• Ask: could the arrow point the other way?
• Ask: what third factor could drive both?
• Prefer experiments when you want to claim cause.
• Test on data from a different source than your training data.

RememberWhen you see "X linked to Y", always ask what else could explain it before you believe that X causes Y.