Positive and negative controls

Lab scientists use two special controls to check that the experiment itself works, before trusting its answer.

• A negative control should show no effect. If it does, something is wrong: contamination, a leak, a bug.
• A positive control should show a known effect. If it doesn't, your method can't detect effects at all.

ExampleA COVID test kit includes a sample that should test negative and one that should test positive. If either gives the wrong answer, you don't trust any of the day's results.

In ML and data science

• Negative control: train on shuffled labels. Accuracy should fall to chance on held-out data. If it doesn't, the test set is leaking.
• Positive control: plant a signal you know is there (add a feature that perfectly predicts the label) and check your pipeline finds it.
• Simple baselines like "always predict the most common class" give a floor that any real result must clear.

A famous 2017 study by Zhang and colleagues showed that large neural networks can perfectly memorise random labels on their training set. So high training accuracy alone proves nothing; held-out performance on real labels is what counts.

RememberBefore trusting a result, check that your setup shows nothing when nothing is there, and something when something is.