Control and comparison

"After taking the new supplement, 80% of students felt more focused." Impressive? You can't tell, because there is nothing to compare with. Maybe 80% would feel more focused anyway, after a good night's sleep or as exams got closer.

The control group

A control group is treated exactly like the treatment group, except for the one thing you are testing. It shows you what would have happened anyway.

An early example

In 1747, the ship's surgeon James Lind took 12 sailors with scurvy, kept them on the same diet, and gave pairs of them different extra treatments: cider, vinegar, sea water, citrus fruit and others. The pair given oranges and lemons recovered fast. Because the sailors were otherwise treated the same, the comparison pointed clearly at citrus.

What a control does

• It absorbs time effects: people get better on their own, seasons change, systems warm up.
• It absorbs regression to the mean and placebo effects.
• It turns "something happened" into "this made a difference".

In ML and engineering

The control is your baseline: the same data, the same budget, without your change. "Accuracy rose to 91%" means nothing without "compared with 88% for the same model without our method".

RememberWithout a comparison, there is no result. Give every experiment a control that differs in only one thing.