Practice quiz: Fair comparisons in ML

4 questions. Pick an answer to see if it is right. You can try as many times as you like. These are for practice: the final test has different questions.

1. 1.A paper tunes its own method for 100 trials but uses default settings for baselines. The problem is...

   NothingAn unequal tuning budget tilts the comparisonToo many trialsDefaults are always best
2. 2.Henderson et al. (2018) showed that two groups of seeds of the same algorithm...

   always matchedcould look like significantly different algorithmscrashedneeded more GPUs
3. 3.Normalising features using the whole dataset before splitting causes...

   faster trainingdata leakageunderfittingnothing
4. 4.For medical scans, the safest way to split train and test is by...

   rowpatientfile sizerandom pixel