Research Foundations › Module 3: Types of research
Choosing a method
Lesson 3 of 3 · 2 min
The question chooses the method. Here is a simple guide.
| If your question is... | A good method is... |
|---|---|
| Does changing X improve Y? | A controlled experiment, changing only X |
| How common is something? | A survey or a measurement of a well-chosen sample |
| Why do people do something? | Interviews or open-ended surveys (qualitative) |
| Is X linked to Y in the real world? | An observational study, with confounders named and handled |
| Which of these methods works best on a task? | A benchmark comparison with fair baselines |
| What would happen in a system we cannot test directly? | A simulation, with its assumptions stated |
- Does changing X improve Y?
- A good method is...: A controlled experiment, changing only X
- How common is something?
- A good method is...: A survey or a measurement of a well-chosen sample
- Why do people do something?
- A good method is...: Interviews or open-ended surveys (qualitative)
- Is X linked to Y in the real world?
- A good method is...: An observational study, with confounders named and handled
- Which of these methods works best on a task?
- A good method is...: A benchmark comparison with fair baselines
- What would happen in a system we cannot test directly?
- A good method is...: A simulation, with its assumptions stated
Computational research
In AI and computing, much research is computational: you run experiments on data and models instead of on people. The same rules apply. You still need a clear question, controlled variables, a fair comparison and honest reporting. A benchmark result is an experiment, and it can be done badly: an untuned baseline, a test set that leaked into training, or a single lucky run. You will learn about these in a later course.
Ask three questions before you choose
- What would convince a sceptical reader? If the answer is "a fair comparison", you need an experiment. If it is "hearing from the people affected", you need qualitative work.
- What can I actually do? Time, data, access, ethics approval.
- What has worked for similar questions? Read the method sections of papers that asked something close to your question.
RememberWrite one sentence: "I will answer my question by ..., because ...". If you cannot finish the sentence, the method is not chosen yet.