Reading and Reviewing Papers › Module 3: Judging a paper

Peer review and preprints

Lesson 1 of 3 · 2 min

Previous

How peer review works

  1. Authors submit a paper to a journal or conference.
  2. An editor (or area chair) checks it is in scope, then sends it to usually two to four experts: the reviewers.
  3. Reviewers read it and write reports: strengths, weaknesses, questions, and a recommendation.
  4. The editor decides: accept, accept with minor or major changes, or reject.
  5. If changes are asked for, the authors revise and reply to each point.

In single-blind review the reviewers know who the authors are, but not the other way round. In double-blind review neither side knows. Many AI conferences use double-blind review, and some, like ICLR, publish the reviews openly on OpenReview.

What peer review does, and does not, do

Peer review filters out many errors and weak claims, and it often makes papers clearer. It is not a guarantee that a paper is correct. Reviewers usually do not re-run experiments, and they can miss mistakes or fraud. Some peer-reviewed papers are later corrected or retracted.

Preprints

Preprints let results be shared quickly, which is why much AI research appears on arXiv first. The trade-off is that nobody has formally checked them. Read preprints with extra care, look for a later peer-reviewed version, and say "preprint" when you cite one.

Checked by experts
Peer-reviewed paper: Yes, before publication
Preprint: No formal review
Speed
Peer-reviewed paper: Months
Preprint: Days
Can still be wrong
Peer-reviewed paper: Yes
Preprint: Yes, more often

RememberPeer review lowers the risk. It does not remove it. Your own critical reading is always needed.