Lesson 7 of 8
Overfitting and Checkpoints
Training loss can improve while real task quality gets worse, so checkpoints must be judged on held-out evaluation data.
Active reading
Understand the idea
Save checkpoints during training and compare them on the same validation set. Watch for memorization, style collapse and regressions on important baseline behavior.
Example
Checkpoint 200, 400 and 600 all run on the same 100 validation examples; choose the best measured result, not the latest checkpoint.
Practice
Apply this lesson to one real task you already have, then write down how you would verify the result.
Common mistake
Selecting the final training step only because it has the lowest training loss.
Key takeaway
The best checkpoint is the one that performs best on your target evaluation, not the one trained longest.