Lesson 3 of 8
Validation Data
Validation data is held out from training so you can test whether the model learned the behavior instead of memorizing examples.
Active reading
Train set
Examples used to update the model.
Validation set
Separate examples used to measure performance.
Leakage
When evaluation examples contaminate training.
Measure on examples the model did not train on
If the same examples appear in training and validation, the score can look good without proving the model generalizes.
Validation should reflect the real tasks and edge cases that matter after deployment.
Example
Hold out a clean set of support tickets before training, then compare the base model and tuned model on exactly that set.
Quick check
Explain this lesson back in one sentence before continuing. If you cannot, reread the example and key takeaway.
Common mistake
Do not tune repeatedly against the same validation set until you effectively overfit to it.
Key takeaway
Keep evaluation data separate so improvement means something.