- Block 7 — Train, Validation and Test
- Objective
- define data leakage rigorously; separate its two disjoint
branches — target leakage, where a feature encodes information about the label
that will not exist when the prediction has to be made, and protocol leakage,
where information flows from the evaluation data into training through the
procedure itself; recognize the eighteen concrete forms these two branches
take; measure, on executed code, the size of the illusion each one produces and
the honest figure it replaces; run a systematic audit; and build a learning
architecture that eliminates whole classes of leakage mechanically rather than
by vigilance.
- Estimated duration
- 90 minutes
- Prerequisites
- chapters 019 (imputation), 022 (categorical encoding),
023 (feature scaling), 024 (feature engineering), 025 (feature selection),
026 (train/validation/test) and 027 (stratification, groups, time)
- Associated quizzes
028.1-quiz-definition-and-severity.md to
028.7-quiz-structural-protection.md