Split a deck of cards into three sets and prevent any leak from the test into the model's choices.
Proof of success: explain "Training, validation and test", complete "The Three Envelopes" and name the limit of the analogy.
For a paper-rocket contest, Sacha has 60 labeled drawings. Forty are used to tune the parameters, ten to choose the size of the network, and ten stay in a sealed envelope. When the first test result is disappointing, someone suggests changing the model and then looking at the same envelope again. Sacha refuses: by tuning to the test over and over, it would become a hidden validation and would no longer measure an honest surprise.
Preview — the rest of the lesson is for enrolled readers.
Your access is tied to your account, not to this link. Sign in with the same email you used in class: your course is waiting, no need to enter the code again.
Sign inNo account yet? Create oneThe first modules of the course are open to everyone. For the rest you have three options: buy this course once and for all, subscribe, or enter the code handed out in class.