- Block 11 — Classification algorithms
- Objective
- state the recursive partitioning algorithm precisely; compute
Gini impurity, Shannon entropy, information gain and the regression criteria by
hand and verify every figure against a fitted
DecisionTreeClassifier; explain
why CART is greedy, why the globally optimal tree is intractable, and measure
what the greed costs; establish how continuous features are split and what
scikit-learn's lack of native categorical handling forces you to do; control
growth with the pre-pruning hyperparameters and with a measured cost-complexity
path; demonstrate the invariance of trees to monotone transformations, their
structural instability, their inability to extrapolate, their axis-aligned
boundaries and the bias of impurity-based importance; and state exactly what a
tree's interpretability licenses. - Estimated duration
- 75 minutes
- Prerequisites
- chapters 009 (parameters and hyperparameters), 021
(outliers), 022 (categorical encoding), 023 (feature scaling), 025 (feature
selection), 026 (train/validation/test split), 029 (fit, predict,
predict_proba), 030 (error, loss and metric), 034 (cross-validation), 035
(hyperparameter tuning)
- Associated quizzes
037.1-quiz-tree-structure.md to
037.7-quiz-strengths-and-limits.md