- Block 12 — Regression Algorithms
- Objective
- establish a rigorous formulation of linear regression and distinguish population parameters from their estimators; derive the normal equations from the squared-loss objective by matrix calculus and solve them by hand; understand why the normal equations are the numerically wrong way to compute the answer and why QR and SVD are the right ones; state the Gauss-Markov theorem precisely and know what each of its assumptions buys; measure multicollinearity and its effect on coefficient magnitude and sign; compute standard errors, t-statistics, confidence intervals and p-values by hand and verify them against a reference implementation; understand why inference after model selection is invalid; know the limits of R² and adjusted R²; run the full residual diagnostic battery, including leverage, studentized residuals and Cook's distance; correct standard errors under heteroscedasticity; separate confidence intervals for the mean from prediction intervals; and recognize when linear regression is the right model and when it is not.
- Estimated duration
- 140 minutes
- Prerequisites
- chapters 006, 010, 016, 017, 021, 029, 030, 031, and 032
- Associated quizzes
044.1-quiz-model, 044.2-quiz-least-squares, 044.3-quiz-residuals, 044.4-quiz-assumptions, 044.5-quiz-evaluation, 044.6-quiz-limitations, 044.7-quiz-positioning