- Block 15 — The classification metrics
- Objective
- define F1 as the harmonic mean of precision and recall, derive
the equivalence of
2PR/(P+R) and 2TP/(2TP+FP+FN), and establish why the
harmonic mean is the right combination — it is bounded above by twice the
smaller term; generalize to F_beta as a weighted harmonic mean, give beta
its two exact interpretations, and show the optimal threshold moving with it;
derive the beta that makes the F-beta optimum coincide with the cost optimum,
measure the residual gap in dollars, and state that direct expected-cost
minimization is at least as good; exhibit three confusion matrices that F1
judges identical and MCC does not; show that F1 is not symmetric under a class
swap while accuracy and MCC are; define macro, micro, weighted and samples
averaging and prove that micro-F1 equals accuracy in single-label multiclass;
measure the range of F1 reachable from one model by threshold alone, and the
optimism of tuning that threshold on the reporting data; bootstrap the F1 of a
model evaluated on 25 positives; and close with a decision table separating the
cases for F1, for F-beta, and for neither. - Estimated duration
- 95 minutes
- Prerequisites
- chapters 011, 029, 030, 034, 050, 051, 052, 053, 054, 055
- Associated quizzes
059.1-quiz-definition-and-equivalence.md to
059.10-quiz-reporting-f1.md