- Block 15 — The classification metrics
- Objective
- complete the set of eight rates derivable from a binary
confusion matrix by defining the four that chapters 053 to 057 left aside —
FPR, FNR, NPV and FOR — together with the FDR that mirrors
precision; organize all eight by their denominator into two families of two
complementary pairs and establish that family membership predicts whether a
number travels between populations; measure that partition on a frozen
classifier across a prevalence sweep; derive NPV and FOR from Bayes'
theorem and verify the derivation against counted cells; expose the negative
predictive value of rare-disease screening as an artefact of rarity; establish
FPR as the ROC x-axis and separate it from the false alarm volume that
staffs a queue; distinguish the classification FDR from the one
Benjamini-Hochberg controls; work one screening instrument through three
prevalences with all eight rates at each; map each rate onto the stakeholder
whose question it answers; implement in scikit-learn the rates it does not
name; attach confidence intervals to all eight; and extend the set to the
multiclass case under one-versus-rest. - Estimated duration
- 90 minutes
- Prerequisites
- chapters 029, 030, 034, 050, 051, 052, 053, 054, 055
and 056
- Associated quizzes
058.1-quiz-fpr-and-the-roc-axis.md to
058.9-quiz-reporting-the-eight-rates.md