Evaluation, Calibration and Explainability

Objective: evaluate the expected system, not just an average score.

Three levels of evaluation

1. Loss: signal optimized during training.
2. ML metrics: precision, recall, F1, ROC-AUC, MAE, etc.
3. System/business metrics: error cost, latency, abstention rate, impact.

The choice depends on the cost of false positives and false negatives. With a rare class, accuracy can be nearly perfect for a useless model.

Calibration

A model is calibrated if, among predictions announced at 80%, about 80% are correct. Softmax confidence is not automatically a reliable probability. Reliability diagram, Brier score and temperature calibration can help.

Robustness and subgroups

Measure by class, source, period and relevant group. Test noise, plausible variations, out-of-distribution data and realistic adversarial examples. Document uncertainty when samples are few.