Activations, Logits and the Final Choice

Audience

ages 10–15

Duration

30 to 40 min

Adventure

2/8

Badge to earn

Score Interpreter

Today's mission

Tell apart raw score, hidden activation, probability and chosen class.

Proof of success: explain "Activations, logits and the final choice", complete "From Score to Shares" and name the limit of the analogy.

The story

At the pattern tournament, Zoé gets three numbers: 2.1; −0.4; 0.8. Those are logits, not probabilities: they are neither restricted to 0–1 nor forced to add to 1. Softmax turns them into positive shares that sum to 1. The category with the highest score can be picked, but the team also keeps the uncertainty and an abstention threshold for cases that are too close.

What is technically true

A convincing answer links an input to a measured output. Keep these three facts in view. Here, your anchor is "Activations, logits and the final choice".

• An activation like ReLU introduces a non-linearity by keeping max(0, z); without non-linearity, several dense layers collapse into a single linear transformation.
• Logits are raw scores; softmax normalizes them together for classification with mutually exclusive categories.
• The largest softmax value is a normalized score, not a verified success frequency; calibration compares these scores to the observed frequencies on separate data.

The vertical map

The path separates activation, logits, normalization and decision rule instead of treating softmax like certainty. Read the diagram from top to bottom: each arrow announces a transformation or a check, never a thought inside the machine.

A useful analogy — and where it breaks

The analogy: Logits are like the raw points from a jury, and softmax then converts them into comparable shares.

Where it breaks: A true probability requires more than a math conversion: if the model is poorly calibrated or out of context, a 90% share can be misleading. An analogy helps you get started; it never replaces the data, the calculations or a test.

Screen-free activity — From Score to Shares

Materials: three logit cards 2, 1 and 0, six tokens, a ReLU rule and a simplified softmax table
Suggested time: 10 to 20 minutes.

1. Rank the three logits without calling them probabilities.
2. Apply ReLU to a second series that includes negative values.
3. Use the table to turn 2, 1 and 0 into approximate shares 0.67, 0.24 and 0.09.
4. Add an abstention rule if the two largest shares are less than 0.10 apart.

What you should notice: score, activation, probability and decision are four different objects that must not be confused.

Optional challenge

Add the same constant to all three logits. Check that the softmax proportions don't change, then explain this invariance.

Safety, fairness and human choice

Forcing a choice can hide uncertainty. In a sensitive situation, allow "not enough to decide" and hand the case to a qualified person.

Responsible question: Who sets the abstention threshold, and which errors should be reviewed before choosing it?

Mini-quiz with answers

1. Must a logit be between 0 and 1?

   Answer: No. It is a raw score that can be negative or greater than 1.

2. Why a non-linear activation between the layers?

   Answer: It lets us represent relationships that a purely linear stack cannot create.

3. Does a softmax output of 0.9 guarantee the right class?

   Answer: No. Calibration and context must be checked on separate data.

Competency badge

Score Interpreter — You earn this badge if you can explain the diagram without reading, show the result of the activity and kindly correct a wrong answer.