From the Artificial Neuron to the Perceptron

Objective: formalize the elementary computation unit and its decision boundary.

A neuron first computes an affine combination, then an activation:

z = wᵀx + b
a = φ(z)

The weights w and the bias b are learned. The activation φ is chosen by the designer. The bias translates the boundary; without it, every linear boundary would pass through the origin.

The perceptron

The binary perceptron outputs 1 if wᵀx + b > 0, else 0. It can learn a linear separation, but not XOR. This limitation is not fixed by adding more examples: the function space of the model is insufficient.

No straight line separates the two classes. A non-linear hidden layer can build intermediate representations that then make the separation possible.

Parameter or hyperparameter?

• weights and biases: parameters, learned by the optimizer;
• number of neurons, activation, learning rate: hyperparameters;
• pre-activation and activation: values computed for a single example.

Careful analogy

The artificial neuron looks like a weighted grid: it adds up features with coefficients, adds a movable threshold, then transmits a signal. The biological analogy should not be taken literally.

Quick check

What is the role of the bias? Why does a simple perceptron fail on XOR? Are the weights set by hand?

Mastery activity — Boundary and XOR

With w=(2,-1) and b=-0.5, compute z then the class of the four points (0,0), (0,1), (1,0), (1,1). Then try to label XOR with a single straight line and geometrically explain the failure. Success: distinguishing a perceptron limitation from an optimization error.

Computation performed by a neuron

Reading: the weights orient the boundary, the bias shifts it and the activation enables a non-linear transformation. A single neuron only solves simple separations; it is the composition of neurons and the optimization of parameters that give the network its capacity. The trap is interpreting a single weight without accounting for the scale of the inputs and the downstream layers.