The Three Learning Paradigms

Learning signal — rigorous definition

Information used by an algorithm to assess the quality of its current hypothesis and to derive from that assessment a direction in which to correct its parameters. It takes one of three forms: a target value attached to each observation, an internal consistency criterion defined on the data alone, or a scalar reward emitted by an environment in response to an action.

Learning paradigm — rigorous definition

A class of problems defined by the nature of the available signal and, consequently, by the form of the optimization problem posed: minimization of an empirical risk over labeled pairs, optimization of an internal structural criterion, or maximization of an expected cumulative reward.

In plain terms

The signal is what lets the algorithm know it is wrong. The paradigm is the family of problems defined by what you have to learn from.

Point of caution

With no signal at all there is no learning in the sense of Mitchell (1997): absent a performance measure P, no experience E can improve anything. An algorithm with nothing to be corrected against is not learning, it is computing.

Rigorous definition

Let X be an input space and Y an output space. We are given a training sample of n pairs

S = { (x_1, y_1), ..., (x_n, y_n) },  x_i ∈ X,  y_i ∈ Y

assumed drawn independently and identically distributed from an unknown, fixed joint distribution P(X, Y). Supervised learning consists in selecting, from a hypothesis class H, a function f: X → Y that minimizes the expected risk R(f) = E[ L(f(X), Y) ], where L is a loss function measuring the discrepancy between the predicted and the observed value. Since P is unknown, the algorithm in practice minimizes the empirical risk computed on S, subject to regularization.

In plain terms

You supply examples whose correct answer is known, and you ask the algorithm to build the function that best reproduces that correspondence, so it can be applied to cases whose answer is unknown.

On the word "supervised"

It refers to the presence of a supervisor — an operator, a process, or a later observation — that provided the correct value for each example. The supervisor acts when the dataset is assembled, never during training. By the time the algorithm runs, the supervisor is gone and only its record remains.

Point of caution

The fixed-distribution assumption is load-bearing. When P(X, Y) shifts between training and production, the empirical risk stops estimating the expected risk and performance degrades. That phenomenon is covered in chapter 082, 082.0-monitoring-and-drift.md.

Rigorous definition

An observation for which the value of the target variable is recorded alongside the explanatory variables, that value being defined unambiguously and applied consistently across the whole dataset.

In plain terms

A row of the table whose answer is already known.

Where labels come from

Delayed observation (the loan was repaid or it was not), human annotation (a radiologist reads a scan), system records (the customer canceled), or physical measurement (the part failed at 412 hours).

Point of caution

The cost of obtaining labels is frequently the limiting factor of a supervised project. Explanatory variables are abundant; labels rarely are. A project plan that budgets for feature engineering but not for labeling is a project plan with a hole in it.

Rigorous definition

A supervised learning problem in which the output space Y is a finite set of classes { $c_1$, ..., $c_k$ }. The model generally estimates the conditional probability P(Y = $c_j$ | X = x); the predicted class results from applying a decision rule to that probability vector.

In plain terms

Place each observation into one of the categories defined in advance.

Sub-types: binary (2 exclusive classes), multiclass (k exclusive classes), multi-label (k classes that may hold simultaneously), ordinal (k ordered classes, for example a risk grade from A to E).

Point of caution

The classes are defined before training and are part of the specification of the problem. A classification model cannot produce a class it has never seen. If a new class appears in production, no threshold tuning will make the model report it; the specification has to be reopened.

Rigorous definition

A supervised learning problem in which the output space Y is a subset of the reals. The model estimates a regression function, most often the conditional expectation E[Y | X = x], by minimizing a squared or absolute loss.

In plain terms

Estimate a number rather than a category.

Terminological warning

In modern usage the word "regression" carries no sense of going backward or falling off. It is inherited from Francis Galton's work on the inheritance of human stature (1886), which described a regression toward the mean: the children of unusually tall parents tended, on average, to be less tall than their parents. The word named a particular statistical phenomenon. It was then extended to the fitting method used to demonstrate that phenomenon, and finally to any prediction of a numeric quantity. No inference should therefore be drawn from the word itself: "regression" means only that the target is a number.

A confusion not to make

Logistic regression is a classification algorithm, despite its name. It regresses the log-odds on the explanatory variables and outputs a probability of class membership. It is covered in chapter 036, 036.0-logistic-regression.md.

A student prepares for an examination using past papers. Each paper comes with its worked solution. The student attempts the question, compares the attempt to the solution, identifies the gap, and adjusts the method. After enough exercises, the student sits an unseen paper whose solution does not yet exist.

Extensions

A student who memorizes the solutions without acquiring the method aces the past papers and fails the unseen one: that is overfitting, covered in chapter 031, 031.0-overfitting-and-underfitting.md. A student who has the solution sheet during the exam obtains an excellent mark that measures nothing: that is data leakage, covered in chapter 028, 028.0-data-leakage.md.

Limits of the analogy

The student understands what they are doing and can transfer the method to a neighboring subject. The model establishes a statistical correspondence only, and does not transfer outside the domain covered by its training data.

Rigorous definition

A class of problems in which one is given a sample { $x_1$, ..., $x_n$ }, $x_i$ ∈ X, with no associated target variable. The objective is to estimate a structure of the distribution P(X): a partition of the observation space, a low-dimensional manifold approximating the point cloud, a density, or regularities of co-occurrence. Optimization is carried out on a criterion defined exclusively from the data, with no reference to an expected value.

In plain terms

You supply observations with no answers attached and ask the algorithm to bring out the internal organization of the data.

On the phrase "unsupervised"

It signals the absence of a reference answer, not the absence of human involvement. The choice of variables, of distance, of the number of groups and of the scaling belongs entirely to the analyst.

Point of caution

An unsupervised algorithm always returns a result, including when the data contain no structure whatsoever. Ask a partitioning method for four groups on a perfectly uniform cloud and you will get four groups. The production of a result is therefore never evidence that a structure exists.

Rigorous definition

The problem of building a partition or a covering of a set of observations into groups such that a within-group similarity measure is maximal and a between-group similarity measure is minimal, in the sense of a distance specified by the analyst. An inertia-based partitioning method minimizes the within-group sum of squares:

argmin  Σ      Σ      || x - μ_j ||²
   C   j=1..k  x ∈ C_j

where $\mu_j$ denotes the center of group $C_j$.

In plain terms

Group together the observations that resemble one another, without knowing in advance which groups exist or how many there are.

Parameters that belong to the analyst

Number of groups, distance measure, scaling, choice of variables. Each of these changes the result; there is no "true" partition that an algorithm would discover independently of them.

Point of caution

The groups produced have no names. Naming and interpreting them is a business act performed after the algorithm has run — never an output of the algorithm.

Rigorous definition

The problem of constructing a map g: X → Z, with dim(Z) < dim(X), that best preserves some property of the original distribution: explained variance for principal component analysis, local neighborhood structure for non-linear embedding methods, reconstruction capacity for autoencoders. The optimized criterion is internal to the data.

In plain terms

Represent the same observations with fewer variables, losing as little information as possible.

Operational purposes: visualization, compression, denoising, preprocessing for a supervised model, mitigation of the curse of dimensionality.

Point of caution

Used upstream of a supervised model, dimensionality reduction must be fitted on the training data alone and then applied unchanged to validation and test data. Fitting it on the full dataset is data leakage, covered in chapter 028, 028.0-data-leakage.md.

Rigorous definition

A learning framework in which an agent interacts sequentially with an environment. At each time step t the agent observes a state $s_t$, selects an action $a_t$ according to a policy π, receives a scalar reward $r_{t+1}$ and observes a new state $s_{t+1}$. The problem is most often formalized as a Markov decision process (S, A, P, R, γ), with γ ∈ [0, 1[ the discount factor. The objective is to find a policy π maximizing the expected discounted cumulative return:

J(π) = E[ Σ  γ^t · r_(t+1) ]
          t≥0

Standard reference: Sutton and Barto, Reinforcement Learning: An Introduction (1998, second edition 2018).

In plain terms

The agent does not learn from correct answers supplied in advance but by successive attempts: it acts, the environment penalizes or rewards it, and it adjusts its strategy so as to accumulate as much reward as possible.

What separates a reward from a label

A label states the correct answer, exists before training, and concerns a single isolated observation. A reward evaluates the answer produced, is generated during interaction, bears on a sequence of actions, and depends on the model's own actions.

Point of caution

The delayed character of the reward creates the credit assignment problem: when a reward arrives at the end of hundreds of actions, determining which of them contributed is the central difficulty of the paradigm.

Learning to ride a bicycle does not proceed from a solution sheet. Nobody can supply, for each millisecond, the exact handlebar angle and torso lean. The instruction "keep your balance" is not a label. The available signal is a consequence: the learner stays upright, or falls. And the link between the wrong movement and its consequence is delayed by a few seconds.

What the analogy makes visible

The skill is acquired through interaction, not by memorizing question-answer pairs. It cannot be transmitted by a lecture, and acquiring it requires a large number of attempts, unsuccessful ones included. A child falls a few dozen times; an artificial agent commonly needs millions of episodes. When each episode has a material or human cost, training under real conditions becomes impractical — hence the requirement for a simulator.

Rigorous definition

A class of problems in which the sample combines a labeled subset $S_l$ = { ($x_i$, $y_i$) } of size l and an unlabeled subset $S_u$ = { $x_j$ } of size u, generally with u >> l. The methods exploit the marginal distribution P(X) estimated on $S_u$ to constrain the estimation of P(Y | X) learned on $S_l$, under regularity assumptions: smoothness, cluster, manifold.

In plain terms

You have many observations and few known answers; the unlabeled mass is used to get more out of the small number of labels available.

Point of caution

The regularity assumptions are not always satisfied. When they are not, adding unlabeled data can degrade performance relative to a model trained on the labeled data alone. Semi-supervised learning is not a free improvement; it is a bet on the geometry of the data.

Rigorous definition

A class of methods in which the target variable is constructed automatically from the internal structure of the data, with no annotator involved. A so-called pretext task is defined so that its label is deducible from the raw data; the model is trained on that task by ordinary empirical risk minimization, and the resulting representation is then reused for the target task.

In plain terms

The algorithm manufactures its own exercises and its own solution sheets from raw data, which removes the annotation cost.

A decisive clarification

The learning mechanics remain strictly supervised: there is a target value, a loss function, a comparison between prediction and target. Only the provenance of the label changes — extracted from the data instead of supplied by an operator. Self-supervised learning is therefore not a fourth paradigm but a way of obtaining labels.

Point of caution

The quality of the representation depends entirely on the relevance of the pretext task. A task solvable through a superficial artifact yields a representation of no value for the target task.

Rigorous definition

The problem of identifying observations whose generation is improbable under the distribution of the observations considered normal. Depending on the availability of labeled anomaly examples, it is formulated as a strongly imbalanced supervised classification, as an unsupervised density or support estimation, or as one-class classification.

In plain terms

Spot what falls outside the ordinary, whether or not you have examples of abnormal cases.

Taxonomic position

Anomaly detection straddles two paradigms. It is not an additional family: it names a business objective that can be addressed within either paradigm depending on the data available.

A marketing team asks for a model predicting "high-potential customers". No variable of that name exists anywhere in the information system. The intention to predict is not enough to create a supervised problem. Two ways out exist, and exactly one must be chosen explicitly: define a target that is measurable and verifiable in the history — for example "revenue over the next 12 months above $5,000" — or accept an exploratory unsupervised approach producing groups to be interpreted.

Correct formulation : "A supervised problem requires a target variable that is defined, measurable and present in the history. Until that variable exists, there is no supervised problem."

Both produce groups, which is enough to sustain the confusion. The discriminating criterion is not the output but the input: do the classes pre-exist in the training data? One consequence shows up regularly in practice: a clustering presented as a predictive model, complete with a claimed accuracy figure — a figure that is meaningless in the absence of a reference partition.

Correct formulation : "Classification assigns an observation to classes defined beforehand and learned from labeled examples; clustering forms groups from similarities alone, with no pre-existing classes and no measurable accuracy."

The absence of labels lightens data collection and gives the impression of a less demanding problem. On validation, the opposite is true: with no ground truth, the project has no objective stopping criterion, and the number of groups, the distance and the scaling are choices that no metric settles. A supervised project ends with a performance figure on a test set; an unsupervised project ends with a business acceptance decision.

Correct formulation : "Unsupervised learning is less demanding in labeled data and more demanding in interpretation: its difficulty moves from collection to validation."

The list "supervised, unsupervised, reinforcement, Deep Learning" mixes two planes of classification. The first three terms name paradigms, defined by the nature of the learning signal; the fourth names a class of models, defined by an architecture. A neural network trained on labeled images is supervised; an autoencoder is unsupervised; a network optimizing a policy from rewards is reinforcement.

Correct formulation : "Deep Learning is a class of models usable within all three paradigms; it sits on a different plane of classification."

To predict churn at 90 days, a team retains as an explanatory variable the number of calls to the cancellation desk. Performance is excellent in validation and worthless in production: at the moment the prediction has to be issued, that call has not yet taken place. The variable is not predictive of the event, it is constitutive of it. This form of data leakage, covered in chapter 028, 028.0-data-leakage.md, is not detectable through metrics — it improves them.

Control rule : for every explanatory variable, verify that its value would genuinely be available at the instant the model is queried in production.

Correct formulation : "The label postdates the explanatory variables; any information contemporaneous with or subsequent to the realization of the target must be excluded from the training set."

"Healthcare is supervised" and "marketing is clustering" are statements with no foundation. Healthcare gives rise to readmission prediction (supervised), to the identification of patient subgroups (unsupervised) and to sequential adaptation of a dosing protocol (reinforcement). Marketing gives rise to churn prediction, to segmentation, and to dynamic offer allocation through bandits.

Correct formulation : "The paradigm is determined by the nature of the available learning signal, never by the industry sector."