Self-Supervised and Contrastive Learning

1. Where does the target come from in self-supervision?
2. What does the augmentation policy concretely define?
3. What is a contrastive false negative?
4. Why use a linear probe?
5. [E] What control proves the value of pretraining?

Answers. 1. From the data itself. 2. The invariances to learn. 3. Two semantically close examples treated as opposites. 4. To measure the separability of the frozen representation. 5. A random encoder at the same downstream budget.

Objective: learn representations without exhaustive human annotation.

Self-supervision builds a target from the data itself: predicting a masked token, the next portion of a signal, a transformation or the relation between two views. It does not mean "without supervision" in the mathematical sense: the target signal is created automatically.