Transfer Learning and Fine-tuning

Objective: adapt a pretrained model with little data without destroying its knowledge.

Transfer starts from a model trained on a large source and adapts it to a target task. The first visual or linguistic representations are often reusable.

Two regimes

1. Frozen extractor: replace the head and train only that.
2. Fine-tuning: unfreeze part or all of the network with a generally lower learning rate.

from torchvision.models import resnet18, ResNet18_Weights
from torch import nn

model = resnet18(weights=ResNet18_Weights.DEFAULT)
for p in model.parameters():
    p.requires_grad = False
model.fc = nn.Linear(model.fc.in_features, 5)

The optimizer must only receive the trainable parameters. After unfreezing, recreate or update the optimizer so it sees the new parameters.