Objective: model an ordered sequence and understand recurrent memory.
A recurrent network reads a sequence element by element. Its hidden state summarizes the past:
h_t = φ(W_x x_t + W_h h_{t-1} + b)The same weights are reused at each step. This recurrence accepts variable lengths but makes temporal computation sequential and exposes it to vanishing gradients.
The LSTM adds a memory state and input, forget and output gates. The GRU combines some gates and has fewer parameters. These gates learn what information to keep, modify or forget.
from torch import nn
rnn = nn.GRU(input_size=64, hidden_size=128, num_layers=2,
batch_first=True, dropout=0.2, bidirectional=True)Preview — the rest of the lesson is for enrolled readers.
Your access is tied to your account, not to this link. Sign in with the same email you used in class: your course is waiting, no need to enter the code again.
Sign inNo account yet? Create oneThe first modules of the course are open to everyone. For the rest you have three options: buy this course once and for all, subscribe, or enter the code handed out in class.