Compute a mini attention with a query, keys, normalized weights and a sum of values.
Proof of success: explain "Attention: queries, keys and values", complete "The Weighted Mix" and name the limit of the analogy.
In "The robot puts the cup away because it is empty", Amine looks for what "it" refers to. For the current token, the model builds a query; each token provides a key compared to that query and a value containing the information to mix. The similarities become weights via softmax. The output is a weighted sum of the values. A large attention signals a strong influence in this calculation, not a full explanation of the decision.
Preview — the rest of the lesson is for enrolled readers.
Your access is tied to your account, not to this link. Sign in with the same email you used in class: your course is waiting, no need to enter the code again.
Sign inNo account yet? Create oneThe first modules of the course are open to everyone. For the rest you have three options: buy this course once and for all, subscribe, or enter the code handed out in class.