Attention Decides Where to Look. Values Decide What Comes Back.
A technical walkthrough demonstrates the arithmetic behind attention in GPT-2 by computing one token's contextual embedding from raw scores, showing that attention weights and values are produced by s…