{"slug": "attention-decides-where-to-look-values-decide-what-comes-back", "title": "Attention Decides Where to Look. Values Decide What Comes Back.", "summary": "A technical breakdown manually computes the attention mechanism inside GPT-2 for a four-word sentence, tracing every number from raw score to contextual embedding to demystify the arithmetic behind transformer attention.", "body_md": "Member-only story\n\n# Attention Decides Where to Look. Values Decide What Comes Back.\n\n*One token, four numbers, and the arithmetic that turns raw attention scores into a contextual embedding inside GPT-2.*\n\nArticle 1- [https://medium.com/p/5807f169db30](https://medium.com/p/5807f169db30)\n\n*Where we are headed. Four weights, one target. draw leans on I six times harder than on itself. The rest of this piece derives those numbers by hand.*\n\n“Attention” sounds like the hard part of a transformer. It is the part everyone name-drops and almost nobody computes on paper. Strip away the branding and what remains is arithmetic a spreadsheet could run: exponentiate a handful of numbers, divide by their sum, and take a weighted average.\n\nThe mechanics of that average are the easy part. The question worth sitting with is *what* gets averaged, and why the two ingredients — the weights and the values — are produced by different machinery inside the model. Get that split clear and the whole block stops being magic.\n\nSo we will not describe attention. We will run it. One four-word sentence, one target token, and every number followed from raw score to the embedding that leaves the block.", "url": "https://wpnews.pro/news/attention-decides-where-to-look-values-decide-what-comes-back", "canonical_source": "https://pub.towardsai.net/attention-decides-where-to-look-values-decide-what-comes-back-75df74bb5f07?source=rss----98111c9905da---4", "published_at": "2026-07-14 02:52:13+00:00", "updated_at": "2026-07-14 03:23:35.864297+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "neural-networks", "natural-language-processing"], "entities": ["GPT-2"], "alternates": {"html": "https://wpnews.pro/news/attention-decides-where-to-look-values-decide-what-comes-back", "markdown": "https://wpnews.pro/news/attention-decides-where-to-look-values-decide-what-comes-back.md", "text": "https://wpnews.pro/news/attention-decides-where-to-look-values-decide-what-comes-back.txt", "jsonld": "https://wpnews.pro/news/attention-decides-where-to-look-values-decide-what-comes-back.jsonld"}}