What an LLM Actually Does: Predicting the Next Word, Explained

A developer created an interactive predictor to demonstrate how large language models generate text by repeatedly predicting the next word. The tool illustrates that the core mechanism is a loop of prediction, selection, and appending, with temperature controlling randomness. This explains both the fluency and the hallucinations of AI chatbots.

"How does ChatGPT think ?" It doesn't. The entire mechanism behind every chatbot is almost anticlimactic: it predicts one next word , adds it, and repeats. I built a tiny interactive predictor so you can be the model — and it explains both the magic and the flaws. 🔮 Be the model: https://dev48v.infy.uk/ai/days/day6-next-token.html https://dev48v.infy.uk/ai/days/day6-next-token.html This is Day 6 of AIFromZero — AI literacy, one concept a day, no code to follow. Given everything so far, the model outputs a probability for every possible next word, picks one, appends it, and runs again with the longer text. Paragraphs, code, poems — all of it is this one step on repeat. "the cat sat on the " → P mat high, P bird low The output isn't one word — it's a number for every word it knows 100,000+ for a real model . Most are near zero; a handful are plausible. The bars in the demo are that distribution, over a tiny vocabulary. After picking a word, it becomes part of the input for the next prediction. Predict → append → predict again. Because each new word conditions on all the previous ones, short local choices add up to coherent long text. Once you have probabilities, how do you choose? Temperature reshapes them before sampling: p = p 1 / temperature ; // then renormalise and sample Drag the slider in the demo and watch the bars sharpen or even out. That one knob is what an API calls "creativity." In my toy, from counting which word followed which in a few sentences a "bigram" with 1-word memory . A real LLM replaces the counting with a giant neural network trained on much of the internet, and its memory spans thousands of words. The mechanism is identical — only the quality of the guess changes. "Just predicting the next word" explains the fluency it has seen how language flows AND the hallucinations: a plausible-sounding next word isn't always a true one. It optimises for likely , not correct . That gap is where made-up facts live — and it's tomorrow's topic. Predict next word → append → repeat; temperature tunes the daring. Understand this loop and "the AI thinks…" stops being mysterious and starts being mechanical. Try being the model https://dev48v.infy.uk/ai/days/day6-next-token.html — click words and watch a sentence build.