AI Foundations 3 - Tokens and Pricing

Large language models process text as tokens rather than words, and providers charge separately for input and output tokens, usually in price-per-million units, with output tokens almost always costing more per token than input tokens. The explainer notes that input tokens cover everything sent to the model — the prompt, attached files, and prior conversation — while output tokens are what the model generates back, and that streaming tokens one at a time does not make a model faster overall but makes the wait feel shorter. It recommends keeping pasted context to what the task needs, reusing a stable prompt prefix where the provider supports caching it, and explicitly requesting short answers to hold down cost and latency.

AI Foundations 3 - Tokens and Pricing Models don’t read words the way you do. They break text into chunks called tokens, and a token isn’t the same thing as a word. “Cat” might be one token. “Unbelievable” might split into two or three. Code is chunked the same way — brackets, keywords, and indentation all count. Why tokens matter Every token costs something, and every token takes time to produce. A short prompt on a small file runs fast and cheap. Paste in a ten-thousand-line log file and ask for a summary, and you’ll feel both the wait and the bill. Input and output tokens Input tokens are everything you send in: your prompt, any files, the conversation so far. Output tokens are what the model generates back. They’re counted and priced separately, and they behave differently — one you control directly, the other you only shape indirectly by asking for shorter or longer answers. Pricing Providers charge per token, usually in price-per-million units, and output tokens almost always cost more per token than input tokens. Makes sense — generating text is the harder, slower half of the job. Streaming Rather than making you wait for the entire response, most models can stream tokens out one at a time as they’re generated. That’s why chat interfaces show text appearing word by word instead of all at once. It doesn’t make the model faster overall, but it makes the wait feel shorter. Token optimization A few habits keep both cost and latency down: - Don’t paste in more context than the task actually needs. - Reuse a stable prompt prefix where the provider supports caching it — repeated setup shouldn’t be repriced every call. - Ask explicitly for a short answer when a short answer is all you need. Models default to being thorough, which is usually not what you’re paying for. Previous: AI Foundations 2 - Hallucinations and Limitations /2026/09/13/ai-hallucinations-and-limits/ Next: AI Foundations 4 - Context /2026/09/13/llm-context-basics/