How LLMs Actually Work

OpenAI’s GPT models and Meta’s LLaMA-family use transformer architectures that convert text into token IDs, then map each ID to a dense vector via an embedding matrix. These vectors are combined with positional encodings and processed through stacked attention and feed-forward layers to predict the next token. Understanding this core machinery explains why LLMs struggle with tasks like counting letters in “strawberry” — they operate on subword tokens, not characters.