Before jumping into APIs, RAG, agents, and AI applications, I wanted to understand what actually happens inside an LLM.
I kept coming across terms like neural networks, deep learning, Transformers, attention, tokens, embeddings, BERT, GPT, and causal language modeling — but they all felt like disconnected pieces.
I could understand each concept individually, but I didn't have a clear picture of how they all connected.
So I decided to step back and build the mental model from the ground up.
This article is my attempt to connect those pieces.
Before understanding LLMs, it helps to understand where they come from.
A neural network is a machine learning model that learns patterns from data.
For example, suppose we want to recognize handwritten digits.
We could give a neural network thousands of images of handwritten digits:
Images of handwritten digits
↓
Neural Network
↓
Learned patterns
↓
Prediction
Instead of manually programming rules such as:
"If the image has this curve and this line, it must be a 3."
we allow the neural network to learn those patterns from examples.
The things the network learns are stored in its parameters, primarily weights and biases.
As we increase the number of layers in a neural network, we enter the world of deep learning.
Machine Learning
↓
Neural Networks
↓
Deep Learning
Different neural network architectures became useful for different types of problems.
For example:
But language has a special challenge.
Consider:
"The animal didn't cross the road because it was tired."
To understand what "it" refers to, we need to understand its relationship with the other words in the sentence.
This is where things like attention become important.
And this eventually leads us to Transformers.
Before Transformers, RNNs and LSTMs were commonly used for language-related tasks.
They processed sequences step by step.
For example:
I → love → machine → learning
This sequential processing created some challenges.
One major problem was that it made training difficult to parallelize efficiently.
It could also become difficult to capture relationships between tokens that were far apart in a long sequence.
Then, in 2017, researchers introduced the Transformer architecture in the paper:
"Attention Is All You Need"
The original Transformer was designed for machine translation.
The basic architecture looked like this:
English sentence
↓
Encoder
↓
Contextual representation
↓
Decoder
↓
French sentence
For example:
"I love cats"
↓
Transformer
↓
"J'aime les chats"
The key idea that made Transformers different was attention.
Instead of processing a sequence strictly one step at a time, the Transformer could use attention to determine relationships between different tokens.
This made it much better suited to capturing context and also allowed much more parallel computation during training.
At a high level, attention answers:
"When I'm processing this token, which other tokens should I pay attention to?"
Consider:
"The animal didn't cross the road because it was tired."
When processing "it", the model needs to determine which other tokens are relevant.
Conceptually:
The animal didn't cross the road because it was tired.
↑
"it"
The model can assign different levels of importance to different tokens.
This is called self-attention, because tokens in the sequence attend to other tokens in the same sequence.
I initially thought attention was something separate from the Transformer.
It isn't.
Attention is one of the core mechanisms inside a Transformer.
For now, the mental model I use is:
Current token
↓
Look at other relevant tokens
↓
Combine useful information
↓
Create a contextual representation
The mathematical details — Query, Key, Value, attention scores, and matrix multiplication — deserve their own article.
The original Transformer architecture contained two major components:
Input
↓
Encoder
↓
Representation
↓
Decoder
↓
Output
A useful beginner mental model is:
Encoder → processes and represents the input
Decoder → generates the output
For translation:
English
↓
Encoder
↓
Representation
↓
Decoder
↓
French
This architecture was designed around a very natural problem:
Take one sequence and transform it into another sequence.
For example:
But later, researchers realized that we don't always need both parts.
This led to different Transformer architectures.
This was one of the things I initially found confusing.
If BERT, GPT, T5, and BART are all based on Transformers, why are they different?
The answer is:
Transformer is an architecture, not one specific model.
Different models can use different parts of the Transformer architecture.
A simple mental model is:
Transformer
│
┌───────────────┼───────────────┐
↓ ↓ ↓
Encoder-only Decoder-only Encoder-decoder
↓ ↓ ↓
BERT GPT T5 / BART
Let's look at each one.
An encoder-only model uses the encoder part of the Transformer.
The encoder processes the input and creates contextual representations.
The most famous example is BERT.
BERT was trained using Masked Language Modeling.
For example:
The cat is [MASK] on the mat.
The model tries to predict the missing token.
Because the encoder can use context from both sides of the masked token, it can build a bidirectional representation.
This makes encoder-only models useful for tasks such as:
The simple mental model:
Encoder-only → understand/represent text
A decoder-only model uses the decoder part of the Transformer.
The most famous example is GPT.
GPT stands for:
Generative Pre-trained Transformer
GPT is trained primarily using causal language modeling.
Its objective is:
Predict the next token using the previous tokens.
For example:
The cat is
↓
sleeping
Then:
The cat is sleeping
↓
next token
The model keeps generating one token at a time.
This is why GPT is naturally suited to text generation.
The simple mental model:
Decoder-only → generate text
Encoder-decoder models use both parts.
The encoder processes the input.
The decoder generates the output.
For example, in summarization:
Article
↓
Encoder
↓
Representation
↓
Decoder
↓
Summary
Or translation:
English
↓
Encoder
↓
Representation
↓
Decoder
↓
French
Examples include T5 and BART.
The simple mental model:
Encoder-decoder → transform one sequence into another
This is the simplest table I use to remember them:
| Model | Architecture | Simple mental model |
|---|---|---|
| BERT | Encoder-only | Understand / represent |
| GPT | Decoder-only | Generate |
| T5 | Encoder-decoder | Input → Output |
| BART | Encoder-decoder | Understand → Generate |
The important thing to remember is:
They are different ways of using the Transformer architecture.
Now we can finally talk about Large Language Models.
LLM stands for:
Large Language Model
Let's break down the name.
"Large" generally refers to the enormous number of learned parameters.
Parameters are numerical values learned during training, primarily:
Modern language models can have billions of parameters.
The model is trained on large amounts of language data and learns patterns and relationships in that data.
It is ultimately a neural network that has learned these patterns through its parameters.
So:
LLM
│
├── Large
│ └── Many learned parameters
│
├── Language
│ └── Learns patterns from language
│
└── Model
└── Neural network
GPT-style models are generally large decoder-only Transformer models.
They are trained primarily using causal language modeling.
The basic objective is surprisingly simple:
Predict the next token based on the previous context.
Suppose the training data contains:
The sky is blue.
The model learns to predict:
The
↓
sky
Then:
The sky
↓
is
Then:
The sky is
↓
blue
This happens across enormous amounts of training data.
During training:
Input context
↓
Predict next token
↓
Compare with actual token
↓
Calculate error
↓
Update parameters
↓
Repeat
Over time, the model becomes better at predicting the next token.
This was one of the most interesting things for me to understand.
If GPT is fundamentally trained to predict the next token, how can it:
The answer is largely context.
The model doesn't necessarily need a separate mechanism for every task.
The prompt provides context about what kind of continuation is expected.
For example:
Translate to French:
I love cats.
The context tells the model that the expected continuation is a translation.
Or:
Summarize this article:
[article]
Now the expected continuation is a summary.
Or:
Write a Python function that sorts a list.
Now the expected continuation is code.
The underlying mechanism is still:
Context
↓
Predict next token
↓
Add token to context
↓
Predict next token
↓
Add token to context
↓
Repeat
This is one of the most important mental models I have taken away:
A relatively simple training objective — next-token prediction — can result in a model capable of many different language tasks.
At this point, another question naturally appears:
If the model predicts tokens, does it actually process words directly?
No.
The model needs to convert our text into numerical representations that a neural network can process.
This starts with tokenization.
For example:
"I love programming"
might become something conceptually like:
["I", " love", " programming"]
The exact tokens depend on the tokenizer.
A token can be:
So:
A token is not necessarily a word.
The token is then mapped to a numerical ID:
Text
↓
Tokens
↓
Token IDs
The Transformer doesn't directly work with the token ID as a meaningful representation.
The token ID is essentially an index.
The model needs a richer numerical representation.
This is where embeddings come in.
Conceptually:
"cat"
↓
Token
↓
Token ID
↓
Embedding
↓
[0.21, -0.42, 0.17, ...]
An embedding is a vector containing many numerical values.
For example, if an embedding has 4,096 dimensions:
[0.21, -0.42, 0.17, ..., 0.31]
it simply means that the vector contains 4,096 numbers.
These numbers allow the Transformer to perform mathematical operations on the representation of the token.
At this point, our mental model becomes:
Text
↓
Tokenization
↓
Tokens
↓
Token IDs
↓
Embeddings
↓
Transformer
I'll go much deeper into tokenization and embeddings in the next article.
After connecting all these concepts, this is the mental map I currently have:
Neural Networks
↓
Deep Learning
↓
Sequence Models
↓
Transformers
↓
Attention
↓
Transformer Architectures
↓
┌────────────────┼────────────────┐
↓ ↓ ↓
Encoder-only Decoder-only Encoder-decoder
↓ ↓ ↓
BERT GPT T5 / BART
↓
LLMs
And if I zoom into a GPT-style LLM:
User Prompt
↓
Tokenization
↓
Token IDs
↓
Embeddings
↓
Transformer Blocks
↓
Attention
↓
Contextual Representation
↓
Next-token prediction
↓
Generated token
↓
Repeat
This finally gave me the map I was missing.
I don't need to know every mathematical detail yet.
I first need to know where each concept belongs.
Now comes the question I'm most interested in:
What actually happens inside an LLM when I type a prompt?
Suppose I ask:
What is the capital of India?
At a very high level, the process looks something like:
"What is the capital of India?"
↓
Tokenization
↓
Token IDs
↓
Embeddings
↓
Positional Information
↓
Transformer Blocks
↓
Attention
↓
Contextual Representation
↓
Output Scores
↓
Softmax
↓
Probability Distribution
↓
Next Token
↓
Repeat
And this is where I want to go next.
Now that I have the high-level map, I want to open up the black box.
In the next article, I'll start from the beginning of the inference process:
Prompt
↓
Tokenization
↓
Token IDs
↓
Embeddings
↓
Positional information
↓
Transformer
↓
Attention
↓
Hidden states
↓
Unembedding
↓
Logits
↓
Softmax
↓
Probability distribution
↓
Next token
I'll break down each step and answer questions like:
The goal is not to start with mathematics.
The goal is to first build an intuitive mental model and then introduce the mathematics once the pieces make sense.
If I had to summarize everything I've learned so far in one sentence:
An LLM like GPT is a large neural network built using the Transformer architecture, trained on language to predict the next token, and capable of many language tasks because of the patterns and representations it learns from enormous amounts of data.
The biggest thing that changed for me was realizing that all these terms aren't isolated concepts.
They fit together:
Neural Networks
↓
Deep Learning
↓
Transformers
↓
Attention
↓
Encoder / Decoder Architectures
↓
GPT
↓
LLMs
↓
Tokens
↓
Embeddings
↓
Transformer Processing
↓
Next-token Prediction
↓
Generated Text
Now that I have the map, I'm ready to understand what's actually happening inside the box.