Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every commit. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product.
Large Language Models often feel magical.
You type:
"Write a Kubernetes deployment for Redis"
and seconds later a working configuration appears.
But under the hood, most modern AI systems are built using a surprisingly simple recipe:
In machine learning literature, these two phases are commonly called:
Understanding these ideas explains not only how models like GPT emerged, but also why modern AI development has become dramatically cheaper and faster.
Let's start with an intuition.
Imagine a child growing up.
For years, they read books, watch movies, listen to conversations, learn history, science, and language.
At this stage, nobody is training them to become a lawyer, doctor, or engineer.
They're simply absorbing information about the world.
Later, they attend medical school.
Now the learning becomes focused:
The broad education comes first.
Specialization comes later.
Modern AI systems follow exactly the same pattern.
Generative pre-training is the broad education.
Discriminative fine-tuning is the specialization.
During pre-training, a model is given massive amounts of text:
The objective is deceptively simple:
Predict the next token.
For example:
The capital of France is ___
The model learns that:
Paris
is likely.
Then it repeats this process trillions of times.
At first glance, this seems too simple to produce intelligence.
Yet something interesting happens.
To predict the next word accurately, the model gradually learns:
It wasn't explicitly taught these things.
They emerged as a side effect of prediction.
Because the model learns to generate data.
After training, it can produce:
The model effectively learns:
"What does valid human-generated content look like?"
This is fundamentally different from traditional classification systems.
A spam classifier only says:
Spam
or
Not Spam
A language model can generate entirely new content.
Hence the term:
Generative Model
Pre-training creates a very capable general-purpose model.
But general knowledge isn't always enough.
Suppose we want:
The pre-trained model knows many things.
Yet it may not perform optimally on a specific task.
This is where fine-tuning enters.
Instead of predicting the next token, we now train the model to make decisions.
For example:
Input:
The customer is extremely unhappy with the product.
Output:
Negative Sentiment
Or:
Input:
Chest X-Ray Image
Output:
Pneumonia
Or:
Input:
Transaction Record
Output:
Fraudulent
The model learns to discriminate between possible outcomes.
Hence the name:
Discriminative Fine-Tuning
The objective changes from:
Generate likely text
to:
Choose the correct answer.
Imagine you're building a support ticket classifier.
Without pre-training:
You would need:
With modern AI:
Start with a pre-trained model.
It already understands:
Then fine-tune it using a few thousand labeled examples.
Example:
"My payment failed twice"
→ Billing
"Unable to login"
→ Authentication
"Application crashes on startup"
→ Bug Report
The model quickly learns your domain-specific categories.
This dramatically reduces both cost and data requirements.
From a neural network perspective, pre-training and fine-tuning reuse the same parameters.
Suppose the model has:
70 billion parameters
During pre-training, these parameters learn general patterns.
During fine-tuning, they are adjusted slightly to become useful for a particular task.
Conceptually:
Pre-Training:
Internet → General Knowledge
Fine-Tuning:
General Knowledge → Specialized Expertise
A useful analogy is:
Pre-Training = Operating System
Fine-Tuning = Installed Application
The application works because the operating system already exists.
Similarly, fine-tuning works because pre-training already built rich representations of the world.
Before the deep learning era, most models were trained from scratch.
Each task required:
Pre-training changed everything.
A single giant model could learn broadly useful representations.
Thousands of downstream applications could then reuse that knowledge.
This idea has become the foundation of modern AI:
The pattern remains remarkably consistent:
Almost every major breakthrough of the past decade follows this recipe.
Interestingly, the industry is beginning to shift again.
Large models have become so capable during pre-training that many tasks no longer require explicit fine-tuning.
Instead of retraining the model, developers often provide:
In many production systems today:
Pre-Training + Prompt Engineering
replaces
Pre-Training + Fine-Tuning
Fine-tuning still matters, especially for specialized domains, but the balance is changing.
The pre-training phase is becoming increasingly powerful.
Generative pre-training and discriminative fine-tuning represent one of the most important ideas in modern machine learning.
The first teaches a model how the world works.
The second teaches it what job to perform.
Once you understand this pattern, many AI systems become easier to reason about. Whether you're working with LLMs, vision models, recommendation systems, or multimodal architectures, you'll repeatedly encounter the same formula:
Learn broadly first. Specialize later.
And that raises an interesting question:
As foundation models continue getting stronger, will fine-tuning eventually become a niche optimization, or will specialization always remain essential for high-performance AI systems?
*AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.
git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.*
Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.
| 🇩🇰 Dansk | 🇪🇸 Español | 🇮🇷 Farsi | 🇫🇮 Suomi | 🇯🇵 日本語 | 🇳🇴 Norsk | 🇵🇹 Português | 🇷🇺 Русский | 🇦🇱 Shqip | 🇨🇳 中文 | 🇮🇳 हिन्दी |
GenAI today is a race car without brakes. It accelerates fast -- you describe something, and large blocks of code appear instantly. But AI agents silently break things: they remove logic, relax constraints, introduce expensive cloud calls, leak credentials, and change behavior -- without telling you. You often find out in production.
** git-lrc is your braking system.** It hooks into
git commit
and runs an AI review on every diff In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen
At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…