# Post-Training: The $100M Question Nobody’s Asking About AI > Source: > Published: 2026-09-04 04:42:38+00:00 True. But it’s only half the story — and it’s the half that gets all the attention. The internet gives a model knowledge. It doesn’t give it manners. Something else does that, and it has a name most people have never heard: post-training. The first phase is called **pretraining**. The model reads a staggering amount of text and plays one game: predict the next word. Trillions of times. What comes out is a **base model** — and a base model is nearly useless as an assistant. All the knowledge is in there. None of it is reachable. Pretraining builds capability. Post-training builds access to it. Post-training runs in three stages. *Pretraining builds capability. Post-training builds *access* to it.* **Post-training** is everything that happens next. It runs in three stages. **Supervised Fine-Tuning.** *Fine-tuning* means training an existing model a bit more. *Supervised* means every example comes with the right answer attached. Show the model thousands of questions paired with genuinely good answers, and let it copy the pattern. It picks up more than “answer the question” — it learns the *shape* of being helpful: explain clearly, use steps when steps help, stop when you’re done. Think of it as an internship: watch an expert, imitate. The surprising part is that quality crushes quantity. A 2023 paper called LIMA found that researchers fine-tuned a 65-billion-parameter model on just 1,000 carefully curated examples — no preference learning, no reinforcement learning at all. It still produced answers people liked better than a competing model trained on 52 times more data. Their conclusion: almost all of a model’s knowledge arrives during pretraining. Only a small amount of good instruction data is needed to unlock it. Zhou et al., “LIMA: Less Is More for Alignment” (2023) — arxiv.org/abs/2305.11206 Feed a model mediocre answers, and it becomes reliably mediocre. Which means a huge share of real AI work isn’t glamorous — it’s people writing great example answers, and other people throwing out the bad ones. mitation gets you competent. It doesn’t get you good — not for the long run. Most questions have many acceptable answers. If someone writes “my dog died last week,” a technically correct answer and a kind answer are very different things. Imitation can’t teach that difference, because every training example was treated as equally right. So the second stage changes the question. Instead of one right answer, you show the model two answers and say which one people preferred. This is **RLHF: Reinforcement Learning from Human Feedback**. Reinforcement learning is trial → feedback → improve. Try something, get a score, adjust — it’s how AI learned chess. *Human feedback* means the score comes from people, not a game’s win condition. Since humans can’t rate millions of attempts, you train a small **reward model** to predict what a human would prefer. It’s a stand-in for human taste that runs at machine speed. Across hundreds or thousands of comparisons, the model internalizes taste. It learns that people want the actual recommendation, not five options. Nobody wrote a rule for that — it emerged from feedback. The effect isn’t small. The paper that introduced the modern RLHF recipe — the direct ancestor of ChatGPT — found that a 1.3-billion-parameter model trained with human feedback produced answers people preferred over a 175-billion-parameter model *without* it. You’ll also see **DPO — Direct Preference Optimization** — a newer shortcut that gets a similar result without a separate reward model. Simpler to run, which is why most open-source teams use it now. Human opinion is a noisy signal. But some problems don’t need a human at all. Math has correct answers. Code either passes its tests or it doesn’t. **RLVR: Reinforcement Learning with Verifiable Rewards.** Same trial-and-improve loop, except here a computer grades the work. The model attempts a problem thousands of times. Each attempt gets checked. What works gets reinforced. This isn’t imitation anymore — it’s practice. And left to practice, the model discovers strategies nobody taught it, including the most important one: working step by step beats guessing. That’s where chain-of-thought comes from. The visible reasoning you see before an answer wasn’t hard-coded — the model found that thinking out loud earns more correct answers, so it does more of it Every reward you set up gets pursued literally. Here’s the uncomfortable part: post-training is where a model’s *character* gets set — not its knowledge, but what it says, what it refuses, how it handles being wrong. Those are decisions made by people. That deserves more public attention than the size of the training set does. **It’s where models get specific.** General knowledge is becoming a commodity. Behavior on top of it isn’t. Post-training is how a model learns an industry, a process, a voice. **It’s why your data is valuable.** The scarce ingredient isn’t compute — it’s examples of good work, done well. Your best reports. The thumbs-up your users already give you. Most companies treat this as exhaust instead of fuel. **It’s the affordable lever.** Building a foundation model costs hundreds of millions of dollars. Post-training an open one costs a rounding error by comparison. The engine is already built — tuning it for your road is the cheap part, and that’s where most of the real value will be made. It’s worth noting how immature this process still is. Post-training determines how a model behaves — what it produces, what it declines, whether it holds a position under pressure. Yet the method is still largely manual: annotators writing demonstrations, choosing between candidate responses. The results aren’t in question — a 1.3-billion-parameter model outperformed one over a hundred times its size on the strength of it. The open problems are: reward hacking is unresolved, so is sycophancy, and there’s no settled answer to whose preferences belong in the training data. Pretraining is converging into a well-understood engineering problem with a known cost. Post-training governs how these systems actually *behave* — and it remains part method, part judgment. That distinction deserves wider recognition. When a model is described as capable or honest, those aren’t properties of scale. They’re design decisions, made by identifiable people, through a process still being worked out. [Post-Training: The $100M Question Nobody’s Asking About AI](https://pub.towardsai.net/post-training-how-language-models-are-taught-to-behave-d77c4dc75f88) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.