# Consequence models: AI that knows what happens next > Source: > Published: 2026-10-12 10:31:01+00:00 Coding agents can write a pull request, run the tests and fix what breaks. Yet someone still watches them. The reason isn't that they write bad code. It's that they can't tell, before they act, what acting will do. This post is about a kind of model we think is missing from almost every agent stack: a **consequence model**. It takes the state of the world and an action an agent is about to take, and returns one thing: the probability that the action works, and how it's likely to fail if it doesn't. An agent works in a loop: read the context, decide, act, look at what happened, check, repeat. Each step can go wrong, and errors compound. If each step succeeds 95% of the time, a 20-step task succeeds about 36% of the time. Now add a check that catches most bad steps before they land, so the agent can retry. Lift each step to 99% and the same 20-step task succeeds about 82% of the time. The agent didn't get smarter. It stopped walking off cliffs. | Steps in the task | 95% per step | 99% per step | |---|---|---| | 10 | 60% | 90% | | 20 | 36% | 82% | | 40 | 13% | 67% | *Chance of finishing a task when each step succeeds 95% or 99% of the time. This is arithmetic (p to the power n), not a benchmark.* That's why long agent tasks are mostly a reliability problem. Catching errors before they happen moves the line further than producing slightly better answers. A language model learns from text: what people wrote about the world. That's a record of what people say happens, not of what does happen when you act. There's a precise version of this idea. Judea Pearl's ladder of causation separates *seeing* (what's observed) from *doing* (what happens if I intervene). The [Causal Hierarchy Theorem](https://ftp.cs.ucla.edu/pub/stat_ser/bareinboim-etal-ch27-acm-2021.pdf) shows that data about seeing almost always leaves questions about doing undetermined. Text gives models secondhand descriptions of interventions, so this is a structural limit rather than an absolute one. But it's why asking a language model "will this work?" mostly tells you how plausible the action sounds. Confidence is the second problem. A model's stated confidence is hard to trust, and OpenAI's [GPT-4 technical report](https://arxiv.org/abs/2303.08774) showed the pre-trained model's probabilities were well calibrated and the post-trained model's much less so. An agent can't set a threshold on a number it can't trust. Think of a friend who has watched thousands of people touch a hot stove, and tells you, before you reach out, how likely you are to get burned and how. `git push --force` on this repository's main branch. Three properties make it useful: The shape isn't new: some decision models already return probabilities over a fixed set of options instead of text. A consequence model points the same idea at a different question: not "which of these is it?" but "what happens if I do this?" New kinds of model tend to appear when a new kind of training signal becomes cheap. Image models followed millions of labelled photos. Language models followed the web. Chat models followed human preference ratings. Reasoning models followed answers that can be checked automatically, like math and code. Agents now produce another one. Every test run, CI job, shell command, API call and reopened support ticket is an action with a recorded result. Nobody has to label it: the outcome is the label. Rerun a flaky test and you learn it was flaky. Merge a patch and you learn whether CI passed. The loop a consequence model sits in (the thresholds are examples; each team sets its own): If an agent pauses only on the few steps it's unsure about, a person stops supervising one agent and starts clearing many agents' exceptions in the morning. Overnight runs become practical, because the risky steps wait for a human and the rest don't. Intercom prices its Fin support agent at $0.99 per outcome ([its pricing page](https://www.intercom.com/pricing), read October 11, 2026). A vendor can only take on that risk if it can predict which jobs it will complete. A consequence model makes that prediction possible for other kinds of work: quote a price per merged pull request, and accept the jobs you expect to land. [MuZero](https://arxiv.org/abs/1911.08265), a game-playing system, plans by searching many moves ahead inside a model of the game that it learned itself. With a model of consequences, an agent can do the same with deploys, migrations and refactors: weigh many plans, then run the one most likely to work. Refunds, configuration changes, deploys and database changes are where agents would be most useful and are least trusted. A calibrated go or no-go on each action means people approve the risky tail instead of every step. Before any of that, there are narrow uses that are worth it on their own: Parts of this are in research and open source: What we haven't found is the piece in between: a small, calibrated model, trained on real outcomes, cheap enough to call on every step and offered as an API. ScaledThought runs inference for agents. Our view is that many of the next useful models won't be bigger language models, but smaller ones that each do one job well: decide, point to the source, estimate a number, predict a consequence. Consequence models are the one we think matters most, because they turn an agent from a tool someone operates into work you can hand off. One note on data: we don't store prompts or completions, so nothing in this post describes training on customer traffic. A model like this would learn from outcomes that the people running agents choose to share. If you're building in this direction, or you'd use a consequence model if one existed, we'd like to hear from you: [hello@scaledthought.com](mailto:hello@scaledthought.com). *Originally published at [scaledthought.com/blog/consequence-models](https://scaledthought.com/blog/consequence-models). I'm a founder of ScaledThought. Intercom and Fin are trademarks of Intercom; ScaledThought isn't affiliated with Intercom.*