{"slug": "you-only-need-the-frontier-model-for-one-single-edit", "title": "You only need the frontier model for one single edit", "summary": "A developer demonstrates that using a cheaper model to execute a plan written by a frontier model (the '/plan' pattern) can cost more than having the frontier model do the entire task, because the expensive part of an agent's work is reading context, not editing. The author proposes an alternative '/prewalk' approach that swaps models mid-task to preserve the expensive model's trajectory without duplicating reads.", "body_md": "# You only need the frontier model for one single edit\n\nMonkey see, monkey do! 🍌\n\n## /plan makes perfect sense. It really shouldn't!\n\nYou've heard this pitch; you may have even shipped it. The expensive model is clearly the better architect, but it feels like a waste to use it for the entire pipeline.\nWhy not let it do the **\"hard part\"?**\n\nRead the code, think deeply, write a precise plan. Then a model a tenth the price executes the plan. Senior architect, junior engineer.\n\nSounds great, right?\n\nFind the red dot above, labeled `Opus 4.8 + /plan†`\n\n. Opus plans read-only, Gemini Flash implements: lands at **$3.18 per task, 12.7 minutes, 84.6% pass**.\n\nOpus doing the entire task by itself, no handoff, no junior: **$2.78, 10.1 minutes, 84.6%**.\n\nThe *\"cost-saving\"* measure costs 14% *more* than not saving. **Huh?**\n\nThe mistake is upstream of the architecture diagram. People price agents the way they price people: senior time is expensive, so minimize senior involvement.\n\nBut the expensive part of an agent's day is not the fixing, building, or even the thinking. **Opus fixing things does not cost money. Opus reading things costs money.**\n\nTake a look at this admittedly anecdotal distribution of where my tokens went; fully automated agents look no different.\n\nNine percent of the tokens are edits. The rest is reading, and this split is not a quirk of one harness, or something you can \"fix\". Trust me, I've tried with snapcompact.\n\nAny agent, any model, any scaffold: the bill is essentially `O(reads)`\n\n.\n\nNow walk through every reason you'd reach for /plan, with that in mind:\n\n**\"I want the deep understanding of the big model.\"** The understanding lives in 100K+ tokens of grounded context: files read, dead ends eliminated, hypotheses tested. The plan document is a 2K-token postcard from that context. The executor gets the postcard, not the understanding, and has to rebuild the rest at its own expense.**\"The task is very complicated.\"** Then you don't want the main agent executing*at all*; a single read-only planning turn isn't the answer. Let it explore, then dispatch sub-agents to do the work. A game of telephone doesn't help you here.**\"I'm cost constrained.\"** Reading is the cost. /plan makes the frontier model read everything at frontier prices, then makes the cheap model read it*again*. You didn't move the expensive part; you duplicated it.\n\nHere's what that looks like in practice, with a diagram I've spent way too much time on:\n\nLook at the top ribbon. Opus reads `base.py`\n\n, `signing.py`\n\n, the test file (twenty cards of gray), then writes its plan and leaves. And what's the first thing Flash does with that beautiful document? It re-reads `base.py`\n\nand the test file, because a plan is not a file and you cannot edit prose. The gray reads just keep stacking, first at Opus prices, then again at Flash prices. There is no version of this where a second reader is the cost optimization.\n\n## Hand off a trajectory, not a fairytale\n\nA plan document is a literal postcard, describing a journey to a model that never took it.\n\nWhat actually *could* transfer something of value is the context window itself:\n\n`/prewalk`\n\ndoes this:\n\n- Start the task on the frontier model with one hidden instruction prefixed:\n*plan deeply, then capture the plan as a todo list, then start.* - The frontier model explores, writes the plan, initializes the todo list.\n- The moment the\n**first edit lands**(the point where it was confident enough to act), you swap to the cheap model and** prune the planning instruction**from context.\n\nThe trick is that:\n\n- The cheap model never goes \"wait, I thought we were planning\". There is no planning instruction left in its context.\n- As far as it knows, it explored around, created a comprehensive plan in the form of a todo list, and then confidently started executing.\n- Even better, it already made one\n**valid move**!*(a free in-context example)*\n\n### How we got here\n\nThe nice thing about running an open-source harness is that you get to chat with people about how they do things, and almost everyone has a completely different setup.\n\nAnyhow: I'd occasionally start easy-to-medium tasks with a frontier model, then switch to Kimi K27 after a few turns so it wouldn't fall into its usual thought loops. I never bothered to measure whether this was rational... until someone else mentioned doing the same thing.\n\nNaturally, I had to benchmark it: are we idiots, or does this actually work, and when?\n\n**First attempt:** swap at a fixed turn, say #4. Obviously bad in hindsight: sometimes the frontier model is still lost at turn four, sometimes it has already finished the whole fix.\n**Second attempt:** swap after the first edit. The model has demonstrated the pattern once, in place, in style. Pretty nice, although still finicky: small models kept declaring the task done out of nowhere.\n\nThe solution was to ask our unwitting herding agent to spell out a plan step by step, and then, once it's ready to execute, init a TODO list with a validation step for each item. It then edits some piece of code, and that's when we trigger the swap.\n\nGating on any edit alone is no good; the todo list still has a very important role here. Our tiny friend can forget the plan, a validation step, or what it's doing entirely, but it cannot forget the todo reminder that bugs it endlessly, giving us free steering.\n\nAnother funny failure mode: GPT 5.6 as the guide really likes creating 60-item TODO lists and completing them in batches (do they just hand out rewards for anything?), so an item limit in the prompt is a must.\n\n### The receipts\n\n**GPT-5.6 Sol**:\n\n| arm | pass | cost | duration |\n|---|---|---|---|\n| Executor: oneshot (GPT 5.6 Luna) | 77% | $0.60 | 570s |\n| /prewalk | 85%(+10%) | $1.04(−39%) | 300s(−47%) |\n| GPT 5.6 Sol: oneshot | 88% | $1.71 | 372s |\n\n97% of Sol's pass rate at 61% of the cost, and it's the *fastest of the three*, because Sol stops burning slow frontier tokens after the opening and Luna doesn't waste turns lost in the woods.\n\n**Opus 4.8**:\n\n| arm | pass | cost | duration |\n|---|---|---|---|\n| Executor: oneshot (Gemini Flash 3.5) | 60% | $1.16 | 360s |\n| /prewalk | 78%(+30%) | $1.46(−47%) | 402s(−34%) |\n| Opus 4.8: oneshot | 85% | $2.78 | 606s |\n\n92% of Opus at 53% of the cost, 1.5× the speed, +18 points over oneshot Flash.\n\nOne more thing before we move on. Scroll back up to the [ribbons from our django-13279 test ride](#django-13279-ribbons) and look for something that *isn't* there: cheating!\n\n## The effect I didn't expect\n\nEvery SWE-bench task is a bug that was really fixed, years ago, in public. The answer to the exam is on GitHub.\n\nBelow: the share of runs that went poking around the web for it. **Filthy cheaters!**\n\nSame model, same scaffolds, almost the same idea, yet wildly different behavior. Why does `/plan`\n\nstill cheat while `/prewalk`\n\ndoesn't?\n\nBest explanation I have: **prewalk starves it, from both ends.** Cheating is what a capable model does when it gets desperate. In the solo traces the GitHub turns start mid-run, once exploration stalls: Sol breaks around turn 14, Opus around turn 12. Prewalk terminates the frontier model at the *beginning* of its effort budget, median ~7 turns: it exits while it's still deriving an approach and landing a first edit (the confident phase), well before its googling phase begins. `/plan`\n\ngets neither mercy: it has no turn limit, and its deliverable (a comprehensive document explaining how the fix *should* work, without ever testing an edit against the code) is exactly the kind of assignment that breeds desperation.\n\nThe executor then inherits the opposite of desperation: a context where the approach already survived contact with the code. Repro written, first edit landed, checklist ticking. Nothing in that context looks like searching, so the imitation machine doesn't search.\n\n## Prefill walked so prewalk could run\n\nNone of this is a new idea. It's the oldest trick in the book: **prefill**. Assistant doesn't do what you want? Start the assistant's turn yourself, and the model continues as if the words were its own.\n\nIt began as a consistency hack prior to grammar-constrained decoding. omp and many others still do it: session titles come from a tiny local model, which just happens to do better when you start its turn with `<title>`\n\n.\nA model that small can't be *argued* into a format, but it can be tricked into one.\n\nThen the red-teamers found the other end. Prefill \"Sure, here's how to…\" and a much larger model sails past its own refusal: it has no channel distinguishing words it said from words placed in its mouth, and consistency with \"having already accepted\" beats the system prompt. Prefill became a standard jailbreak class, powerful enough that it's now banned at the inference layer nearly everywhere, starting with Anthropic since Sonnet 4.5 IIRC. JSON mode and structured outputs paved over the legitimate uses, and it faded away.\n\nBut the principle can't stop working, because it isn't a funny quirk; it's what autoregression *is*.\nYou can't hand a frontier model ten prefilled tokens anymore; some won't even let you disable thinking, precisely so that you can't maliciously prefill turns (which the model will *hopefully* realize while thinking, from the absent assistant thinking blocks).\nBut nothing stops you from handing it ten innocently prefilled **turns**: exploration that already happened, a todo list mid-checkmark.\n\nIt ships in omp as of today as `--prewalk`\n\n, `--prewalk-into <model>`\n\n, or just `/prewalk`\n\n.\nIt should be easy to implement essentially anywhere so if you get a chance to give it a go, do let me know if you have any success with it!", "url": "https://wpnews.pro/news/you-only-need-the-frontier-model-for-one-single-edit", "canonical_source": "https://stencil.so/blog/prewalk", "published_at": "2026-07-15 05:08:26+00:00", "updated_at": "2026-07-15 05:48:23.838306+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools", "ai-infrastructure"], "entities": ["Opus", "Gemini Flash", "Claude"], "alternates": {"html": "https://wpnews.pro/news/you-only-need-the-frontier-model-for-one-single-edit", "markdown": "https://wpnews.pro/news/you-only-need-the-frontier-model-for-one-single-edit.md", "text": "https://wpnews.pro/news/you-only-need-the-frontier-model-for-one-single-edit.txt", "jsonld": "https://wpnews.pro/news/you-only-need-the-frontier-model-for-one-single-edit.jsonld"}}