I run an LLM in the core loop of a small production app: Mening corrects language learners' writing every day. Unit economics put the everyday call on a cheap, fast model tier, and the output feeds a database, so it has to obey a contract: strict JSON, a closed set of error categories, explanations in the learner's UI language, no invented "fixes".
The cheap tier is genuinely good at the task. It is terrible at following the rules around the task. This post is about the escalation ladder I climbed before accepting the house rule in the title, with real numbers from the two fights that taught me it.
Chinese learners kept getting a specific non-correction: the model would insert 了 into sentences that were already fine, then explain why the learner "needed" it. A native-speaker check said the original sentences were correct.
I climbed the prompt ladder one rung at a time, deploying and watching live traffic after each:
Four iterations, and the model still did it. Not always - which is worse than always, because it looks fixed until it isn't.
The fix that held wasn't wording. The parsing boundary now drops two classes of edits before anything reaches the database: no-op edits, where the "wrong" and "correct" strings are equal, and pure 了-insertion edits. A few lines of Go, zero regressions since. The prompt still asks nicely; the parser doesn't care whether the model listened.
The contract says: explain errors in the user's UI language (say, Russian), never in the target language (say, Finnish). A real user reported Finnish explanations. Intermittent, of course.
Before touching anything I measured it: the same seven real submissions, 21 runs, exact production request shape. Six out of 21 responses drifted (29%), and drift was all-or-nothing per response - the model commits to one language for the entire JSON.
Then I measured the fixes everyone reaches for first:
That last one is the cheapest lesson in this post: the system prompt is far from the generation point, and for a small model, distance matters. A rule that must survive belongs in the user turn, next to the data that fights it.
But 0/21 on a 21-run sample is not a guarantee, so the boundary got a guard anyway. offScriptExplanations
checks the writing system of every explanation in the response against the UI language's script. Deliberately dumb on purpose:
When it fires, the correction runs one repair retry with the violation spelled out. And if the retry is still wrong, the wrong-language answer is kept, not dropped - a correction the user has to squint at beats no correction at all. Guards should degrade, not destroy.
Every error the model reports lands in a closed set of six categories, enforced twice - a CHECK constraint in SQLite and validation at the parse boundary. Anything outside the set is rejected, and any fields the model invents are dropped on the floor.
That last part turned out to be a free security property. The model output is the only untrusted input in the system, and the parser treats it accordingly: no tool calls to hijack, no extra fields to smuggle instructions through, a hard token cap. Prompt injection against this pipeline mostly has nowhere to go, not because the prompt says "ignore injections", but because the boundary only accepts the shape it expects.
What I now do, in order, when a cheap model breaks a rule:
Rungs 1-4 reduce the failure rate. Only rung 5 sets it to zero, and the rules that reach rung 5 are exactly the ones where "rarely" is unacceptable.
The prompt is a request. The parser is a contract.
The app this comes from is mening.app - daily writing practice that remembers which mistakes you keep repeating. The memory side of it is written up in error memory under the hood.