{"slug": "program-aided-language-models-stop-making-the-model-do-the-math", "title": "Program-Aided Language Models: stop making the model do the math", "summary": "A developer introduced Program-Aided Language Models (PAL), which offload arithmetic from language models to a code interpreter. PAL splits reasoning and computation: the model writes a program describing the method, and an interpreter executes it exactly. This approach avoids the model's weakness in exact calculation, making results deterministic and verifiable.", "body_md": "Ask a language model how many times the letter \"r\" shows up in \"strawberry\" and there's a decent chance it says two. Ask it to compound $2,500 at 7% for eight years and it might quietly switch to simple interest and hand you $3,900 instead of $4,295.47. The reasoning sounds fine. The number is wrong. And the model is exactly as confident either way.\n\nThis isn't a bug you can prompt your way out of, because it's baked into what the model *is*. A language model predicts the next **token**. When it writes `1234 × 5678 =`\n\n, it isn't running a multiplication — it's guessing the most likely sequence of digit-tokens given everything it saw in training. There's no carry logic, no place value, no actual arithmetic underneath. You get a plausible-looking number: right length, believable leading digits, often just wrong. Anything that needs exact calculation, careful bookkeeping, or counting lands squarely in the model's weakest spot.\n\n**Program-Aided Language Models (PAL)** fix this with one move: don't ask the model for the answer, ask it for a *program*.\n\n🖩 **Interactive demo (CoT slips the arithmetic; PAL's code is really executed in your browser):** [https://dev48v.infy.uk/prompt/day27-pal.html](https://dev48v.infy.uk/prompt/day27-pal.html)\n\nPlain Chain-of-Thought asks one system — the model — to do two very different jobs: work out the *method* and crunch the *numbers*. The method part it's great at. The number-crunching is the part it's worst at. So a single slipped multiplication poisons an otherwise sound line of reasoning.\n\nPAL splits those two jobs apart. The model still does the reasoning — it decides which quantities matter, names them as variables, and writes the operations that combine them. But instead of stating a final number, it emits a short program and stops. Then a **real interpreter** runs that program and produces the number.\n\nCompound interest, the PAL way:\n\n```\nprincipal = 2500\nrate = 0.07\nfor y in range(8):\n    principal *= (1 + rate)   # interest ON interest\nprint(round(principal, 2))    # 4295.47\n```\n\nThe model never computed `1.07**8`\n\n. It described *how* to compute it and let Python do the arithmetic. That's the whole trick: the part the model is bad at is delegated to a system that's exact by construction.\n\n**It's deterministic.** An interpreter returns the same value every time — no sampling, no temperature, no drift. Run it once or a hundred times, you get 4295.47.\n\n**It's verifiable.** A wrong Chain-of-Thought is a wall of prose you have to re-check by hand. A wrong PAL program is a bug on a specific line. You can read it, unit-test it, diff it against a spec, or run it on new inputs to confirm the method generalises.\n\n**The model can be a little sloppy and still be right.** As long as the *method* in the code is sound, the exact arithmetic is handled downstream. That's a much lower bar than getting every multiplication right in your head.\n\nThink of PAL as Chain-of-Thought where the chain is executable. Same reasoning skeleton — break the problem into steps — but the steps are code, and something actually runs them.\n\nIf you've seen the ReAct pattern or function calling, PAL will feel familiar. It's a specific, powerful case of tool use where the tool is a code interpreter and the model's \"action\" is emitting a program. This is exactly what a modern code-interpreter feature does: the model writes Python, a sandbox runs it, the output comes back. Framed that way, PAL drops straight into agents — for any sub-task that's cleanly computable (math, date arithmetic, parsing, tallying), the agent reaches for the code tool instead of doing it in its head. The generalisation is simple: don't reason about what you can compute; compute it.\n\nPAL means executing text a model generated, so treat it as untrusted input. **Never eval it in your own process with full privileges.** A bad or manipulated program could read secrets, hit the network, delete files, or loop forever. Run it in an isolated sandbox — a locked-down subprocess, a container, a restricted VM — with no filesystem or network access, a memory cap, and a hard timeout. Capture only the output.\n\nAnd PAL has limits. It only helps when the problem can be *formalised* as code: math, logic, conversions, dates, counting, data manipulation. It does nothing for taste, judgment, or genuinely open-ended questions. It also won't save you from a bad plan — if the model picks the wrong formula, the interpreter will compute the wrong answer precisely and confidently. PAL removes arithmetic slips, not reasoning slips.\n\nThe rule of thumb is easy to remember: if a calculator or a five-line script could solve it, let the model write that script — and let the machine that's good at math do the math.\n\nPlay with all three problems in the demo: [https://dev48v.infy.uk/prompt/day27-pal.html](https://dev48v.infy.uk/prompt/day27-pal.html)", "url": "https://wpnews.pro/news/program-aided-language-models-stop-making-the-model-do-the-math", "canonical_source": "https://dev.to/dev48v/program-aided-language-models-stop-making-the-model-do-the-math-1d15", "published_at": "2026-07-07 10:49:40+00:00", "updated_at": "2026-07-07 10:58:24.169776+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "developer-tools"], "entities": ["PAL", "Python", "Chain-of-Thought", "ReAct"], "alternates": {"html": "https://wpnews.pro/news/program-aided-language-models-stop-making-the-model-do-the-math", "markdown": "https://wpnews.pro/news/program-aided-language-models-stop-making-the-model-do-the-math.md", "text": "https://wpnews.pro/news/program-aided-language-models-stop-making-the-model-do-the-math.txt", "jsonld": "https://wpnews.pro/news/program-aided-language-models-stop-making-the-model-do-the-math.jsonld"}}