An AI costing engine should never let the model do the costing. That sounds like a contradiction until you split the job in two: a deterministic engine computes every price from admin-owned rate tables, and the language model’s only role is to explain, flag, and answer questions about a number it was handed — never to produce one.
The stakes are specific to pricing. A hallucinated paragraph in a draft blog post is an editing problem. A hallucinated number in a quote is a commercial commitment — a figure a customer can screenshot and a sales team may feel bound to honor. And the published research on model arithmetic says the risk is not hypothetical: language models fail at exactly the kind of multi-digit, multi-step computation a quote requires, and when they fail they tend to fail confidently, with a plausible-looking value rather than a refusal.
This guide covers the documented failure class, the architecture that removes it, the specific API mechanisms — forced tool use and strict structured outputs — that make an invented price structurally impossible rather than merely discouraged, a stage-by-stage responsibility table, the drift signals to audit for, and where the regulatory direction of travel is heading. It is the build-side counterpart to buying quoting software off the shelf.
- 01The model never computes; the engine never explains.A deterministic costing engine owns rate lookup, surcharges, tax, and totals from admin-owned rate tables. The LLM narrates the engine’s output and flags anomalies — a read-only narrator, not a calculator.
- 02The fence is mechanical, not a prompt.Forced tool use (Anthropic’s tool_choice) plus strict schema enforcement (strict: true on tool definitions, OpenAI’s Structured Outputs) makes emitting a bare number structurally impossible — not just discouraged by instructions.
- 03The arithmetic failure class is documented, not folklore.A 2023 study measured GPT-4 at 59% on 3-digit multiplication falling to 4% at 4 digits and 0% at 5 — and 2024–2026 research shows the failure class persisting across generations, with fabrication rising as problems grow.
- 04Audit the explanation, not just the computation.Every dollar figure in the model’s prose must match a value actually passed to it in tool input. Arithmetic appearing inside the explanation — “40 × $12 = $480, plus…” — is the tell that the boundary has eroded.
- 05This is the anti-CPQ build, and regulators favor it.Traditional CPQ already prices deterministically without an LLM. The risk arrives when a chat layer is bolted on top. Keeping determinism and adding the model only as narrator also aligns with tightening logging and explainability expectations.
01 — The Failure ClassModels don’t run arithmetic — they pattern-match it. #
The case against letting a language model compute a price starts with a body of research that predates today’s frontier models but established the failure class cleanly. A September 2023 study, “GPT Can Solve Mathematical Problems Without a Calculator”, measured GPT-4’s raw multi-digit multiplication accuracy at just 4.3% without tool assistance. The same body of work documented a cliff, not a slope: 59% accuracy on 3-digit-by-3-digit problems, 4% at 4 digits, 0% at 5 digits.
Those are 2023-vintage, GPT-4-specific figures — they establish that the failure class is real and documented, not that today’s models fail at that exact rate. But the class did not disappear with newer generations. Independent benchmarking reported by TechCrunch in October 2024 found GPT-4o’s accuracy dropping below 30% beyond four-digit-by-four-digit multiplication. For current frontier models, no equivalent fresh benchmark was reviewed for this post — the honest framing is qualitative: multi-digit arithmetic inside a free-form generation loop is still not a guarantee you can bill against.
3-digit × 3-digit
Raw accuracy without tools, per the September 2023 arXiv study. Passable-looking on small problems — which is exactly what makes the cliff below it dangerous.
4-digit × 4-digit
One digit more and accuracy collapses. A quote with line items in the thousands of dollars lives on this side of the cliff, not the other one.
5-digit × 5-digit
Total failure at five digits in the 2023 measurement. Era-specific numbers — but the shape of the curve is the durable lesson for anyone pricing by prose.
The more recent evidence is arguably worse for pricing, because it characterizes how models fail. A 2026 forensic benchmark on LLM matrix-computation tasks — a fully enumerated 6,600-output evaluation — found that hallucination, not mere abandonment, was the dominant failure mode as problems grew: 17.0% of outputs at 3×3, 27.2% at 4×4, 47.1% at 5×5. The researchers distinguish two types: abandonment, where the model disengages from the computation, and fabrication, where it invents a plausible-looking value. Fabrication is the dangerous one for pricing — because it looks exactly like a real number.
Hallucination rate by problem size · LLM matrix computation
Source: arXiv:2605.16675, 6,600-output enumerated evaluation, 2026Root-cause analysis explains why tool-off works so well: research aggregated in the 2023 “Embers of Autoregression” line of work attributes LLM arithmetic failure to superficial pattern-matching over digit sequences rather than execution of an arithmetic algorithm, with errors concentrating in the middle digits of a computation. The model was never running an algorithm to begin with — so the fix is not a better prompt asking it to be careful. The fix is removing the computation from the model entirely.
02 — The PatternOne system computes, one system narrates. #
The agent-audited costing engine pattern splits a quote into two systems with a hard boundary between them. The deterministic engine is ordinary software: it reads rate tables that admins own and edit, applies surcharge and discount rules in a fixed order, handles tax and currency, and produces a total plus an itemized breakdown. Same inputs, same output, every time — testable with ordinary unit tests. The language model sits on top as a read-only narrator: it receives the engine’s structured output and turns it into plain language, flags line items that look anomalous against history, and answers follow-up questions by citing — never recomputing — the engine’s figures.
This is deliberately the opposite of the “AI does the pricing” pitch. Configure-price-quote software — a market analysts estimate in the low-to-mid single-digit billions of dollars and growing at a double-digit compound rate, with Mordor Intelligence alone projecting roughly $3.63B in 2026 growing to $7.55B by 2031 — already solves rules-based pricing without any LLM. Aggregated roundups report roughly 85% of B2B organizations integrating some form of CPQ solution (a figure from 2023 data still cited in 2026). Most rate-card businesses do not lack pricing software. The new risk arrives specifically when a chat or agent interface gets bolted onto that existing rate card — and the new discipline this post describes is keeping CPQ’s determinism while adding the model only as an explanation layer. If you are weighing buying that layer instead of building it, our CPQ buyer’s guide is the buy-side counterpart to this build-side pattern.
The engine
Deterministic code over admin-owned rate tables. Rate lookup, surcharges, discounts, tax, currency, final total. Unit-testable, versioned, and auditable line by line. No model in the path.
The narrator
The LLM receives the computed breakdown as structured tool output and explains it in plain language, flags anomalies against history, and answers follow-ups by citing the engine’s line items.
The pattern slots into a larger quoting architecture — server-truth state, resumable sessions, CRM handoff — that we cover separately in the agent-built quote funnel pattern; this post is about the pricing-computation layer inside it. And to be clear about scope: how a quote flow converts is a design discipline of its own, covered in our multi-step quote flow guide — nothing here is conversion advice. This is the architecture that makes the number trustworthy, which is what teams building CRM and quoting automation have to get right before any of the rest matters.
03 — The Mechanical FenceMake a bare price impossible to emit, not just discouraged. #
Most “don’t let the AI make up prices” advice stops at a system prompt and a disclaimer. The pattern’s differentiator is that the boundary is mechanical: the APIs now ship features that make an un-audited number structurally impossible to emit, and a costing build should use all of them.
Forced tool use. Anthropic’s tool-use documentation is explicit that the default tool-choice behavior — where the model decides per turn whether to call a tool — can be overridden: “To require a tool call rather than rely on prompting, set tool_choice
.” In a costing context, that means the turn that produces a price must route through the pricing tool. The model cannot answer “how much would this cost?” with free-form text containing a number it dreamed up, because the only legal move is a tool call.
Strict schemas. The second half of the fence is schema enforcement on the tool call itself. Anthropic’s docs put it directly: “Add strict: true
to your custom tool definitions to ensure Claude’s tool calls always match your schema exactly” — guaranteed conformance rather than best-effort. OpenAI’s equivalent, Structured Outputs, makes the same promise for its function-calling API:
"Setting strict to true will ensure function calls reliably adhere to the function schema, instead of being best effort."— OpenAI function calling documentation
OpenAI’s strict-mode requirements are narrow and load-bearing in a useful way: every object in the schema needs additionalProperties: false
, and every property must appear in required
— optional fields are modeled as nullable types, not omitted. That rigidity is a feature here. It means the schema itself is what stops a stray estimated_price
field from slipping past validation: if the explanation payload only has fields for the engine’s line items, there is nowhere for an invented figure to live. One practical design constraint follows: OpenAI notes that strict-schema requests undergo extra processing on first use and are then cached, so keep the costing-explanation schema stable and let the content vary — a different schema per SKU would pay the latency tax repeatedly.
One honest caveat keeps validation in the loop even inside the fence. Anthropic documents that a model can under-specify or guess a missing required tool parameter — inferring a value the user never gave — and states plainly: “This behavior is not guaranteed, especially for more ambiguous prompts and for less capable models.” Schema conformance guarantees shape, not truth. Your application still validates that the parameters the model passed match what the user actually said, and that the figures in the final explanation match what the engine actually returned.
*“never invent a price”*is a request. Forced tool use plus a strict schema is a
type system: the model literally has no output channel through which a bare number can reach the customer. The industry mechanism has moved from “hope the model formats correctly” to “make incorrect output structurally impossible to emit” — per the first-party tool-use documentation from
Anthropicand OpenAI. That shift is the whole reason this pattern is buildable today.
04 — Responsibility SplitWhere the boundary goes, stage by stage. #
Existing CPQ and AI-pricing content tends to treat “AI plus pricing” as one undifferentiated blob. The table below is the granular version: each stage of a quote’s lifecycle, who owns it, what the model is permitted to touch, and the verification check that keeps the boundary honest.
| Stage | Deterministic engine | LLM may touch | Verification check |
|---|---|---|---|
| Computation — never the model’s job | |||
| Rate lookup | Reads admin-owned rate tables; versioned, permissioned edits | Nothing | Unit tests pin every rate row; table edits require an admin identity in the audit log |
| Surcharges & discounts | Applies rules in a fixed, documented order | Nothing | Golden-file tests: known inputs must reproduce known totals on every deploy |
| Tax & currency | Computes from configured jurisdictions and rate sources | Nothing | Reconcile a sample of quotes against the tax engine’s own records |
| Final total | Sums line items; emits an itemized structured breakdown | Nothing | Breakdown must internally sum to the total before it ever reaches the model |
| Language — where the model earns its keep | |||
| Anomaly flagging | Computes thresholds and historical comparisons | Narrates the flag (“this line is well above the usual range”) | Every flagged claim must reference an engine-computed comparison, not the model’s impression |
| Explanation | Supplies the breakdown as tool output | Writes the plain-language walkthrough of the line items | Every dollar figure in the prose must string-match a value in the tool input |
| Follow-up Q&A | Re-quotes on demand for any changed inputs | Answers by citing existing figures or triggering a fresh engine call | “What if” answers containing a number the engine never produced are rejected before display |
The right-hand column is the part most builds skip. The engine being deterministic is necessary but not sufficient — the audit has to extend to the model’s output, because the failure mode that matters commercially is the explanation silently reintroducing a different number than the one the engine computed. The model referencing $4,231.50 in prose is fine; the application should validate that every dollar figure appearing in the explanation matches a value that was actually passed to it as tool input, not merely that it looks plausible.
05 — Worked ExampleAn illustrative quote, end to end. #
Here is the shape of a single quote moving through the pattern, with a fully invented rate card for a hypothetical logistics company — every figure below is illustrative, not a real rate. The engine receives the configuration (40 handling units, 260 miles, 18 cartons), reads its rate tables, and emits the breakdown. The model receives that breakdown as tool input and writes the explanation.
| Line item | Rate-table input | Formula applied | Engine output |
|---|---|---|---|
| Crew handling | $12.00 per unit | 40 units × $12.00 | $480.00 |
| Line-haul | $2.15 per mile | 260 miles × $2.15 | $559.00 |
| Packing materials | $4.50 per carton | 18 cartons × $4.50 | $81.00 |
| Subtotal | — | $480.00 + $559.00 + $81.00 | $1,120.00 |
| Fuel surcharge | 8% of subtotal | 0.08 × $1,120.00 | $89.60 |
| Total | — | $1,120.00 + $89.60 | $1,209.60 |
The narrator’s job with this payload: “Your quote comes to $1,209.60. The largest component is line-haul at $559.00 for the 260-mile leg, followed by crew handling at $480.00 for 40 units. The 8% fuel surcharge adds $89.60 on the $1,120.00 subtotal.” Every figure in that paragraph string-matches a value in the tool input. Now the failure version: “40 units at roughly $12 is about $480, so with distance you’re looking at around $1,250 all-in.” The second version contains arithmetic the model performed itself and a total the engine never produced — and that distinction, checkable by a simple validator, is the entire audit.
Notice also what the admin owns. Change the fuel surcharge from 8% to a different rate in the rate table and every subsequent quote reflects it — no prompt edits, no fine-tuning, no model behavior to re-verify. Pricing policy lives in data that operations staff control, which is exactly where a business wants it.
06 — Drift SignalsIs the model still explaining — or has it started asserting? #
The boundary erodes gradually, not in one dramatic failure. The monitoring problem is assertion creep: transcript patterns where the narrator drifts from citing the engine to producing claims of its own. These four signals are worth building explicit checks for — each one is detectable in the transcript without any model introspection.
Arithmetic appears in the prose
The explanation starts to contain computation — “so 40 units × $12 = $480, plus the 8% surcharge…” — instead of quoting the engine’s line items. The moment the output performs multiplication or addition in natural language rather than citing structured tool output, the boundary has eroded.
A figure no tool result contains
A dollar amount appears in the explanation that string-matches nothing in the tool input. This is the fabrication failure mode the 2026 benchmark research identifies as dominant at higher complexity — dangerous precisely because an invented value looks like a real one.
A what-if answered without a tool call
The customer asks “what would 60 units cost?” and the model answers with a computed new number instead of calling the engine again. Forced tool use on pricing turns makes this structurally impossible — if it appears, the tool-choice configuration has regressed.
Confidence language near a number
Phrases like “I’m confident this price is correct” next to a figure. Research on LLM confidence describes self-reported certainty as generated phrasing shaped by training — not a calibrated probability. Engine outputs never reassure; a narrator that does is editorializing.
The projection worth making here: as agent interfaces spread across quoting stacks, the differentiator between builds will not be whether the demo looks fluent — every build’s demo looks fluent. It will be whether the team can produce, for any quote a customer ever saw, the engine version, the rate-table state, the tool payload, and the validated explanation that shipped. Teams that treat assertion creep as a monitored production metric, the way they treat error rates, will be the ones that can put an agent in front of paying customers and keep it there.
07 — Audit & RegulationThe audit trail is the product, not the paperwork. #
Everything above produces a pleasant side effect: the pattern is an audit trail by construction. Every price traces to a rate-table version, an engine computation, and a validated explanation — which is roughly the shape regulators are converging on for AI-touched decisions generally. The EU AI Act’s transparency and logging provisions for high-risk AI systems took effect in August 2026, requiring detailed operation logs — inputs, decisions produced, logic applied — to support traceability, with penalties for non-compliant high-risk systems reaching roughly €35M. Whether a B2B rate-card explainer of the kind described here would itself qualify as high-risk under the Act’s categories is not something we assert — the point is the direction of travel: logging and explainability expectations are tightening, and a build that already satisfies them costs nothing extra.
fields per AI-touched decision
2026 compliance literature recommends logging at least 12 fields, including: UTC timestamp, decision ID, authenticated identity, model identity and version, inputs with source, the rule invoked, human-readable reasoning, output, downstream action, and an integrity proof. Practitioner guidance, not codified law — but a useful checklist.
penalty ceiling · high-risk systems
Logging and traceability provisions for high-risk AI systems effective August 2026. Whether this pattern is in scope is unconfirmed — treat it as the direction of travel, not a classification.
comment period closes
The FTC published a proposed enforcement policy statement on personalized pricing on August 19, 2026 — a proposal out for comment, not a rule in force, and disclosure-focused rather than prohibitory.
The FTC item deserves precise framing, because rushed coverage will flatten it. What the agency proposed on August 19, 2026 is an enforcement policy statement, open for public comment through September 18, 2026. It defines personalized pricing as setting a price based on analysis of a consumer’s personal data and conclusions about what that individual is willing to pay — the “surveillance pricing” framing from the agency’s January 2025 preliminary report — and warns that doing so without adequate disclosure may violate Section 5 of the FTC Act. It is disclosure-focused, not prohibitory, and the agency explicitly disclaims authority to ban the practice outright.
Our read — author analysis, not an FTC position — is that the proposal targets a different risk category than this pattern entirely: person-level differential pricing to consumers, not a deterministic B2B engine computing one price per input configuration. A rate-card engine that produces the same quote for the same configuration regardless of who asks is close to the opposite of surveillance pricing. But the shared theme across the FTC proposal, the EU AI Act, and the practitioner literature is the same: be able to show your work. Where a costing engine sits inside a broader operations plan — capacity, staffing, the rest of the cost base — is a separate exercise we cover in our AI operations costing guide, and pressure-testing a build like this before it faces customers is exactly the kind of engagement our AI transformation practice runs.
08 — ConclusionThe agent that never invents a number. #
Keep the determinism. Add the model as a narrator, nothing more.
The research record is unambiguous about the failure class: models pattern-match arithmetic rather than execute it, the documented 2023-era accuracy cliff was steep, and the 2026 evidence says the dangerous variant — fabricating a plausible value — grows with problem complexity. None of that has to matter to your quote, because none of it needs to be in the path. A deterministic engine over admin-owned rate tables computes every figure; the model explains figures it was handed.
The fence is mechanical and buildable today: forced tool use so a pricing turn cannot resolve without calling the engine, strict schemas so the payload cannot carry an invented field, and an output validator confirming every dollar figure in the prose matches a value in the tool input. Then audit for drift — prose arithmetic, unmatched figures, what-ifs answered inline, confidence language — as a production metric, not a launch checklist.
The market context makes the discipline easy to justify. Rules-based pricing software is a multi-billion-dollar established category that most B2B organizations already run; the LLM layer is the new, risky part, and the regulatory direction — EU AI Act logging, FTC-proposed disclosure expectations — rewards exactly the build that can show its work. The anti-CPQ pitch writes itself: not “AI does the pricing,” but “AI explains pricing a system you control computed.” That is the version a customer can trust and an auditor can verify.