{"slug": "stop-letting-your-llm-make-the-final-call-on-your-data", "title": "Stop letting your LLM make the final call on your data", "summary": "A developer argues that LLMs should not make final compliance judgments on data, proposing a split architecture where LLMs handle extraction and deterministic Python code handles rule enforcement. The approach ensures verifiability and auditability, with a concrete example of fixing an age-related bug by deriving the 'minor' flag from extracted data rather than relying on the model.", "body_md": "# Stop letting your LLM make the final call on your data\n\n[Claude](/en/tags/claude/)or GPT-4 and ask, \"Does this comply?\"\n\nSure, it works most of the time. But for an audit tool, \"most of the time\" is a failure.\n\nThe problem is that when the LLM produces the verdict, the verdict inherits all the flaws of the LLM. It's non-deterministic; a plan might pass on Monday and get flagged on Tuesday. It's unfalsifiable because you can't see the actual threshold the model used—only a probability distribution. Most importantly, it's unauditable. If you tweak the prompt to fix one edge case, you have no way of knowing what other \"judgments\" shifted in the process. You aren't getting evidence; you're getting a second opinion with no paper trail.\n\nMy solution was a hard constraint: no LLM call is allowed in the judgment path.\n\n## Splitting the workflow into Zones\n\nThe trick is to separate language tasks from logic tasks. LLMs are incredible at extraction but mediocre at strict rule enforcement.\n\n**Zone A (Probabilistic):** This is where the LLM lives. It takes free-form English and extracts structured data (e.g., \"3 sets, 8-10 reps, 75% of 1RM\").**Zone B (Deterministic):** This is where Python lives. It takes that structured data and compares it against a hard-coded range (e.g., is 75% between 70% and 85%?).\n\nThe architecture looks like this:\n\n`Free Text`\n\n→ `[LLM Extraction]`\n\n→ `Structured Data`\n\n→ `[Python Rules]`\n\n→ `Verdict`\n\n## When the boundary leaks\n\nIn a real-world AI workflow, this boundary leaks constantly. I hit a bug where the extractor correctly pulled `age_years: 14`\n\nfrom a request, but the routing logic failed. I had asked the model to also provide a `minor`\n\nboolean. Since the input text didn't explicitly use the word \"minor,\" the model left it null. The system then cheerfully evaluated a 14-year-old's plan against adult guidelines.\n\nThe fix was a simple line of code, not a better prompt:\n\n```\n# Age is a number we already have. Do not ask the model\n# to also tell us what that number means.\nif plan.get(\"age_years\") is not None and plan[\"age_years\"] < 18:\n    plan[\"minor\"] = True\n```\n\nThe lesson here is that any time you \"hope\" a model will infer something that follows mechanically from data you already possess, your boundary is in the wrong place.\n\nThis approach is objectively slower to build. Writing a \"check this\" prompt takes an afternoon; building a structured extraction pipeline and a rule engine takes weeks. But the result is a system that is actually verifiable. If a plan is rejected, you can point to the exact line of code and the exact extracted value that triggered the fail. That's the only way to build a professional-grade LLM agent system that people can actually trust.\n\n[Next Claude 3.5 Sonnet beats GPT-4o at designing complex file systems →](/en/threads/5690/)", "url": "https://wpnews.pro/news/stop-letting-your-llm-make-the-final-call-on-your-data", "canonical_source": "https://promptcube3.com/en/threads/5691/", "published_at": "2026-08-09 16:14:18+00:00", "updated_at": "2026-08-09 16:21:21.915119+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-safety"], "entities": ["Claude", "GPT-4"], "alternates": {"html": "https://wpnews.pro/news/stop-letting-your-llm-make-the-final-call-on-your-data", "markdown": "https://wpnews.pro/news/stop-letting-your-llm-make-the-final-call-on-your-data.md", "text": "https://wpnews.pro/news/stop-letting-your-llm-make-the-final-call-on-your-data.txt", "jsonld": "https://wpnews.pro/news/stop-letting-your-llm-make-the-final-call-on-your-data.jsonld"}}