{"slug": "drawing-the-line-what-deserves-an-llm-and-what-doesn-t", "title": "Drawing the Line: What Deserves an LLM and What Doesn't", "summary": "A developer building an LLM-powered support agent outlines a decision framework for determining which components should use AI versus deterministic software, emphasizing that AI interprets intent while software enforces policy. The approach uses a static risk-tier lookup to prevent prompt injection from reclassifying actions, and retrieval is treated as a hybrid where fuzzy results sit behind a hard, testable contract.", "body_md": "*The three questions I use to decide what stays deterministic*\n\nPart 2 of an ongoing experiment: building an LLM-powered support agent with deterministic boundaries. The\n\n[companion repo]grows with the series.\n\nEvery AI feature reaches the same fork. This piece here — does it get a model, or does it get a method?\n\nGo wrong one way and you've built a rules engine that can't read a sentence. Go wrong the other and a language model is deciding whether someone gets their money back.\n\n\"The AI interprets intent, software enforces policy\" is easy to say — it's the principle this series [started from](https://dev.to/tonal/an-agent-on-a-leash-or-why-my-ai-agent-doesnt-make-business-decisions-1o1). Applying it to a specific component on a specific Tuesday is the hard part.\n\nThis post is the ruler I use for that.\n\nFor every component, in order:\n\n``` php\nflowchart LR\n    A[\"New component\"] --> B{\"Is the answer a fact?\"}\n    B -- \"no\" --> AI[\"AI\"]\n    B -- \"yes\" --> C{\"Costs money or trust?\"}\n    C -- \"no\" --> P[\"Either\"]\n    C -- \"yes\" --> D{\"Can a test pin it today?\"}\n    D -- \"yes\" --> SW[\"Deterministic software\"]\n    D -- \"no\" --> HY[\"Software contract around AI\"]\n    classDef box fill:#eef2f6,stroke:#8fa3b8,color:#24313f\n    classDef decision fill:#f7f4ec,stroke:#b3a988,color:#24313f\n    classDef ai fill:#eef0f4,stroke:#8fa3b8,color:#24313f\n    class A,AI,P,HY box\n    class SW box\n    class B,C,D decision\n```\n\nApplied to Post 1's components: intent interpretation goes to the AI. Refund eligibility to software. Retrieval targets end up hybrid (the interesting case, below). Refund execution is software plus a human gate. Policy exceptions are a job for a rule engine.\n\nRetrieval deserves its own paragraph, because it's where most teams get tripped up. \"Let the AI find the right knowledge base article\" sounds like an AI decision. It isn't — at least not entirely.\n\nThe *results* are fuzzy: similarity search returns plausible articles, ranked, sometimes wrong. You cannot write `assert(search(\"refund\") == refundsArticle)`\n\nand mean it.\n\nBut the *call* is rigid: which function runs, with what arguments, against which index. That part is plain software with a typed signature, and it's fully unit-testable:\n\n```\n// The results are probabilistic. The invocation isn't.\npublic interface KnowledgeBase {\n    /** Returns up to k articles ranked by semantic relevance.\n     *  Ranking quality is evaluated statistically, never asserted exactly. */\n    List<KnowledgeArticle> search(Query query);\n}\n```\n\nThat split — fuzzy contents behind a hard contract — is what makes retrieval safe to hand to the model as a *tool*. The agent decides when to search; it never gets to redefine what searching means.\n\nHere's the part that took me longest to appreciate. Deciding who decides is itself a decision — so who makes *that* one?\n\nIf the answer is \"the LLM classifies each action's risk tier at runtime,\" the whole architecture collapses: the boundary becomes another probabilistic output that can be talked into moving. Prompt injection doesn't need to break a rule if it can reclassify the action the rule applies to.\n\nSo in this system, the classification of every action is a static lookup — code, not judgment:\n\n```\n// dev/tonal/support/domain/RiskPolicy.java\npublic enum RiskTier { LOW, MEDIUM, HIGH, VERY_HIGH }\n\npublic final class RiskPolicy {\n\n    private static final Map<ActionType, RiskTier> TIERS = Map.of(\n            ActionType.SUMMARIZE_TICKET,    RiskTier.LOW,\n            ActionType.DRAFT_RESPONSE,      RiskTier.LOW,\n            ActionType.CLASSIFY_TICKET,     RiskTier.MEDIUM,\n            ActionType.PROCESS_REFUND,      RiskTier.HIGH,\n            ActionType.MODIFY_ORDER,        RiskTier.HIGH,\n            ActionType.UPDATE_PERMISSIONS,  RiskTier.HIGH,\n            ActionType.CANCEL_SUBSCRIPTION, RiskTier.VERY_HIGH,\n            ActionType.DELETE_DATA,         RiskTier.VERY_HIGH);\n\n    public static RiskTier tierFor(ActionType action) {\n        return TIERS.get(action); // null = unclassified = fails closed\n    }\n}\n```\n\nTwo deliberate choices in there:\n\n`Map.of`\n\nover clever logic.And the tests are about the map itself, not just lookups:\n\n```\n@Test\nvoid everyActionMustHaveATier() {\n    for (ActionType action : ActionType.values()) {\n        assertThat(RiskPolicy.tierFor(action))\n                .as(\"action %s must be classified\", action)\n                .isNotNull();\n    }\n}\n\n@Test\nvoid destructiveActionsAreNeverLowRisk() {\n    assertThat(RiskPolicy.tierFor(ActionType.DELETE_DATA)).isEqualTo(RiskTier.VERY_HIGH);\n    assertThat(RiskPolicy.tierFor(ActionType.CANCEL_SUBSCRIPTION)).isEqualTo(RiskTier.VERY_HIGH);\n}\n```\n\nThe first test is the important one: it pins exhaustiveness. Nobody can silently add an action type next sprint and forget to give it a risk tier — CI fails until they classify it. The boundary enforces itself.\n\nThe same three questions classify decisions in any domain where models meet consequences: clinical triage systems route judgment but never prescribe (fact, high cost); loan underwriting separates scoring models from disbursement logic; industrial safety controllers treat perception as input but interlocks as law. Wherever you look, the durable systems aren't the ones with the smartest model — they're the ones where nobody had to trust the model on a question that has a right answer.\n\nPrevious posts in this series:", "url": "https://wpnews.pro/news/drawing-the-line-what-deserves-an-llm-and-what-doesn-t", "canonical_source": "https://dev.to/tonal/drawing-the-line-what-deserves-an-llm-and-what-doesnt-11lj", "published_at": "2026-08-27 10:56:48+00:00", "updated_at": "2026-08-27 11:19:02.875134+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-safety", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/drawing-the-line-what-deserves-an-llm-and-what-doesn-t", "markdown": "https://wpnews.pro/news/drawing-the-line-what-deserves-an-llm-and-what-doesn-t.md", "text": "https://wpnews.pro/news/drawing-the-line-what-deserves-an-llm-and-what-doesn-t.txt", "jsonld": "https://wpnews.pro/news/drawing-the-line-what-deserves-an-llm-and-what-doesn-t.jsonld"}}