{"slug": "a-practical-framework-for-deciding-when-not-to-use-an-llm", "title": "A Practical Framework for Deciding When NOT to Use an LLM", "summary": "A developer published a practical framework for deciding when not to use a large language model, arguing that tasks solvable with deterministic rules, SQL queries, standard APIs, search algorithms, or traditional machine learning should avoid LLM inference to prevent added cost, latency, security exposure, and maintenance complexity. The framework evaluates five factors — task complexity, correctness requirements, latency, total operating cost, and operational risk — and recommends decomposing workflows like invoice processing so that extraction may use OCR or an LLM while tax calculations, purchase-order matching, and refund decisions stay in deterministic business logic, decimal arithmetic, and rules engines.", "body_md": "Not every software problem needs a large language model (LLM). If a task can be solved reliably using deterministic rules, SQL queries, standard APIs, search algorithms, or traditional machine learning, adding an LLM may increase cost, latency, security exposure, and maintenance complexity without improving the outcome.\n\nEngineering leaders should avoid using an LLM when the system requires exact calculations, predictable outputs, strict execution guarantees, very low latency, or decisions that cannot tolerate unverified results.\n\nEvaluate five factors before selecting an LLM: task complexity, correctness requirements, latency, total operating cost, and operational risk. Use an LLM when understanding or generating flexible natural language creates measurable value that simpler technologies cannot provide efficiently.\n\nOne common architectural mistake is selecting the technology before defining the task. Invoice processing, for example, involves document extraction, field validation, tax calculations, purchase-order matching, discrepancy detection, and approval. Those operations do not necessarily require the same technology.\n\nDocument extraction might benefit from OCR, document AI, or an LLM when layouts are highly variable. Tax calculations should use deterministic business logic. Purchase-order matching can use queries and predefined rules. Unusual discrepancies may require statistical anomaly detection or human review.\n\nAsk first: Does this task require language understanding or generation, or does it simply require executing known business logic? If the answer is known business logic, start with conventional software.\n\nFinancial reconciliation, tax computation, inventory balances, and contractual pricing should rely on validated algorithms and appropriate numerical representations. An LLM can interpret a request, but calculations should remain deterministic.\n\nfrom decimal import Decimal\n\ndef calculate_discount(price, rate):\n\n    price = Decimal(str(price))\n\n    rate = Decimal(str(rate))\n\n    if price < 0 or not 0 <= rate <= 1:\n\n        raise ValueError(\"Invalid price or discount rate\")\n\n    return (price * (1 - rate)).quantize(Decimal(\"0.01\"))\n\nprint(calculate_discount(\"250.00\", \"0.15\")) # 212.50\n\nUse instead: Validated business logic, decimal arithmetic, calculation libraries, or specialized numerical systems.\n\nA refund policy with explicit eligibility periods, excluded categories, and approval thresholds is better represented as a rules engine or conventional application logic.\n\ndef refund_decision(days_since_purchase, amount, eligible):\n\n    if not eligible or days_since_purchase > 30:\n\n        return \"reject\"\n\n    if amount > 500:\n\n        return \"manual_review\"\n\n    return \"approve\"\n\nThis simplified example is not a complete refund policy. A production system would also need authentication, transaction validation, fraud controls, and audit records. Use instead: Rules engines, state machines, or workflow orchestration.\n\nLLM inference can add network, input-processing, generation, and tool-call delays. It may be inappropriate for industrial control loops, low-level network packet handling, high-frequency transaction processing, and safety-critical device control. Use instead: Optimized algorithms, embedded logic, or specialized low-latency models with measured timing guarantees.\n\nIf revenue figures already exist in an authoritative database, query that system directly rather than asking a model to generate the result.\n\nSELECT SUM(amount) AS total_revenue\n\nFROM paid_invoices\n\nWHERE paid_at >= '2026-09-01'\n\n  AND paid_at < '2026-10-01';\n\nThis query assumes revenue means paid invoice amounts; financial reporting may require different accounting definitions. An LLM may translate a user question into a constrained query, but authorization and aggregation should remain under application control.\n\nDo not introduce an LLM if the implementation cannot satisfy data handling, retention, access control, and deployment requirements. Private deployments and governed managed services can support sensitive workloads, but controls must be demonstrated. OWASP identifies prompt injection and sensitive information disclosure among important LLM application risks.\n\nFraud detection, demand forecasting, defect classification, anomaly detection, and predictive maintenance may be better served by statistical methods or task-specific machine learning. An LLM can explain the result without generating the underlying prediction. Compare approaches on representative data.\n\nDo not approve autonomous production use without representative test cases, defined unacceptable outcomes, and an evaluation or review mechanism. A controlled prototype or human-reviewed workflow is preferable until measurable acceptance criteria exist.\n\nRequirement\n\nUsually preferred approach\n\nReason\n\nExact arithmetic\n\nDeterministic code\n\nDefined, testable numerical behavior\n\nBusiness rule enforcement\n\nRules engine\n\nPredictable and auditable decisions\n\nStructured data retrieval\n\nSQL or APIs\n\nAuthoritative records\n\nKeyword or exact-match search\n\nSearch engine\n\nEfficient retrieval without generation\n\nDemand forecasting\n\nStatistical or ML models\n\nOptimized for prediction\n\nFlexible document summarization\n\nLLM\n\nVariable natural language\n\nSemantic document discovery\n\nEmbeddings and retrieval\n\nConceptually related content\n\nNatural-language interface\n\nLLM plus controlled tools\n\nIntent translated into validated operations\n\nHigh-risk automated actions\n\nDeterministic controls and oversight\n\nPermission and execution boundaries\n\nThese are starting points, not universal rules. Benchmark against your own acceptance criteria.\n\nTotal Cost of Ownership = Development + Infrastructure + Inference + Evaluation + Monitoring + Maintenance + Exception Handling.\n\nCost per successful task = Total operational cost / Successfully completed tasks.\n\nConsider a hypothetical workload of one million requests per month:\n\nMetric\n\nDeterministic service\n\nLLM-based service\n\nCost per attempt\n\n$0.001\n\n$0.02\n\nMonthly attempts\n\n1,000,000\n\n1,000,000\n\nDirect processing cost\n\n$1,000\n\n$20,000\n\nSuccessful task rate\n\n99.9%\n\n97%\n\nDirect cost per successful task\n\n~$0.001\n\n~$0.0206\n\nIllustrative scenario only, not observed vendor prices or published benchmarks. The systems may solve different problems, and a deterministic service might not handle some language-intensive tasks. Compare time to deployment, cost per completed task, failure rates, review requirements, maintenance, monitoring, and security overhead.\n\nIf the task does not require flexible language interpretation or generation, begin with deterministic software.\n\nBenchmark rules, search, APIs, or specialized ML. Do not compare an LLM only against doing nothing.\n\nDefine how incorrect, incomplete, or inconsistent results are detected and handled.\n\nTest latency, throughput, cost, availability, privacy, and security under realistic workloads.\n\nApprove the LLM only when its benefits are material and risks can be managed.\n\nDecision flow: Require language intelligence? If no, use deterministic software. If yes, benchmark simpler alternatives. If a simpler option meets acceptance criteria, prefer it. Otherwise, test error containment, operational constraints, and measurable incremental value before approving controlled LLM implementation.\n\nThis five-gate framework is a proposed engineering method, not an externally validated standard.\n\nA SaaS platform needs to route requests such as duplicate charges, account-email changes, and API errors. Three architectures are possible.\n\ndef route_ticket(message):\n\n    text = message.lower()\n\n    if \"charged twice\" in text or \"duplicate charge\" in text:\n\n        return \"billing\"\n\n    if \"change email\" in text or \"update email\" in text:\n\n        return \"account\"\n\n    if \"api error\" in text or \"api is returning errors\" in text:\n\n        return \"technical\"\n\n    return \"manual_review\"\n\nThis is transparent and testable but has limited coverage when customers use unexpected wording or describe multiple issues.\n\nAn LLM can interpret varied expressions, but adds inference costs, model evaluation, output validation, and additional failure modes.\n\nUse explicit rules for well-defined requests, a lightweight classifier for common variations, an LLM for ambiguous language, and human review for low-confidence or high-impact cases. The routing policy remains under application control.\n\nCompare classification precision and recall, misrouting rate, manual-review rate, P95 latency, cost per correctly routed ticket, and maintenance effort. A hybrid system is preferable only if it provides measurable benefits.\n\nSeparate language interpretation from authoritative business execution. An LLM can translate intent into a structured request; deterministic services should validate schemas, enforce permissions, apply business rules, and execute critical operations.\n\nUser request\n\n    |\n\nLLM interprets intent\n\n    |\n\nStructured request\n\n    |\n\nSchema validation\n\n    |\n\nAuthorization + business rules\n\n    |\n\nDeterministic API execution\n\n    |\n\nVerified result\n\n    |\n\nUser response\n\nStructured outputs can constrain response format, but valid syntax does not guarantee correct interpretation or authorized execution. This is especially important for AI agents connected to enterprise systems.\n\nProblem definition: What requires an LLM, and what alternatives were considered?\n\nBaseline: How well does the strongest non-LLM approach perform?\n\nQuality: What dataset and metrics evaluate correctness?\n\nReliability: What happens if the model fails or returns invalid output?\n\nSecurity: Can the model access or change anything beyond user authorization?\n\nPerformance: What are P50, P95, and P99 latencies under load?\n\nEconomics: What is the cost per successful business task?\n\nOwnership: Who maintains prompts, datasets, configurations, and monitoring?\n\nChange management: How will model and data updates be retested?\n\nExit strategy: Can the model be replaced without redesigning the application?\n\nWhen simpler approaches meet the requirements with less complexity, or when critical model errors cannot be detected, contained, or reviewed.\n\nNot universally. LLMs suit flexible language tasks, while specialized models may be more efficient for structured prediction and forecasting.\n\nThey can interpret requests, but authoritative rules, calculations, permissions, and transactions should generally remain deterministic.\n\nIt can translate natural language into a validated query, but the database should execute structured retrieval and aggregation.\n\nWhen flexible language interpretation adds value but critical steps need predictable execution and strong controls.\n\nUse comparative evidence that a simpler architecture meets quality, latency, security, and operational requirements at lower total cost.\n\nThe decision to use an LLM should be based on engineering requirements, not technology momentum. Large language models can provide valuable capabilities for flexible language and unstructured information, but they are not the right answer to every software problem.\n\nBefore introducing an LLM, establish a non-LLM baseline, define measurable acceptance criteria, and test whether the model delivers enough additional value to justify its cost and risk.\n\nA useful rule for architecture reviews: If removing the LLM does not materially reduce the system’s ability to meet its requirements, the LLM probably does not belong in the critical path.\n\nThe goal is to build the simplest system that reliably delivers the required business outcome.", "url": "https://wpnews.pro/news/a-practical-framework-for-deciding-when-not-to-use-an-llm", "canonical_source": "https://dev.to/olivia-white/a-practical-framework-for-deciding-when-not-to-use-an-llm-2hp8", "published_at": "2026-10-15 14:35:53+00:00", "updated_at": "2026-10-15 14:52:34.335971+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "mlops", "developer-tools"], "entities": [], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/a-practical-framework-for-deciding-when-not-to-use-an-llm", "markdown": "https://wpnews.pro/news/a-practical-framework-for-deciding-when-not-to-use-an-llm.md", "text": "https://wpnews.pro/news/a-practical-framework-for-deciding-when-not-to-use-an-llm.txt", "jsonld": "https://wpnews.pro/news/a-practical-framework-for-deciding-when-not-to-use-an-llm.jsonld"}}