{"slug": "stop-designing-agentic-ai-systems-backwards-start-with-constraints-then-choose", "title": "Stop Designing Agentic AI Systems Backwards: Start With Constraints, Then Choose the Architecture", "summary": "A developer argues that agentic AI systems are often designed backwards, starting with technology rather than product constraints. They propose a constraint-first approach structured around latency, cost, failure, and evaluation (LCFE), using an AI incident-resolution assistant as an example to show how starting with constraints can change the architecture.", "body_md": "There is a pattern I keep seeing when designing Agentic AI systems.\n\nWe start by asking:\n\nThese are useful questions.\n\nBut they are often asked **too early**.\n\nThe result can be an architecture that is technically impressive but operationally difficult, expensive, slow, and surprisingly hard to trust.\n\nA better approach is to reverse the order:\n\nStart with the product outcome. Define the constraints. Then design the architecture. Choose the tools last.\n\nI have found a useful way to structure those constraints around four dimensions:\n\n**LCFE**\n\n**L — Latency**\n\n**C — Cost**\n\n**F — Failure**\n\n**E — Evaluation**\n\nThis is not a framework that says every agentic system must look the same. It is a way of forcing architectural decisions to start with the realities of the product rather than the capabilities of the technology.\n\nIn this article, I’ll walk through a concrete incident-automation example and show how starting with constraints can completely change the architecture.\n\nImagine we want to build an **AI Incident Resolution Assistant** for an engineering organization.\n\nThe goal sounds straightforward:\n\nWhen a production incident is raised, the AI should investigate the incident, gather context, identify the likely cause, recommend or perform remediation, and verify the result.\n\nNow imagine the team starts with the technology.\n\nThe first architecture might look like this:\n\n```\n                    User / Incident\n                           |\n                           v\n                    ┌──────────────┐\n                    │ Triage Agent │\n                    └──────┬───────┘\n                           |\n                           v\n                   ┌────────────────┐\n                   │ Research Agent │\n                   └───────┬────────┘\n                           |\n            ┌──────────────┼──────────────┐\n            v              v              v\n        Logs Agent     Metrics Agent   Knowledge Agent\n            |              |              |\n            └──────────────┼──────────────┘\n                           |\n                           v\n                  ┌─────────────────┐\n                  │ Remediation     │\n                  │ Agent           │\n                  └────────┬────────┘\n                           |\n                           v\n                  ┌─────────────────┐\n                  │ Validation Agent│\n                  └────────┬────────┘\n                           |\n                           v\n                       Resolution\n```\n\nIt looks sophisticated.\n\nWe have agents.\n\nWe have tools.\n\nWe have MCP.\n\nWe have orchestration.\n\nWe have reasoning.\n\nWe have memory.\n\nWe have autonomous remediation.\n\nTechnically, there is nothing wrong with building this.\n\nThe problem is that **we don't yet know whether the product needs it**.\n\nWe have designed the solution before defining the constraints.\n\nBefore selecting an LLM or orchestration framework, define the actual outcome.\n\nFor our incident system, maybe the product requirement is:\n\nResolve common production incidents within 10 minutes, while reducing manual engineer effort and keeping high-risk actions under human approval.\n\nThat statement is much more useful than:\n\nBuild a multi-agent incident-resolution system.\n\nNow we can ask the questions that actually drive architecture.\n\nWhat response time is acceptable?\n\nFor example:\n\nWhat can a successful resolution cost?\n\nFor example:\n\nWhat happens when something goes wrong?\n\nFor example:\n\nHow do we know the system is actually useful?\n\nFor example:\n\nNow we have something much more valuable than a technology stack.\n\nWe have **engineering constraints**.\n\nThis is where the design can change dramatically.\n\nSuppose historical incident data shows:\n\n| Incident Type | Percentage | Typical Resolution |\n|---|---|---|\n| High CPU | 25% | Scale service |\n| Pod crash loop | 20% | Inspect logs + restart |\n| Certificate expiry | 10% | Renew certificate |\n| Database connection pool | 15% | Restart / tune service |\n| Deployment regression | 10% | Roll back |\n| Unknown / complex | 20% | Deep investigation |\n\nSuddenly, the idea of having every incident go through a fully autonomous multi-agent workflow looks questionable.\n\nFor many incident types, the process is already known.\n\nFor example:\n\n``` php\nIncident\n   |\n   v\nClassify\n   |\n   +---- Known pattern? ---- Yes ----> Deterministic workflow\n   |\n   No\n   |\n   v\nAgentic investigation\n```\n\nThis is a major architectural insight:\n\nAn agent should exist where reasoning variability exists.\n\nIt should not exist simply because the technology makes it possible.\n\nAfter applying the constraints, our architecture might become:\n\n```\n                    Incident Event\n                           |\n                           v\n                  ┌─────────────────┐\n                  │ Fast Classifier  │\n                  └────────┬────────┘\n                           |\n               ┌───────────┴────────────┐\n               |                        |\n               v                        v\n       Known Incident              Complex / Unknown\n               |                        |\n               v                        v\n      Deterministic Workflow      Agent Runtime\n               |                        |\n               |              ┌─────────┼──────────┐\n               |              v         v          v\n               |           Logs Tool Metrics Tool KB/Search\n               |              |         |          |\n               |              └─────────┼──────────┘\n               |                        |\n               |                        v\n               |                 Diagnosis\n               |                        |\n               |                        v\n               |               Policy / Guardrails\n               |                        |\n               |                ┌───────┴────────┐\n               |                |                |\n               |              Safe            High Risk\n               |                |                |\n               |                v                v\n               |          Auto Remediation   Human Approval\n               |                |                |\n               └────────────────┴────────────────┘\n                                |\n                                v\n                           Verification\n                                |\n                                v\n                         Outcome / Escalation\n```\n\nNotice what disappeared.\n\nWe may no longer need:\n\nThe system is actually **simpler**.\n\nBut it is also more production-oriented.\n\nLatency is one of the easiest constraints to ignore during an AI prototype.\n\nA demo can take 45 seconds and still look impressive.\n\nA production incident-response system may not have that luxury.\n\nSuppose our requirement is:\n\nP95 time to diagnosis < 30 seconds\n\nNow work backwards.\n\nA possible latency budget:\n\n```\nEnd-to-end P95 = 30s\n\nClassification       2s\nRetrieval            4s\nTool calls          10s\nLLM reasoning       10s\nOrchestration        2s\nBuffer                2s\n--------------------------\nTotal                30s\n```\n\nNow imagine someone proposes adding a reranker.\n\nThe reranker adds another 3 seconds.\n\nWe should not automatically say:\n\n\"Reranking improves retrieval, so let's add it.\"\n\nInstead ask:\n\nDoes reranking produce enough evaluation improvement to justify 3 seconds of our latency budget?\n\nSuppose evaluation shows:\n\n```\nWithout reranking:\nDiagnosis accuracy = 91%\n\nWith reranking:\nDiagnosis accuracy = 91.8%\n```\n\nAn extra 3 seconds may not be worth it.\n\nBut if it changes:\n\n```\n91% → 97%\n```\n\nthe architectural decision becomes much easier.\n\nThis is the important mindset:\n\nEvery piece of complexity has to earn its place in the latency budget.\n\nThe same principle applies to:\n\nAgentic systems can become expensive surprisingly quickly.\n\nImagine a naïve incident workflow:\n\n```\nTriage LLM call\n      +\nResearch LLM call\n      +\nReasoning LLM call\n      +\nTool-selection LLM call\n      +\nRemediation LLM call\n      +\nValidation LLM call\n      +\nRetries\n```\n\nNow multiply that by thousands of incidents.\n\nThe important metric is not:\n\n\"How cheap is our LLM call?\"\n\nIt is:\n\n\"How much does it cost us to successfully resolve an incident?\"\n\nConsider two architectures.\n\n```\nEvery incident\n      ↓\nLarge model\n      ↓\nMultiple agent loops\n      ↓\nMultiple tools\n      ↓\nExpensive reasoning\n```\n\nAverage cost:\n\n**£0.80 / incident**\n\n```\nClassifier\n    ↓\nKnown pattern?\n    ↓\nYes → deterministic workflow\n\nNo\n    ↓\nStronger model\n    ↓\nAgentic investigation\n```\n\nAverage cost:\n\n**£0.18 / incident**\n\nSuppose both achieve similar overall resolution rates.\n\nArchitecture B is clearly more attractive.\n\nThis naturally leads to a **tiered model strategy**:\n\n```\n              Incoming incident\n                     |\n                     v\n              Small / cheap model\n                     |\n            ┌────────┴────────┐\n            |                 |\n         Simple            Complex\n            |                 |\n            v                 v\n      Workflow model      Strong model\n                              |\n                              v\n                        Deep reasoning\n```\n\nUse the expensive reasoning capability where it creates measurable value.\n\nNot everywhere.\n\nThis is where production systems differ from demos.\n\nA demo assumes:\n\n```\nInput\n  ↓\nLLM\n  ↓\nTool\n  ↓\nSuccess\n```\n\nA production system assumes:\n\n```\nInput\n  ↓\nLLM\n  ↓\nTool\n  ↓\nTimeout\n  ↓\nRetry\n  ↓\nMalformed response\n  ↓\nValidation failure\n  ↓\nFallback\n  ↓\nHuman escalation\n```\n\nFor an incident agent, failure scenarios might include:\n\nThe metrics API is unavailable.\n\nThe agent should not invent metrics.\n\nThe knowledge base returns nothing.\n\nThe agent should explicitly recognize missing evidence.\n\nThe model produces:\n\n```\n{\n  \"service\": \"payment-api\",\n  \"replicas\": \"many\"\n}\n```\n\nBut the tool requires an integer.\n\nThe system must validate the tool request before execution.\n\nThe agent wants to execute:\n\n```\nDROP DATABASE\n```\n\nThe system should never treat the LLM's intention as permission.\n\nThe permission boundary belongs to the runtime.\n\nAn agent can continue:\n\n```\nthink → tool → observe → think → tool → observe\n```\n\nwithout making progress.\n\nSo we need:\n\nThis leads to a principle that is particularly important for agentic systems:\n\nThe model proposes actions. The runtime decides which actions are allowed.\n\nThe model should not become the security boundary.\n\nThis is perhaps the most important shift when moving from an LLM application to an agent.\n\nFor a simple chatbot, evaluation might look like:\n\nWas the answer helpful?\n\nFor an incident agent, that is not enough.\n\nWe need to evaluate the **trajectory and outcome**.\n\nFor example:\n\n```\nIncident\n   ↓\nCorrect classification?\n   ↓\nRelevant evidence retrieved?\n   ↓\nCorrect tools selected?\n   ↓\nReasoning supported by evidence?\n   ↓\nSafe action selected?\n   ↓\nAction executed successfully?\n   ↓\nIncident actually resolved?\n   ↓\nOutcome correctly verified?\n```\n\nA response such as:\n\n\"The issue appears to be high CPU.\"\n\nmay sound intelligent.\n\nBut if the service remains unhealthy, the agent hasn't completed the task.\n\nThe real evaluation might therefore include:\n\nDid the incident get resolved?\n\nDid the agent choose the appropriate tools?\n\nDid its diagnosis rely on relevant evidence?\n\nDid it avoid disallowed actions?\n\nHow many tool calls and model calls were needed?\n\nDoes the same scenario succeed repeatedly?\n\nThis is why agent evaluation needs to go beyond traditional prompt evaluation.\n\nA useful production evaluation strategy has two layers.\n\nBefore deployment:\n\n```\nBenchmark scenarios\n       ↓\nTool-use evaluation\n       ↓\nTrajectory evaluation\n       ↓\nSafety tests\n       ↓\nRegression suite\n       ↓\nRelease gate\n```\n\nAfter deployment:\n\n```\nReal traffic\n    ↓\nTracing\n    ↓\nSuccess / failure signals\n    ↓\nProduction evaluation\n    ↓\nNew failure cases\n    ↓\nAdded to benchmark suite\n    ↓\nNext release\n```\n\nThis creates a feedback loop:\n\nProduction failures become future evaluation cases.\n\nThat is one of the most important pieces of a mature agentic architecture.\n\nMCP is a good example of why tool-first thinking can be dangerous.\n\nIf we start with:\n\n\"We have MCP, where can we use it?\"\n\nwe are already designing backwards.\n\nInstead ask:\n\n\"Which capabilities does the product need, and what is the safest and most efficient interface for those capabilities?\"\n\nMaybe MCP is a good fit for:\n\nBut that doesn't mean every operation needs to pass through MCP.\n\nA deterministic internal workflow may call a service API directly.\n\nA highly sensitive operation may need a tightly controlled internal execution service.\n\nThe correct question is not:\n\n\"Can MCP do this?\"\n\nIt is:\n\n\"What interface best satisfies the product's latency, cost, failure and security constraints?\"\n\nThen choose MCP when it fits.\n\nThe same logic applies to orchestration frameworks.\n\nA graph framework can be extremely useful when you need:\n\nBut using a graph framework does not automatically make a system more production-ready.\n\nFor our incident system, the architecture might contain:\n\n```\nDeterministic workflow\n        +\nAgent runtime\n        +\nPolicy layer\n        +\nTool layer\n        +\nEvaluation layer\n        +\nObservability\n```\n\nLangGraph might be part of the implementation.\n\nOr it might not be.\n\nThe architecture should determine the tool choice.\n\nNot the other way around.\n\nMemory is another feature that is often added because an agent \"should remember things.\"\n\nBut what should it remember?\n\nFor an incident platform, maybe we need:\n\nCurrent incident context:\n\n```\nIncident ID\nService\nRecent logs\nMetrics\nRecent deployment\nActions already attempted\nCurrent hypothesis\n```\n\nReusable information:\n\n```\nKnown failure patterns\nRunbooks\nService documentation\nHistorical incidents\nOperational policies\n```\n\nThese are not necessarily the same thing.\n\nAnd some information should not be treated as permanent truth simply because a previous agent generated it.\n\nThat is a failure-mode question:\n\nHow stale can memory become before it becomes dangerous?\n\nAgain, the constraint drives the architecture.\n\nA common mistake is to treat \"human approval\" as one generic feature.\n\nInstead, classify actions by risk.\n\nFor example:\n\n```\nLOW RISK\nRead logs\nRead metrics\nRead deployment status\n        ↓\nAutonomous\n\nMEDIUM RISK\nRestart pod\nScale service\nClear cache\n        ↓\nPolicy-based\n\nHIGH RISK\nRollback production\nChange configuration\nDatabase operation\n        ↓\nHuman approval\n```\n\nNow autonomy becomes a **policy decision**, not an LLM personality trait.\n\nThis is a much more reliable way to build agentic systems.\n\nNotice what happened.\n\nWe started with:\n\n```\nMulti-Agent Everything\n```\n\nand ended with:\n\n```\n                      Incident\n                          |\n                          v\n                    Classification\n                          |\n              ┌───────────┴───────────┐\n              |                       |\n        Known pattern             Complex case\n              |                       |\n              v                       v\n      Deterministic flow        Agent runtime\n                                      |\n                              Tools + Retrieval\n                                      |\n                                  Reasoning\n                                      |\n                              Policy / Guardrail\n                                      |\n                         ┌────────────┴────────────┐\n                         |                         |\n                   Auto-action              Human approval\n                         |                         |\n                         └────────────┬────────────┘\n                                      |\n                                  Verification\n                                      |\n                                      v\n                                   Outcome\n```\n\nThis architecture has fewer moving parts.\n\nBut it has something more important:\n\n**Each part exists for a reason.**\n\nWhen starting a new Agentic AI system, I now prefer to work through this sequence.\n\nWhat business or user outcome are we trying to improve?\n\nNot:\n\n\"Build an AI agent.\"\n\nBut:\n\n\"Reduce incident resolution time by 40%.\"\n\nWrite down:\n\n```\nLatency:\nP95 target\n\nCost:\nCost per successful outcome\n\nFailure:\nKnown failure modes + recovery expectations\n\nEvaluation:\nHow success will be measured\n```\n\nAlso consider:\n\nAsk:\n\nIs this deterministic, probabilistic, or mixed?\n\nA useful pattern is:\n\n```\nDeterministic → workflow\n\nProbabilistic → agent\n\nMixed → workflow + agent\n```\n\nMany real-world systems fall into the third category.\n\nDon't make the whole system autonomous.\n\nMake the **right parts** autonomous.\n\nBefore implementing the happy path, define:\n\n```\nTool timeout\nInvalid output\nLow confidence\nStale context\nUnauthorized action\nLoop detected\nProvider unavailable\nMissing evidence\n```\n\nIf you cannot clearly describe how you'll measure whether the system works, you probably aren't ready to choose the architecture.\n\nOnly now ask:\n\nAt this stage, technology selection becomes much easier because the problem has already constrained the solution space.\n\nThe simplest way I think about this is:\n\n```\n             PRODUCT OUTCOME\n                    |\n                    v\n              CONSTRAINTS\n          ┌─────┬─────┬─────┬─────┐\n          |     |     |     |     |\n          L     C     F     E   Security\n          |     |     |     |     |\n          └─────┴─────┴─────┴─────┘\n                    |\n                    v\n                ARCHITECTURE\n                    |\n                    v\n             IMPLEMENTATION\n                    |\n       ┌────────────┼────────────┐\n       v            v            v\n     Models        Tools      Frameworks\n```\n\nCompare that with the common approach:\n\n```\nModels\n  ↓\nFramework\n  ↓\nMCP\n  ↓\nAgents\n  ↓\nArchitecture\n  ↓\n\"Now let's figure out the constraints.\"\n```\n\nThat second approach is where a lot of unnecessary complexity starts.\n\nThe most important lesson isn't that everyone should use LCFE.\n\nIt is the **order of thinking**.\n\nAgentic AI gives engineers an enormous amount of flexibility.\n\nWe can create:\n\nBut technical possibility is not the same as product value.\n\nA sophisticated architecture can still be the wrong architecture.\n\nThe best production system might contain fewer agents, fewer model calls and fewer tools than the original prototype.\n\nAnd that is not a failure.\n\nIt is often a sign that the architecture has finally started responding to the product instead of the technology.\n\nWhen designing Agentic AI systems, the temptation is to ask:\n\n\"What can we build with these tools?\"\n\nI think a better question is:\n\n\"What is the simplest system that can reliably achieve the required outcome within our constraints?\"\n\nStart with:\n\n**Outcome → Constraints → Architecture → Tools**\n\nThink about:\n\n**Latency → Cost → Failure → Evaluation**\n\nThen decide where agents, models, MCP, RAG, memory and orchestration actually belong.\n\nBecause a production-grade agentic system isn't the one with the most components.\n\nIt is the one where **every component has earned its place**.\n\nAnd sometimes, the best agentic architecture is the one where you discover that you don't need an agent for half the problem.", "url": "https://wpnews.pro/news/stop-designing-agentic-ai-systems-backwards-start-with-constraints-then-choose", "canonical_source": "https://dev.to/himanjan/stop-designing-agentic-ai-systems-backwards-start-with-constraints-then-choose-the-architecture-5125", "published_at": "2026-08-26 21:51:56+00:00", "updated_at": "2026-08-26 22:19:45.787020+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-products", "ai-infrastructure", "developer-tools"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/stop-designing-agentic-ai-systems-backwards-start-with-constraints-then-choose", "markdown": "https://wpnews.pro/news/stop-designing-agentic-ai-systems-backwards-start-with-constraints-then-choose.md", "text": "https://wpnews.pro/news/stop-designing-agentic-ai-systems-backwards-start-with-constraints-then-choose.txt", "jsonld": "https://wpnews.pro/news/stop-designing-agentic-ai-systems-backwards-start-with-constraints-then-choose.jsonld"}}