There is a pattern I keep seeing when designing Agentic AI systems.
We start by asking:
These are useful questions.
But they are often asked too early.
The result can be an architecture that is technically impressive but operationally difficult, expensive, slow, and surprisingly hard to trust.
A better approach is to reverse the order:
Start with the product outcome. Define the constraints. Then design the architecture. Choose the tools last.
I have found a useful way to structure those constraints around four dimensions:
LCFE
L — Latency
C — Cost
F — Failure
E — Evaluation
This is not a framework that says every agentic system must look the same. It is a way of forcing architectural decisions to start with the realities of the product rather than the capabilities of the technology.
In this article, I’ll walk through a concrete incident-automation example and show how starting with constraints can completely change the architecture.
Imagine we want to build an AI Incident Resolution Assistant for an engineering organization.
The goal sounds straightforward:
When a production incident is raised, the AI should investigate the incident, gather context, identify the likely cause, recommend or perform remediation, and verify the result.
Now imagine the team starts with the technology.
The first architecture might look like this:
User / Incident
|
v
┌──────────────┐
│ Triage Agent │
└──────┬───────┘
|
v
┌────────────────┐
│ Research Agent │
└───────┬────────┘
|
┌──────────────┼──────────────┐
v v v
Logs Agent Metrics Agent Knowledge Agent
| | |
└──────────────┼──────────────┘
|
v
┌─────────────────┐
│ Remediation │
│ Agent │
└────────┬────────┘
|
v
┌─────────────────┐
│ Validation Agent│
└────────┬────────┘
|
v
Resolution
It looks sophisticated.
We have agents.
We have tools.
We have MCP.
We have orchestration.
We have reasoning.
We have memory.
We have autonomous remediation.
Technically, there is nothing wrong with building this.
The problem is that we don't yet know whether the product needs it.
We have designed the solution before defining the constraints.
Before selecting an LLM or orchestration framework, define the actual outcome.
For our incident system, maybe the product requirement is:
Resolve common production incidents within 10 minutes, while reducing manual engineer effort and keeping high-risk actions under human approval.
That statement is much more useful than:
Build a multi-agent incident-resolution system.
Now we can ask the questions that actually drive architecture.
What response time is acceptable?
For example:
What can a successful resolution cost?
For example:
What happens when something goes wrong?
For example:
How do we know the system is actually useful?
For example:
Now we have something much more valuable than a technology stack.
We have engineering constraints.
This is where the design can change dramatically.
Suppose historical incident data shows:
| Incident Type | Percentage | Typical Resolution |
|---|---|---|
| High CPU | 25% | Scale service |
| Pod crash loop | 20% | Inspect logs + restart |
| Certificate expiry | 10% | Renew certificate |
| Database connection pool | 15% | Restart / tune service |
| Deployment regression | 10% | Roll back |
| Unknown / complex | 20% | Deep investigation |
Suddenly, the idea of having every incident go through a fully autonomous multi-agent workflow looks questionable.
For many incident types, the process is already known.
For example:
Incident
|
v
Classify
|
+---- Known pattern? ---- Yes ----> Deterministic workflow
|
No
|
v
Agentic investigation
This is a major architectural insight:
An agent should exist where reasoning variability exists.
It should not exist simply because the technology makes it possible.
After applying the constraints, our architecture might become:
Incident Event
|
v
┌─────────────────┐
│ Fast Classifier │
└────────┬────────┘
|
┌───────────┴────────────┐
| |
v v
Known Incident Complex / Unknown
| |
v v
Deterministic Workflow Agent Runtime
| |
| ┌─────────┼──────────┐
| v v v
| Logs Tool Metrics Tool KB/Search
| | | |
| └─────────┼──────────┘
| |
| v
| Diagnosis
| |
| v
| Policy / Guardrails
| |
| ┌───────┴────────┐
| | |
| Safe High Risk
| | |
| v v
| Auto Remediation Human Approval
| | |
└────────────────┴────────────────┘
|
v
Verification
|
v
Outcome / Escalation
Notice what disappeared.
We may no longer need:
The system is actually simpler.
But it is also more production-oriented.
Latency is one of the easiest constraints to ignore during an AI prototype.
A demo can take 45 seconds and still look impressive.
A production incident-response system may not have that luxury.
Suppose our requirement is:
P95 time to diagnosis < 30 seconds
Now work backwards.
A possible latency budget:
End-to-end P95 = 30s
Classification 2s
Retrieval 4s
Tool calls 10s
LLM reasoning 10s
Orchestration 2s
Buffer 2s
--------------------------
Total 30s
Now imagine someone proposes adding a reranker.
The reranker adds another 3 seconds.
We should not automatically say:
"Reranking improves retrieval, so let's add it."
Instead ask:
Does reranking produce enough evaluation improvement to justify 3 seconds of our latency budget?
Suppose evaluation shows:
Without reranking:
Diagnosis accuracy = 91%
With reranking:
Diagnosis accuracy = 91.8%
An extra 3 seconds may not be worth it.
But if it changes:
91% → 97%
the architectural decision becomes much easier.
This is the important mindset:
Every piece of complexity has to earn its place in the latency budget.
The same principle applies to:
Agentic systems can become expensive surprisingly quickly.
Imagine a naïve incident workflow:
Triage LLM call
+
Research LLM call
+
Reasoning LLM call
+
Tool-selection LLM call
+
Remediation LLM call
+
Validation LLM call
+
Retries
Now multiply that by thousands of incidents.
The important metric is not:
"How cheap is our LLM call?"
It is:
"How much does it cost us to successfully resolve an incident?"
Consider two architectures.
Every incident
↓
Large model
↓
Multiple agent loops
↓
Multiple tools
↓
Expensive reasoning
Average cost:
£0.80 / incident
Classifier
↓
Known pattern?
↓
Yes → deterministic workflow
No
↓
Stronger model
↓
Agentic investigation
Average cost:
£0.18 / incident
Suppose both achieve similar overall resolution rates.
Architecture B is clearly more attractive.
This naturally leads to a tiered model strategy:
Incoming incident
|
v
Small / cheap model
|
┌────────┴────────┐
| |
Simple Complex
| |
v v
Workflow model Strong model
|
v
Deep reasoning
Use the expensive reasoning capability where it creates measurable value.
Not everywhere.
This is where production systems differ from demos.
A demo assumes:
Input
↓
LLM
↓
Tool
↓
Success
A production system assumes:
Input
↓
LLM
↓
Tool
↓
Timeout
↓
Retry
↓
Malformed response
↓
Validation failure
↓
Fallback
↓
Human escalation
For an incident agent, failure scenarios might include:
The metrics API is unavailable.
The agent should not invent metrics.
The knowledge base returns nothing.
The agent should explicitly recognize missing evidence.
The model produces:
{
"service": "payment-api",
"replicas": "many"
}
But the tool requires an integer.
The system must validate the tool request before execution.
The agent wants to execute:
DROP DATABASE
The system should never treat the LLM's intention as permission.
The permission boundary belongs to the runtime.
An agent can continue:
think → tool → observe → think → tool → observe
without making progress.
So we need:
This leads to a principle that is particularly important for agentic systems:
The model proposes actions. The runtime decides which actions are allowed.
The model should not become the security boundary.
This is perhaps the most important shift when moving from an LLM application to an agent.
For a simple chatbot, evaluation might look like:
Was the answer helpful?
For an incident agent, that is not enough.
We need to evaluate the trajectory and outcome.
For example:
Incident
↓
Correct classification?
↓
Relevant evidence retrieved?
↓
Correct tools selected?
↓
Reasoning supported by evidence?
↓
Safe action selected?
↓
Action executed successfully?
↓
Incident actually resolved?
↓
Outcome correctly verified?
A response such as:
"The issue appears to be high CPU."
may sound intelligent.
But if the service remains unhealthy, the agent hasn't completed the task.
The real evaluation might therefore include:
Did the incident get resolved?
Did the agent choose the appropriate tools?
Did its diagnosis rely on relevant evidence?
Did it avoid disallowed actions?
How many tool calls and model calls were needed?
Does the same scenario succeed repeatedly?
This is why agent evaluation needs to go beyond traditional prompt evaluation.
A useful production evaluation strategy has two layers.
Before deployment:
Benchmark scenarios
↓
Tool-use evaluation
↓
Trajectory evaluation
↓
Safety tests
↓
Regression suite
↓
Release gate
After deployment:
Real traffic
↓
Tracing
↓
Success / failure signals
↓
Production evaluation
↓
New failure cases
↓
Added to benchmark suite
↓
Next release
This creates a feedback loop:
Production failures become future evaluation cases.
That is one of the most important pieces of a mature agentic architecture.
MCP is a good example of why tool-first thinking can be dangerous.
If we start with:
"We have MCP, where can we use it?"
we are already designing backwards.
Instead ask:
"Which capabilities does the product need, and what is the safest and most efficient interface for those capabilities?"
Maybe MCP is a good fit for:
But that doesn't mean every operation needs to pass through MCP.
A deterministic internal workflow may call a service API directly.
A highly sensitive operation may need a tightly controlled internal execution service.
The correct question is not:
"Can MCP do this?"
It is:
"What interface best satisfies the product's latency, cost, failure and security constraints?"
Then choose MCP when it fits.
The same logic applies to orchestration frameworks.
A graph framework can be extremely useful when you need:
But using a graph framework does not automatically make a system more production-ready.
For our incident system, the architecture might contain:
Deterministic workflow
+
Agent runtime
+
Policy layer
+
Tool layer
+
Evaluation layer
+
Observability
LangGraph might be part of the implementation.
Or it might not be.
The architecture should determine the tool choice.
Not the other way around.
Memory is another feature that is often added because an agent "should remember things."
But what should it remember?
For an incident platform, maybe we need:
Current incident context:
Incident ID
Service
Recent logs
Metrics
Recent deployment
Actions already attempted
Current hypothesis
Reusable information:
Known failure patterns
Runbooks
Service documentation
Historical incidents
Operational policies
These are not necessarily the same thing.
And some information should not be treated as permanent truth simply because a previous agent generated it.
That is a failure-mode question:
How stale can memory become before it becomes dangerous?
Again, the constraint drives the architecture.
A common mistake is to treat "human approval" as one generic feature.
Instead, classify actions by risk.
For example:
LOW RISK
Read logs
Read metrics
Read deployment status
↓
Autonomous
MEDIUM RISK
Restart pod
Scale service
Clear cache
↓
Policy-based
HIGH RISK
Rollback production
Change configuration
Database operation
↓
Human approval
Now autonomy becomes a policy decision, not an LLM personality trait.
This is a much more reliable way to build agentic systems.
Notice what happened.
We started with:
Multi-Agent Everything
and ended with:
Incident
|
v
Classification
|
┌───────────┴───────────┐
| |
Known pattern Complex case
| |
v v
Deterministic flow Agent runtime
|
Tools + Retrieval
|
Reasoning
|
Policy / Guardrail
|
┌────────────┴────────────┐
| |
Auto-action Human approval
| |
└────────────┬────────────┘
|
Verification
|
v
Outcome
This architecture has fewer moving parts.
But it has something more important:
Each part exists for a reason.
When starting a new Agentic AI system, I now prefer to work through this sequence.
What business or user outcome are we trying to improve?
Not:
"Build an AI agent."
But:
"Reduce incident resolution time by 40%."
Write down:
Latency:
P95 target
Cost:
Cost per successful outcome
Failure:
Known failure modes + recovery expectations
Evaluation:
How success will be measured
Also consider:
Ask:
Is this deterministic, probabilistic, or mixed?
A useful pattern is:
Deterministic → workflow
Probabilistic → agent
Mixed → workflow + agent
Many real-world systems fall into the third category.
Don't make the whole system autonomous.
Make the right parts autonomous.
Before implementing the happy path, define:
Tool timeout
Invalid output
Low confidence
Stale context
Unauthorized action
Loop detected
Provider unavailable
Missing evidence
If you cannot clearly describe how you'll measure whether the system works, you probably aren't ready to choose the architecture.
Only now ask:
At this stage, technology selection becomes much easier because the problem has already constrained the solution space.
The simplest way I think about this is:
PRODUCT OUTCOME
|
v
CONSTRAINTS
┌─────┬─────┬─────┬─────┐
| | | | |
L C F E Security
| | | | |
└─────┴─────┴─────┴─────┘
|
v
ARCHITECTURE
|
v
IMPLEMENTATION
|
┌────────────┼────────────┐
v v v
Models Tools Frameworks
Compare that with the common approach:
Models
↓
Framework
↓
MCP
↓
Agents
↓
Architecture
↓
"Now let's figure out the constraints."
That second approach is where a lot of unnecessary complexity starts.
The most important lesson isn't that everyone should use LCFE.
It is the order of thinking.
Agentic AI gives engineers an enormous amount of flexibility.
We can create:
But technical possibility is not the same as product value.
A sophisticated architecture can still be the wrong architecture.
The best production system might contain fewer agents, fewer model calls and fewer tools than the original prototype.
And that is not a failure.
It is often a sign that the architecture has finally started responding to the product instead of the technology.
When designing Agentic AI systems, the temptation is to ask:
"What can we build with these tools?"
I think a better question is:
"What is the simplest system that can reliably achieve the required outcome within our constraints?"
Start with:
Outcome → Constraints → Architecture → Tools
Think about:
Latency → Cost → Failure → Evaluation
Then decide where agents, models, MCP, RAG, memory and orchestration actually belong.
Because a production-grade agentic system isn't the one with the most components.
It is the one where every component has earned its place.
And sometimes, the best agentic architecture is the one where you discover that you don't need an agent for half the problem.