# The More Powerful the AI, the More the Architecture Matters

> Source: <https://dev.to/tonal/-446k>
> Published: 2026-09-19 12:37:01+00:00

*The boundaries I designed, the gaps I haven't solved, and why the difference matters*

Part 14 findings of an experiment: building an LLM-powered support agent with deterministic boundaries. The [companion repo](https://github.com/antoniolopescorreia/reliable-ai-support) contains the full code.

If someone asked me over a coffee what I've been building, the short version is this.

**A support agent that can work out whether you're owed a refund, and cannot give you one.**

The AI reads the message and finds the documents. Software decides eligibility, owns the risk tiers, scopes every lookup to whoever is actually logged in, and parks anything expensive in a queue for a human. Fourteen posts, one repo, and the whole thing runs with no API key.

``` php
flowchart LR
    M["Customer message"] --> AI["AI: intent, retrieval"]
    AI --> SW["Software: scoping,<br/>eligibility, risk tier"]
    SW --> G{"Gate"}
    G -->|"LOW"| X["Runs"]
    G -->|"MEDIUM to HIGH"| H["Human approves"]
    G -->|"VERY HIGH"| P["Proposed only,<br/>a person executes"]
    classDef step fill:#eef2f6,stroke:#8fa3b8,color:#24313f
    classDef decision fill:#f7f4ec,stroke:#b3a988,color:#24313f
    classDef good fill:#ecf2ed,stroke:#93b39d,color:#3d5344
    class M,AI,X step
    class SW,H,P good
    class G decision
```

Every guarantee in there has a test that fails when it stops being true — the gate, the scoping, the risk tiers. The one I'd argue about longest is the dullest of them:

```
/** The only packages a domain class may reference: itself and the JDK. */
private static final String[] DOMAIN_ALLOWLIST = {
        "dev.tonal.support.domain..", "java..", "javax.."
};

@Test
void domainDependsOnlyOnItselfAndTheJdk() {
    classes().that().resideInAPackage("dev.tonal.support.domain..")
            .should().onlyDependOnClassesThat()
            .resideInAnyPackage(DOMAIN_ALLOWLIST)
            .check(production);
}
```

An allowlist, not a blacklist: the business rules may reference themselves and the JDK, and nothing else. No enumerating of forbidden layers, which would need updating every time I add a package. Add a JSON library to a policy class and the build goes red — I checked, by breaking it on purpose and watching it fail.

It exists because this kind of architecture decays quietly. Nobody announces the afternoon they import a framework into the business rules. It just happens, and a few months later "the AI doesn't decide business questions" isn't true any more — and nobody noticed the day it stopped being true.

**A sentence in a README can't catch that. A failing build can.**

Quite a lot. The interesting question is which parts were decisions.

Multi-tenancy was. Per-tenant risk policies mean the tier lookup becomes runtime configuration, and "who gets to decide" is the last thing I want configurable by a form. I'd want evidence that a second tenant genuinely needs different tiers before I'd trade that away.

Real persistence was too, though it sounds lazier. The approval queue and the audit trail are ports with in-memory adapters behind them, and the ports already spell out what a real implementation owes you: durability, notification, permanence. Writing that adapter is an afternoon. Deciding what it must guarantee was the part worth the thinking.

And no LLM-as-judge in the eval suite, which I keep being tempted by. A model grading a model has its own variance and its own bad days, so it would need its own eval before I'd believe a word of its verdicts. That's a project, not a checkbox.

**There's a difference between "I decided not to" and "I haven't got to it".**

Saying which is which out loud is the only thing stopping every gap from becoming retroactive wisdom.

Nothing rate-limits proposals, so somebody patient could bury a reviewer in plausible-looking refund requests until approving becomes reflex. An approved refund executed twice after a retry isn't prevented. And nothing notices if the knowledge base quietly gets worse while every test stays green. Those sit in the failure-mode catalogue marked Planned, which is a better place for them than my head.

Here's the part that would make me sceptical if I were reading someone else's series.

It has never served a real customer. Every scenario in the eval suite is one I invented, which makes it good at catching regressions and useless against the thing I didn't think of. Real users are more imaginative than I am, and they aren't trying to be fair.

The suite also grades deterministic classifiers, so anyone can clone the repo and get my exact numbers. That's a genuine win for reproducibility and a genuine limit on what I've shown: the version swap I graded was one component against another, not one provider against another. And I've never measured end-to-end latency with a hosted model in the loop.

So: a design I believe in, tests for the parts that are testable, and no war stories. If it ever does meet production traffic, the thing I most want to know is whether the audit trail catches what the evals missed.

What's the gap in your system that everyone calls deliberate?