The control layer is the product, not the model

Gary Bernhardt argues that the control layer above AI models, not the models themselves, is the true product in agentic AI systems. He advocates for deterministic state machines, typed boundaries, and guardrails to ensure predictable, safe workflows. The post highlights that production-ready agents require engineering in routing, validation, state management, and audit trails rather than unconstrained model calls.

The control layer is the product, not the model Building agentic AI? I co-run a 6-week cohort where you ship a production-ready agent, not another API wrapper. Gary Bernhardt posted something this week that names a phenomenon we're teaching in our agentic AI cohort: Everyone seems fixated on the models, but I think there's so much low-hanging fruit in the control layer above the model. "Agent" and "harness" sell that layer short. There's so much more that we can do beyond "read input, send to model, run commands it returns." He's right. The model is a brain in a jar. Useful, fast, occasionally wrong, stateless. Everything that turns it into a product lives in the code that wraps it: the routing, the validation, the state, the audit trail. Gary calls that the control layer . I'm stealing the term. One of the replies under the tweet https://x.com/garybernhardt/status/2059869845265752156 nailed the design goal in a single question: do you actually know what the agent is going to do? That's what a control layer buys you. Not magic, not autonomy, predictability. A workflow where, by the time the model is called, the next move is already constrained to something safe. Why "agent" and "harness" sell it short When a developer says "I'm building an agent", they usually mean a while True loop that pings an LLM, parses a tool call, runs it, feeds the result back, and repeats. That pattern works for demos. It rarely survives contact with a real workflow. The word "harness" makes the wrapping code sound passive, a strap that holds the model in place. It's actually the control layer where the engineering happens. The model is a function call inside it. Once you flip that mental model, you stop asking "which LLM should I use" and start asking "what guarantees does my control layer make?" and "how can I make the inherently unpredictable model fit into a predictable workflow?" These are the questions production teams have to answer. Pattern 1: deterministic state machines, not unconstrained agents An agent without constraints decides what to do next from inside the model. A state machine /blog/build-finite-state-machine-python/ decides outside the model and gives the model one bounded job at each step. The pipeline runs categorize → validate → confirm → persist , and the LLM only ever gets called inside one of those buckets. This shifts control flow back to your code, where you can test it, log it, and reason about it. The expense agent we build in our cohort, which I broke down in How an AI expense agent is actually structured /blog/ai-agent-architecture-python/ , follows exactly this pattern: Protocol-defined LLM boundary, Pydantic-validated outputs, service layer holds the state, human-in-the-loop HITL confirms before anything writes. Four layers, no free-roaming agent, constraints at every step. Pattern 2: the model behind a typed boundary The model should be one swappable function call inside your control layer, not a dependency threaded through every layer. In our cohort the LLM lives behind a Python Protocol: a small interface the service layer depends on, so nothing downstream knows or cares whether the call goes to OpenAI or Anthropic. Once the boundary is a Protocol, the decisions people reach for "routing" to solve become wiring instead of rewrites. Picking a cheap fast model for a 12-way classification and saving the expensive one for hard reasoning is a one-line change. Falling back to a second provider when the first is rate-limited is a small factory, not a refactor. Swapping OpenAI for Anthropic, two SDKs that disagree on almost every detail, touches one file because the boundary absorbs the difference. And it makes the whole pipeline testable. Tests pass a mock that satisfies the Protocol, so you exercise every path without an API call incurring latency or cost. Pattern 3: evaluators and guardrails The model's output is not the user's output. Between the two sits validation: schema checks, business rules, PII filters, sometimes a second model grading the first one's work. This is the generator-evaluator split and it's an important pattern apart from HITL I've found for AI code that has to be right. The generator proposes. The evaluator approves or rejects. When the evaluator rejects, control loops back with feedback, not a stack trace. It's also the layer that catches the worst failure mode of multi-step agents. What production AI agents actually require /blog/production-ai-agents-real-workflows/ goes deeper on the four questions the control layer answers before any action runs: state, idempotency, audit, rollback. Pattern 4: structured generation A raw string from the model is the start of your problems. You can't store it, validate it, or test it well. The fix is to constrain output at the boundary: the model is allowed to speak, but only in shapes your code understands. Where the typed boundary in Pattern 2 decides where the model sits in your code, structured generation decides what shape it's allowed to emit. Pydantic plus your model's structured outputs gives you typed data instead of strings, which means the next layer of your control flow becomes ordinary Python. I covered this in Build the data layer before you touch the LLM /blog/build-data-layer-before-llm/ , explaining why we teach students to build the schema before they make a single API call. The frontier models make the headlines. The control layer ships the product. Gary's tweet names a gap that has been there the whole time, between the people optimizing benchmarks and the people building products. The control layer is the product, not the model. If you want to build AI products, that's where you need to spend your time. If you want a working walkthrough of the patterns above, the 10 small agentic AI exercises /blog/learn-agentic-ai-python-10-step-journey/ Juanjo https://juanjoseexposito.com and I shipped, run in the browser and cover the arc from a 3-line model call to a complete loop with HITL. They're the conceptual map. The cohort https://pythonagenticai.com is the same map, end to end. Six weeks, no frameworks, the control layer built explicitly, with code review at every step. By the end you can answer that one question: you know what your agent is going to do. Most AI tutorials end at "call the API." This cohort ends with a deployed agent: function calling, structured outputs, three interfaces, Docker, 95%+ test coverage. Six weeks of real engineering, not notebooks. Join the next Agentic AI cohort → https://pythonagenticai.com