{"slug": "my-trust-gradient", "title": "My trust gradient", "summary": "A developer writing in Harness Engineering describes a \"trust gradient\" for working with non-deterministic LLM coding agents, moving from line-by-line review to diff review to requirement verification as deterministic layers prove themselves. The approach splits recurring model mistakes into constraints (types, linters, tests, hooks, gates) that reject output, and capabilities (external tools) that compute or fetch answers instead of letting the model guess. Each layer must catch real instances of a mistake on real changes before the developer stops checking for it manually.", "body_md": "Chapter 3 of *Harness Engineering* opens with two runs of the same agent, on the same prompt, in the same repo, two days apart. On Tuesday it added a handler in routes/health.ts, wrote one happy-path test, and stopped. Ten lines. On Wednesday it noticed an old status.ts, unified the two, refactored both, broke a test, marked the test flaky, and opened a pull request claiming it was done. Four hundred lines. Nobody had touched the model, and nobody had touched the rules.\n\nAnything you build on an LLM varies like this, because the model is non-deterministic. I don’t try to make the model deterministic. I layer determinism around it with constraints and capabilities, and I test each layer before I lean on it. As the layers prove themselves, I step back. That’s my trust gradient: I started by reading every line, moved to reading the diff, and I’m now working toward verifying the requirements. It’s how I’ve been working, shared as one example rather than a prescription.\n\nThe Tuesday and Wednesday runs are the reason I started from doubt. If the same input can give you ten lines one day and four hundred the next, a good result tells you very little about the next one. So doubt wasn’t paranoia. It was the correct reading of the evidence.\n\nAt first I read every line an LLM wrote. That was slow. It often took longer than writing the code myself would have. But it produced something more useful than the code: a list of the mistakes that kept coming back. Mine were stylistic preferences, code reuse idioms I didn’t like, and mocking. None were exotic, and they came back run after run.\n\nThat list is what got me off the bottom of the gradient. A mistake that recurs is one you can name. Once I could name it, I could stop catching it by eye.\n\nI moved each recurring mistake into one of two kinds of layer.\n\nA **constraint** narrows what the output can be. Types, linters, formatters, tests, small slices, hooks, and gates are all constraints. The model can still generate anything it likes, but the constraint rejects whatever falls outside the line, and it rejects it the same way every time.\n\nA **capability** takes the work away from the model. Instead of letting it generate an answer, I hand it a tool that computes or fetches the answer. Most of my capabilities reach outside the harness, meaning the environment the agent runs in. Connecting to an external service is the typical one: rather than guess what a service returns, the model calls it and gets the real response.\n\nTo pick between them, I ask what I can do with the result. If I can check it, I constrain it. If I can compute it, I make it a capability.\n\nNeither kind of layer got my trust on arrival. A deterministic check answers only the question you wrote into it, and an agent will find whatever it doesn’t cover. So each check has to catch every case of the mistake without firing on false positives. Before I stopped reading for a mistake, I watched its layer catch real instances of it on real changes.\n\nCapabilities needed the same care, because an external service brings its own variance. In tests I verify against data rather than live services. Mocks are only for the adapter tests at that boundary.\n\nThat’s how I moved from reading every line to reading the diff. I didn’t decide to trust the model more. Instead, the line-level questions moved into layers that answer them the same way every run. Once those layers had caught real mistakes, I stopped asking those questions myself. I read the change as a whole instead.\n\nThe same goes for everything with a model inside it, not only generated code. Agents, sub-agents, skills, and MCP servers are all non-deterministic. So each one starts at the bottom of the gradient and moves up only as its layers hold.\n\nThe layers do more than save review time. Once I’ve tested a deterministic layer, it holds still. So when every check sits in the layer that suits it, most of the variance comes from one place, the model. That means when a result surprises you, you can rule things out.\n\nAn agent of mine once started re-running tests it didn’t need to, over and over, to find failures. The test suite was a deterministic layer I’d already tested, so I could set the tests aside. The rules hadn’t changed either. That left context as the likeliest cause: it had bloated to the point where the agent re-ran tests to find failures it had already found.\n\nWithout the layers, that same symptom could have come from anywhere: a flaky test, a rule that changed meaning, a model update, or a bloated context. If you can’t rule any of those out, you’re guessing. Worse, every fix you try changes more than one thing at once.\n\nThe layers turn that guessing into a controlled experiment, because the deterministic parts hold still while I change one variable and watch what moves. I care about that more than the review time it saves.\n\nThat’s also why a check in the wrong layer causes so much trouble. The common version is a rule written into the prompt instead of enforced by a linter or a hook. A prompt is part of what the model reads, so the rule holds on some runs and not on others. Wednesday’s agent marked a broken test as flaky. A line in the prompt telling it not to do that holds some days, but a gate that refuses the change holds every day.\n\nThe capability version is asking the model to work out something a script could compute. It will usually get close. Close is worse than wrong, because close passes when you skim it.\n\nEither way, you get unexpected results: behavior that changes between runs with nothing changed on your side, and no single place to look for the cause. Putting each check in the right layer is how I stopped getting those.\n\nI’m in the middle of moving from review to verify. I already verify changes, but I haven’t yet used verification in place of code review. Once I do, the questions left for me are the ones no linter can answer. Does the feature do what the card says? And does it fit how the product is actually used, and deliver the business value it was supposed to?\n\nI’m not claiming verify is the ideal state. I’m exploring what’s possible, and the only way to find out whether it works is to try it. But when I try it, I want to rely on the result, which is what the existing layers are for. They hold the mechanical questions still. So if verification lets something through, I can tell whether the gap is there or in a layer below it.\n\nThat’s the same controlled experiment as before, one level up. My diff-level review is in [What’s left for me to review](https://blog.tacoda.dev/whats-left-for-me-to-review-a70af1351e22), so I won’t repeat it here.\n\nTrust also moves back down the gradient, on evidence. Say an agent action gets tied to an event or a lever it shouldn’t have been. That’s evidence for a proposal to move it down. So is anything that causes an incident.\n\nA model swap might be too, which is part of why my team runs evals. A new model is one variable, and a change to the rules is another, since updated rules produce different output. Evals hold everything else fixed and change one variable, the same controlled experiment the layers make possible. That’s how we prove a change made a difference, rather than believing it did.\n\nThe cost is time. Reading every line is slow, and building and proving each layer is slower than trusting the tool on day one. Someone who skips ahead will probably ship sooner for a while.\n\nHere’s what you gain: the ability to move up abstraction layers and build more complex pieces out of the components. You move from caring about syntax to caring about architecture to caring about business value. You can start at the top, but then you have to trust everything underneath it.\n\nIt’s also not the only way. It’s the path I’ve been on, and it has worked for me. Because I can rely on the environment I’ve built around the agent, I’m moving toward more automation and bigger changes, and an increase in velocity as a side-effect.\n\nThe next step is the one I haven’t taken: verification checks in place of code review. Once I’ve built them, I’ll write up what shape they take, what their tradeoffs are, and how they affect quality.", "url": "https://wpnews.pro/news/my-trust-gradient", "canonical_source": "https://tacoda.medium.com/my-trust-gradient-ea1b8ec267e4?source=rss-bf474619cf47------2", "published_at": "2026-10-22 19:47:07+00:00", "updated_at": "2026-10-22 20:11:26.028573+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops", "artificial-intelligence"], "entities": ["Harness Engineering"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/my-trust-gradient", "markdown": "https://wpnews.pro/news/my-trust-gradient.md", "text": "https://wpnews.pro/news/my-trust-gradient.txt", "jsonld": "https://wpnews.pro/news/my-trust-gradient.jsonld"}}