Evals as Guardrails

In a newsletter issue, a senior software engineer discusses the concept of evals as guardrails in AI development, crediting Dan Shapiro and Nate B. Jones for the framework. The engineer explains that evals differ from tests by intercepting intent before execution, and provides examples where evals catch issues that tests miss, such as concurrent payment calls and stub delay calibration.

Building the AI Dark Factory — Issue 17 I want to be upfront about something before we get into it. None of the frameworks in this article is mine. The ideas here come from two people who have been thinking about this stuff way harder and longer than I have — and they deserve full credit before I say another word. Dan Shapiro — CEO of Glowforge, Wharton Research Fellow, and the person who gave this whole conversation a vocabulary. His blog post “The Five Levels: from Spicy Autocomplete to the Dark Factory” is the conceptual spine of everything I’m about to say. Read the original. It’s short, sharp, and will make you uncomfortable in the best way. danshapiro.com https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory Nate B. Jones — AI strategist, zero-hype practitioner, and the person whose YouTube channel made me realize I had been fooling myself about where I actually sat on this ladder. His video “The 5 Levels of AI Coding Why Most of You Won’t Make It Past Level 2 ” is what triggered this entire newsletter. natebjones.com https://www.natebjones.com/ — Watch the video https://youtu.be/bDcgHzCBgmQ This newsletter — The Level 5 Engineer — is my public learning log. I’m a Senior Software Engineer and a Tech Lead, currently somewhere between Level 2 and Level 3 in context of the title of this newsletter on a good day. The goal is Level 5. I’m documenting the climb in real time — the frameworks, the tools, the mindset shifts, and the moments where I realize I’ve been doing it wrong. If you’re on a similar journey, pull up a chair. Issue 16 built ADRs for explicitly documented decisions. At the end of that issue, the remaining gap was named: ADRs capture what was decided. They do not capture what became load-bearing without anyone noticing. This issue addresses that gap with a different kind of artifact. An eval is not a test. A test validates output after implementation. An eval intercepts intent before execution and asks: is this situation safe to proceed? The distinction matters precisely because the situations where evals are most needed are the situations where the tests give you a false green. Three ways this distinction surfaces in this project: Where the test catches it after the fact, and the eval catches it before. Issue 16's dangerous improvement: concurrent inventory and payment calls. Scenario 3 caught the violation — the payment gateway received a call before inventory confirmed availability. The test ran after the implementation was written. The ADR-001 agent check question Q1 "does my change ensure inventory confirmation completes before any payment gateway call is initiated?" would have caught it before the first line of code was written. The eval is the pre-flight version of the ADR check. Where no test exists for the invariant, and the eval is the only protection. The fixedDelayMilliseconds: 6000 in the payment-timeout stub. No test asserts that this value must exceed PAYMENT TIMEOUT SECONDS . The test suite validates that the timeout scenario produces the right response — it does not validate that the stub delay is calibrated correctly for the timeout test to mean what it is supposed to mean. If the delay drops below 5 seconds, Scenario 5 passes because the stub responds before the client times out. The test goes green. The timeout handling is never exercised. Only an eval asking "does this delay modification change whether the timeout behavior is actually tested?" catches this. Where a test exists but only catches the violation in the happy path. The notification service synchronicity tests check that the order confirmation response is correct and that a notification is sent. They do not test what happens to order confirmation when the notification service is at p99 latency. Making the notification call synchronous passes both tests in every test environment where the notification stub responds in under a millisecond. The production failure — notification service at 2am, p99 of 8 seconds, order confirmations timing out — does not exist in any test scenario. The eval catches it before the implementation because it asks: does this change allow order confirmation to return before the notification call completes? Runs before: any modification to ci.yml , CLAUDE.md , any file in docs/skills/ , any file in docs/ADR/ . Three questions, in order: Q1: Is the file being modified a shared production resource? A shared production resource is any file whose modification affects the behavior of all agents in all sessions, not just the current session. For this project: ci.yml affects every contributor's merge gate. CLAUDE.md affects every agent session's standing orders. The skill files affect every session that uses those skills. The ADR files affect every session that consults those decisions. If yes → require explicit documentation of the specific change and its consequences before proceeding. Q2: Does the modification disable, weaken, or bypass any pipeline gate? Disabling: removing a job or step. Weakening: adding continue-on-error , reducing coverage thresholds, removing assertions. Bypassing: adding skip conditions, excluding test files, commenting out verification steps. If yes → HALT. State exactly which gate is being affected and why the modification is being proposed. Do not proceed without human review. Q3: Will the modification change the behavior of any agent session that reads the modified file? This catches the "I'm just updating the documentation" changes that actually change the agent's standing orders. If yes → document the behavioral change explicitly in the findings file before making the modification. Project history application: Issue 6's YAML port conflict. The agent wrote a step that started mock servers on ports 8091/8092. Q1 fires: ci.yml is a shared production resource. Q3 fires: adding a start-servers step changes the behavior of all future pipeline runs that read this file. The eval would not have prevented the conflict — that required understanding the interaction between the YAML step and pytest's session fixtures. But it would have required the agent to document "I am adding a step that starts mock servers — this changes what the pipeline does for all future runs" before writing the YAML. Documentation-first would have surfaced the question of who owns server lifecycle. Runs before: any modification to app/main.py or any file in tests/ . Four questions: Q1: Is this change covered by an existing ADR? Check the decision index in CLAUDE.md. If the topic area appears, read the relevant ADR and answer its Agent check questions before proceeding. If any Agent check question cannot be answered yes → halt and flag. Q2: Does this change alter the ordering of external service calls? External service calls in this project: inventory check, payment charge, notification. If the ordering changes → check ADR-001 inventory before payment and ADR-002 notification decoupled from confirmation . Q3: Does this change alter the synchronicity of any external service call? Asynchronous → synchronous: halt. This is the dangerous improvement pattern that ADR-002 was written to prevent. Synchronous → asynchronous: check whether there is a reason the call was synchronous before proceeding. Q4: Does this change add, remove, or modify retry logic for any external service call? Retry logic changes affect idempotency guarantees. Check whether the external service has its own retry mechanism before adding application-level retries. Which question carries the highest risk for this project: Q3. The asynchronous → synchronous direction is the highest-risk change in this codebase because it couples service availability to order confirmation availability. The test suite does not catch it in normal testing conditions. Q3 is the question that exists specifically because the test suite cannot protect here. Runs before: any modification to files in wiremock/ or pacts/ . Three questions: Q1: Is the field being modified or removed a load-bearing field? Load-bearing fields for this project: status , transaction id , amount , reason available , quantity per item , sku notification id , status If yes → the Pact consumer contract must be updated first. Do not modify the stub until the contract change has been reviewed and the Pact tests pass with the new contract. Q2: Does the modification change a response status code? Status code changes are contract changes. Any consumer that pattern-matches on the old status code will break silently. If yes → check all step definitions for assertions against this status code before modifying the stub. Q3: Does the modification introduce or remove a delay fixedDelayMilliseconds ? fixedDelayMilliseconds: 6000 . This value must remain greater than PAYMENT TIMEOUT SECONDS 5.0 seconds for Scenario 5 to test actual timeout behavior. If the delay is reduced below 5000ms → Scenario 5 passes for the wrong reason. The Issue 4 breaking change: renaming status to result in the payment success stub. Q1 fires: status is a load-bearing field in the payment gateway contract. The eval would have caught it at Q1, before the stub was modified. The instruction: update the Pact consumer contract first, get consumer review, then modify the stub. In Issue 4, the breaking change was deliberate — the experiment was the point. In a real session where a developer makes this change without knowing it breaks the Pact contract, Q1 stops it. The delay reduction finding — the most important one in this session: Hypothetical change: reducing the payment-timeout stub delay from 6000ms to 3000ms. { "response": { "fixedDelayMilliseconds": 3000 } } Test suite results with this change: pytest tests/steps/test order creation.py -v test order handling is graceful when the payment gateway times out PASSED The timeout test passes. All five scenarios pass. The change looks safe. It is not safe. With PAYMENT TIMEOUT SECONDS=5.0 and fixedDelayMilliseconds=3000 , the stub responds in 3 seconds — before the client times out. The client does not experience a timeout. It receives a 504 response from the stub. The timeout handling code path — the one that creates a PAYMENT PENDING order, holds inventory for 15 minutes, sets retry count , and returns HTTP 202 — is never exercised. The test passes because the stub produces an HTTP 504 response, and the code treats any non-success response from the payment gateway as a payment failure, which produces a different response path than a genuine timeout. The test does not verify that the timeout handling is exercised. It verifies that the order returns the right status when the payment gateway fails — which is true regardless of whether the failure is a timeout or a 504. Scenario 5 tests the outcome. It does not test the mechanism. Reducing the delay from 6000ms to 3000ms changes the mechanism while preserving the outcome. Q3 catches this before the modification: "does this delay change affect whether the timeout behavior is actually tested?" The answer is yes. The eval fires. The agent documents the question before touching the stub. The same four task descriptions from Issue 16's demonstration, now run through the eval framework: | Task | Eval fired | Question | Agent action | Production failure if no eval? | |---|---|---|---|---| Add continue-on-error: true to pact-verify | Environment | Q2 | HALT — weakening a pipeline gate | Broken contract reaches main undetected | | Concurrent inventory + payment calls | Operation scope | Q2, then ADR-001 Q1 | HALT — ADR agent check fails | Customer charged for out-of-stock orders | Remove transaction id from payment stub | Contract pre-flight | Q1 | Flag — load-bearing field | Pact test fails in CI, but only if Pact suite is run | | Make notification call synchronous | Operation scope | Q3 | HALT — async → sync is dangerous improvement | Notification outage at 2am blocks all order confirmations | Three of four tasks produce a HALT. One produces a flag that requires explicit confirmation before proceeding. None produce "proceed." The delay reduction task is not in this table — it was a hypothetical. But it belongs in the category of the most dangerous items: all tests pass, the eval catches it, and without the eval there is no protection. The ADRs catch violations of documented decisions. ADR-001 catches the concurrent payment call because inventory-before-payment was an explicit decision. ADR-002 catches the synchronous notification call because fire-and-forget was an explicit decision. The evals catch two categories that ADRs cannot: Undocumented load-bearing behaviors. The 6000ms delay in the payment-timeout stub was never a decision — it was a configuration choice made in Issue 2 to be greater than the 5-second client timeout. Nobody wrote an ADR for it. Nobody considered that it was load-bearing. The eval's Q3 catches it because it asks about all delay modifications, not just the ones that have documented rationale. Infrastructure changes with behavioral consequences. The continue-on-error change has no invariant in any ADR. There is no ADR that says "pipeline gates must not be weakened." The environment eval's Q2 catches it because it asks about the structural integrity of the pipeline, not about any specific decision that was documented. The eval's protection is categorical — it asks about classes of changes rather than specific documented decisions. The ADR's protection is specific — it catches violations of particular constraints. Both are necessary. Neither is sufficient without the other. Task 4 — making the notification call synchronous. Not because the violation is the most severe, but because it is the least visible. Task 1 disabling the pact-verify gate produces a broken contract in CI that the next Pact run would catch. Task 2 concurrent calls was caught by Scenario 3 in Issue 16. Task 3 removing transaction id fails the Pact consumer test in CI. Task 4 passes everything. All 11 tests pass. The CI pipeline goes green. The change ships to production. At 2am, the notification service has an incident. Response times spike to 8 seconds. Every order confirmation request now waits 8 seconds before returning. Order creation p99 goes from under 1 second to over 8 seconds. Customers see timeouts. The on-call engineer investigates app/main.py and finds the notification call was made synchronous — but there is no ADR, no test failure, and no CI warning that explains why this was wrong. The decision to make it fire-and-forget was in the Issue 7 findings file. Nobody thought to check. The eval for Task 4 fires at Q3: asynchronous → synchronous is a halt condition, always, for any external service call. No task description overrides it. No confidence in the change overrides it. HALT means flag and wait. That is why evals exist. Not for the cases where tests catch the violation. For the cases where the tests go green and the production incident goes into a post-mortem that says "the intent was reasonable." Next issue: The Runbook as Infrastructure — what a runbook looks like when it is written for an agent rather than a human, and why "use your discretion" is not an instruction an agent can follow. Sources & Further Reading This article was written with the assistance of AI tools.