# Checking agent code against your intent

> Source: <https://leangineer.com/blog/checking-agent-code/>
> Published: 2026-11-22 16:04:33+00:00

**Recommendation:** Have agents write formal requires/ensures properties from the change's intent and machine-check the proof before review, but still read the code, since a proof only covers the property you wrote and the assumptions it lists.
More in the playbook "Working with AI coding agents": <https://wpnews.pro/playbooks/coding-agents>

## 1Introduction

Coding agents now write more code than I can read. One answer is to stop reading. Robert C. Martin, better known as Uncle Bob, says his current strategy “is to not read any of the code written by my agents”. Instead the code has to “run the gauntlet” of his tests, quality metrics and mutation testing<sup>[1](blog/checking-agent-code#fn-bob)</sup>. I disagree. I think you should read all of it. The way to keep that possible is to get agents to write less code, and better code, before it reaches you.

More tests don’t get you there. In Dan Luu’s August runs, asking agents to use Rust’s built-in test framework roughly doubled the number of tests they wrote, “but this didn’t result in better correctness”<sup>[2](blog/checking-agent-code#fn-luu)</sup>. A newer model tested better a month later<sup>[3](blog/checking-agent-code#fn-astra)</sup>. Still, a test an agent writes encodes the agent’s own idea of what the code should do.

On 18 November Microsoft Research described a code reviewer that works on the “better” part<sup>[4](blog/checking-agent-code#fn-nfvpost)</sup>. It starts from the intent of a change, meaning what the change is supposed to do, and writes that intent down as precise properties. An agent tries to prove that the code has each property, and a small, trusted program checks the proof. The researchers call this neuro-formal verification (NFV), and on 38 recent pull requests it checked 217 properties<sup>[5](blog/checking-agent-code#fn-prs)</sup>. Harmonic’s Aristotle, Midspiral’s LemmaScript and Don Syme’s Lean Squad work in a similar way.

This post asks three questions about tools like these. What does such a proof tell you? That the code has a property you wrote down, under the assumptions it lists. What can’t it tell you? Whether the property was the right one, whether it was proved about the code that runs, and whether it says what you meant. Can you stop reading the code? No, but less of it reaches you, and what does has already been checked against the intent. Figure 1 shows the loop. Like my [last post](https://leangineer.com/blog/realworld-in-lean/), this one borrows the layout of a math paper.

## 2Verdicts

In 2014 a bug in OpenSSL, later named Heartbleed, let anyone on the internet read a server’s memory, including passwords and secret keys<sup>[6](blog/checking-agent-code#fn-heartbleed)</sup>. The intent behind the fix fits in one sentence, which Aristotle uses as its example: “The response is never longer than the request.” Three steps turn a sentence like that into a checked verdict.

Construction 1 (The model). The code is translated into Lean or Dafny, two languages built for writing proofs about programs. Aristotle compiles C, Rust and Verilog into Lean, and LemmaScript compiles TypeScript into Dafny or Lean, both with an ordinary compiler. NFV has an agent do the translation instead. That works for almost any language but adds a step you have to trust.

Construction 2 (The property). The intent is written down as a formal property. Shuvendu Lahiri, who wrote the NFV paper, calls this intent formalization<sup>[7](blog/checking-agent-code#fn-lahiri)</sup>. NFV derives properties from a pull request’s description, issues and discussions. In LemmaScript the intent and the property sit in comments above an ordinary TypeScript function, as in this example from its [demo](https://github.com/midspiral/lemmascript-claimcheck/blob/main/examples/demo.ts):

```
//@ contract Clamps x into the inclusive range [lo, hi]; the result never falls outside it.
//@ requires lo <= hi
//@ ensures \result >= lo && \result <= hi
export function clamp(x: number, lo: number, hi: number): number {
  if (x < lo) return lo;
  if (x > hi) return hi;
  return x;
}
```

The `contract` line is the intent in English. The `requires` and `ensures` lines are the property. They say that if `lo <= hi` when the function is called, the return value (`\result`) lies between `lo` and `hi`. TypeScript ignores all three lines, so the code that runs doesn’t change.

Construction 3 (The proof). An agent searches for a proof and a checker decides whether it holds. In Harmonic’s words, “AI writes the proofs and Lean checks every step of them, so a result never rests on the AI being right.” The checker is a small program at the core of Lean or Dafny, and it accepts a step only if the step follows from the ones before it.

Definition 1. *A *proof-backed verdict* on a property is one of three things.*

- *Proved* , with the list of assumptions the proof rests on.
- *Refuted* , with an input on which the property fails.
- *Unresolved* , if neither a proof nor a counterexample was found in time.

Remark 1 (Saying "unresolved"). A checker that can say “unresolved” is right more often when it does answer. On NFV’s benchmark of small Python programs, 92% of its verdicts were right. A model asked directly answered every program, but only 72% of its answers were right, and none came with anything that could be checked<sup>[8](blog/checking-agent-code#fn-nfv)</sup>.

## 3Limits

A proof covers what its property says and nothing else. Harmonic’s [documentation](https://aristotle.harmonic.fun/docs/overview) puts it plainly, warning that “a behavior that isn’t listed as a property hasn’t been verified.” So a proof is only worth as much as its property.

Definition 2. *A proved property *matches your intent* if it is a property that matters, it was proved about the code that runs, and it says what you meant.*

The checker covers none of these three parts, and each of them fails in practice.

*A property that matters.* In Luu’s August runs, agents implementing the Zstd compression format in Rust were told to use Lean or another proof tool. They “mostly did arithmetic proofs that didn’t hit the bug-prone or risk surface areas”<sup>[2](blog/checking-agent-code#fn-luu)</sup>. Don Syme says much the same of Lean Squad, which proved more than 1,200 theorems about three open-source codebases: “Most theorems are glorified testing”<sup>[9](blog/checking-agent-code#fn-squad)</sup>.

*The code that runs.* An agent translating a program can model a slightly different program. The NFV blog post admits that “the proof alone cannot establish that the formalization faithfully captures the source program and supplied property.” A compiler is more predictable, but it still simplifies. LemmaScript treats TypeScript numbers as exact integers, so its proofs say nothing about floating-point rounding. Midspiral’s crosscheck looks for gaps like that. It runs the proved model and the real JavaScript on the same inputs and compares the results, which is called differential testing<sup>[10](blog/checking-agent-code#fn-ls)</sup>.

*What you meant.* Midspiral’s claimcheck caught this in an election tally that an agent had built and proved. The intent was that the order of the ballots doesn’t affect the tally. The proved property only showed that adding a ballot at the front of the list gives the same count as adding it at the end. Dafny accepted it, and it looks right at a glance. But it covers only one way of reordering the ballots<sup>[11](blog/checking-agent-code#fn-claimcheck)</sup>.

Remark 2 (Agents move the goal). An agent rewarded for finding a proof can also make the problem easier. The NFV blog post lists the ways: “by overconstraining the environment, simplifying the code, or weakening the property”. The authors also let a single agent translate and prove in one go. It proved the property for 98% of the buggy programs as well as 98% of the correct ones, so half of its verdicts were wrong.

## 4Safeguards

The tools guard against these failures with checks that don’t depend on the agent following instructions. That matters, because instructions turn out to be fragile. Luu’s own testing skill, a file of instructions an agent loads for a task, helped GPT-5.6 Sol and hurt the newer GPT-6 Astra<sup>[3](blog/checking-agent-code#fn-astra)</sup>. There are four safeguards.

*Freeze the goal.* NFV locks the translated program and the property before the proof search starts, so the agent can add proof steps but can’t change what is being proved. LemmaScript’s seal does the same across commits. Its check fails if a proved property is changed or removed, or in its README’s words, “The prover proves the present; the seal protects the past.”

*Read the property back.* Midspiral’s claimcheck has one model restate the formal property in plain English without seeing the intent, and a second model compare the two. LemmaScript runs it on every `contract` line. The demo file from earlier has a function whose contract claims more than its property:

```
//@ contract Returns the larger of a and b.
//@ ensures \result >= a
export function largest(a: number, b: number): number {
  return a >= b ? a : b;
}
```

The code is correct and the property is proved, but the property never mentions `b`. Its restatement, “returns a number that is greater than or equal to a”, isn’t “the larger of a and b”, so the [report](https://github.com/midspiral/lemmascript-claimcheck/blob/main/examples/demo.guarantees.md) marks the function as disputed<sup>[11](blog/checking-agent-code#fn-claimcheck)</sup>.

*List the assumptions.* Aristotle and NFV list the assumptions behind every verdict. Harmonic’s write-up on the LibTomMath library, where Aristotle fixed three bugs and proved the repaired code, has a section called “Outside the proof” for what wasn’t covered<sup>[12](blog/checking-agent-code#fn-tom)</sup>.

*Stop early.* In NFV’s experiments, proofs that took many attempts were more likely to be wrong. The authors put that down to each attempt being another chance to weaken the goal. So NFV caps the search at five rounds and reports anything still open as unresolved<sup>[13](blog/checking-agent-code#fn-rounds)</sup>.

## 5Review

All of this changes what reaches you, and how you read it.

*Feedback from the checker.* The checker sends wrong code back to the agent before it reaches you. Midspiral shows this with a bill-splitting app. Here is its `allocate` function after the agent’s fix, with most of the `requires` lines left out:

```
//@ contract Splits total across weights so every unit is accounted for.
//@ requires total >= 0
// ...
//@ ensures sum(\result) === total
function allocate(total: number, weights: number[]): number[] {
  const W = sum(weights);
  const result = weights.map(w => Math.floor(total * w / W));
  const left = total - sum(result);  // the leftover, lost to flooring
  for (let k = 0; k < left; k++)
    result[k] = result[k] + 1;  // hand them back out
  return result;
}
```

The agent’s first version ended at the `Math.floor` line, which rounds each share down, so 100 cents split three ways came to 99. The `ensures` line failed. The agent read the error and added the loop that hands out the leftover, and the property stayed as it was. Midspiral notes that “this correction happened without a human in the loop”<sup>[14](blog/checking-agent-code#fn-eventab)</sup>.

*Reading order.* You read the properties first and ask whether each one is what you meant. Then you read all of the code with them in mind, and finally the assumptions and whatever is unresolved. The proofs are the checker’s job. A refuted property comes with an input on which the code fails, which is easier to act on than a hunch. NFV’s comments on pull requests are laid out this way (Figure 2).

*Less to read.* A property is short. The one `ensures` line of `allocate` states what the whole function must do. LemmaScript’s rules also keep each decision in one verified function and forbid recomputing it anywhere else, since a copy would be “a second, unverified implementation of a verified fact”<sup>[15](blog/checking-agent-code#fn-rules)</sup>. Generated tests can go the other way. A reader of Luu’s blog needed a 300,000-line generated test harness for a 30,000-line rewrite, and called the result negative<sup>[16](blog/checking-agent-code#fn-leventov)</sup>.

*Refactors.* Refactors and rewrites are the easiest case, because their intent is to behave as before and the old code says what that means. Differential testing compares the old and new versions directly.

In Luu’s evals, differential testing was the third-worst technique with GPT-5.6 Sol. With GPT-6 Astra it was narrowly the best on his hardest task, implementing the Pandoc document converter (Figure 3)<sup>[17](blog/checking-agent-code#fn-diff)</sup>. The bad result is the one to remember. With Sol, agents “did the same thing twice and encoded the same bug in both versions”, so the comparison only helps if the agent didn’t write both sides. Aristotle offers the proof version of the same check for C code rewritten in Rust.

Remark 3 (Where it's worth it). Proofs pay off most for the logic the rest of an app depends on, such as state changes, money and permissions. A LemmaScript codebase keeps that logic in a verified core, apart from the user interface, network and storage. For code that doesn’t justify a proof, the same safeguards work with a property-based test, which checks a property on many generated inputs. You write it before the code, freeze it, have a second agent read it back, and keep the tests few enough to read yourself.

If you want to try this, here’s where to start:

- [LemmaScript](https://github.com/midspiral/LemmaScript) for TypeScript, open source and in tech preview.
- NFV's [code reviewer](https://github.com/microsoft/neuro-formal-verification) for pull requests in most languages, an open-source prototype that runs as a GitHub Agentic Workflow.
- [Aristotle](https://www.harmonic.fun/aristotle/) for Rust, C and Verilog, a commercial service.
- [Lean Squad](https://github.com/githubnext/agentics/blob/main/docs/lean-squad.md) for Rust or C++ with Lean on the side, a GitHub Agentic Workflow you add to a repository.
- Property-based testing libraries for the light end in any language, such as Hypothesis for Python, fast-check for TypeScript and proptest for Rust.

## 6Lean

Lean already sits underneath several of these tools. It checks the proofs in Aristotle and Lean Squad, and LemmaScript can compile TypeScript into it. In all of them the model from Construction 1 is still something you have to trust.

If the code itself is written in Lean, there is nothing to translate. In my RealWorld app, the function that turns an article’s title into its URL slug has a theorem about it. The server calls that same function when it creates an article:

```
-- RealWorld/Slug.lean
theorem ofTitle_valid (title : String) : Valid (ofTitle title).toList := by
  ...

-- RealWorld/Service.lean, creating an article
let slug ← withUniqueSlug env (Slug.ofTitle a.title) fun db slug => Db.insertArticle db slug me.id a
```

`Valid` says a slug is lowercase letters and digits with single hyphens between words. The theorem is about the code that runs. NFV’s README calls this the limit of the whole approach, “checking a formal assertion about a closed program with no external environment in a language with a trusted logical encoding.” A real app still has a database, a network and C libraries outside the proof, but there is no model of your own code left to get wrong.

Theorems aren’t the only tool, either. Lean is a general-purpose language, and a claim can rest on whatever it can bear. The community [style guide](https://github.com/saviorand/lean-style-guide) I started orders the options in a principle from Paul Butcher: “Give each claim the strongest guarantee it admits: a type, then a theorem, then a property test, then an example.” Three of the RealWorld app’s guarantees come from types alone. If a handler’s types don’t match its route, for example, the code doesn’t compile. Property-based tests run with [Plausible](https://github.com/leanprover-community/plausible), and the loop in Figure 1 works the same with a frozen property test in place of a theorem.

Remark 4 (Agents and Lean today). Agents don’t use Lean well on their own yet. In Luu’s Astra runs, agents told to use Lean 4 came out slightly below those given no instructions on both tasks, as Figure 3 shows for Pandoc. The gaps are within the noise, but neither shows Lean helping<sup>[18](blog/checking-agent-code#fn-lean)</sup>. In those evals, though, Lean sat beside Rust code, so the agents proved things about their own model of it, the gap this section removes. A program written in Lean is a setup his evals didn’t test. The style guide is my attempt at the guidance agents need, and I haven’t measured whether it helps.

Question 1. *Can writing down the intent be made cheap enough to do for every change? Everything above starts from properties someone wrote down, and only the developer can tell whether they are the right ones. Lahiri makes the same point, that nothing but the user can judge whether a specification is correct. NFV already drafts properties from the pull request, and its authors suggest asking the developer when the description is too vague. How far that can go remains open.*

Writing real software in Lean, so that the proofs are about the code that runs, is what this site is about. The Lean backend of my RealWorld app came to 1,033 lines, the shortest of the four implementations I compared<sup>[19](blog/checking-agent-code#fn-rw)</sup>, and the [RealWorld post](https://leangineer.com/blog/realworld-in-lean/) shows how far a full-stack web app in Lean gets today.

Read all of your agent’s code. In Lean, there’s less of it and nothing to translate.
