{"slug": "a-differential-oracle-making-agentic-code-prove-its-own-correctness", "title": "A differential oracle: making agentic code prove its own correctness", "summary": "A self-taught engineer developed a differential oracle evaluation layer for agentic code, using two independently written implementations of a match-3 game to verify correctness. The system runs invariant tests across a React build and a native Java engine, catching bugs that would be missed by single-implementation testing. The oracle is part of a broader autonomous dev harness with self-reviewing loops and human-in-the-loop approval for irreversible actions.", "body_md": "Six months ago I was running a kitchen. I taught myself to build agentic systems, and the thing I'm proudest of is the part nobody demos: the evaluation layer.\n\nCoding agents are easy to demo and hard to *trust*. The moment an agent touches a real codebase — deploys, commits, user-facing changes — you need what any production system needs: review, tests, and a way to catch your own regressions.\n\nI build a match-3 game. Match-3 resolution has thousands of edge cases — cascades, chain reactions, special-piece rules — where the \"right answer\" isn't obvious and you can't check them all by hand.\n\nSo instead of trusting one implementation, I use two — written independently. The same game logic runs in a **React build** and a **native Java engine**, and logic-invariant tests hold both to the same rules. If the two ever disagree on a board state, one of them is wrong. Agreement between two independently-built systems is a far stronger signal than either one passing its own tests — it has caught real bugs I'd never have found by playing.\n\nRunnable repo: [https://github.com/egnaro9/evals-differential-oracle](https://github.com/egnaro9/evals-differential-oracle) — two implementations + invariant tests, plus a deliberately-buggy version that both nets catch. `pytest`\n\npasses over ~9k random boards.\n\nThe rules themselves are also tests — e.g. *a special gem is generated only by a direct match of 4+, never a 3.* When I planted a bug that awarded one for a 3-run, the differential oracle disagreed **and** the invariant checker flagged it.\n\nThe oracle sits inside an autonomous dev harness: a self-reviewing loop (Strategy → Execution → Critic → Evaluation → Ops) with a cold, independent critic, per-role model routing, and a human-in-the-loop autonomy ladder that automates the launches but never the approval on anything irreversible.\n\nWrite-up: [https://github.com/egnaro9/agentic-dev-harness](https://github.com/egnaro9/agentic-dev-harness) · portfolio: [https://egnaro9.github.io](https://egnaro9.github.io)\n\nI'm a self-taught engineer looking for a remote (US) role in agentic / AI / evals engineering. If this is what your team cares about, I'd love to talk.", "url": "https://wpnews.pro/news/a-differential-oracle-making-agentic-code-prove-its-own-correctness", "canonical_source": "https://dev.to/erik_hill_d7e478733fc524d/a-differential-oracle-making-agentic-code-prove-its-own-correctness-4ce7", "published_at": "2026-07-15 02:58:44+00:00", "updated_at": "2026-07-15 03:29:01.381095+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "developer-tools", "ai-research"], "entities": ["egnaro9"], "alternates": {"html": "https://wpnews.pro/news/a-differential-oracle-making-agentic-code-prove-its-own-correctness", "markdown": "https://wpnews.pro/news/a-differential-oracle-making-agentic-code-prove-its-own-correctness.md", "text": "https://wpnews.pro/news/a-differential-oracle-making-agentic-code-prove-its-own-correctness.txt", "jsonld": "https://wpnews.pro/news/a-differential-oracle-making-agentic-code-prove-its-own-correctness.jsonld"}}