Your AI Says Tests Passed. Can You Prove It?

A developer has launched Thea Software, an open-source project that acts as a "verdict layer for AI-written software," requiring deterministic checks and recorded evidence before AI-generated code changes can ship. The project addresses the gap where an agent's claim that tests passed is indistinguishable from a check that was skipped, using a protocol that maps each change to proof and enforces the verdict at build time. The developer argues that as AI coding becomes standard, teams must verify agent output rather than trust its summaries.

Stop trusting AI coding agents. Last week, I caught myself doing something I never thought I'd do as an engineer: I accepted an AI-generated summary as proof. The agent had a clean explanation. It sounded confident. It claimed tests passed. The reasoning looked solid. And for a moment, that felt like verification. It wasn't. As AI coding becomes normal, I think we're developing a new blind spot: We're starting to trust claims instead of evidence. For years, software delivery looked like this: Developer → Code → CI → Deploy Today, it's increasingly becoming: Agent → Code → Claim → Verification → Deploy That extra step matters. An AI agent can tell you: But what certifies those statements? One realization keeps bothering me: A skipped check and a successful check can look surprisingly similar when you're only reading an agent's summary. Humans are wired to trust polished explanations. Large language models are exceptionally good at producing polished explanations. That's a dangerous combination. The more capable our agents become, the more important independent verification becomes. Most discussions around AI coding focus on: Those are useful questions. But I think the more important question is: How do we prove an AI-generated change is safe to ship? Not trust. Not confidence. Proof. That's the idea behind Thea Software , an open-source project I'm building. Protocol: a change maps to proof; Thea selects evidence and the build enforces the verdict evidence https://github.com/HLIntel/TheaSoftware what-it-measurably-buys . Agents: llms.txt https://github.com/HLIntel/TheaSoftware/llms.txt ; chats: CHAT.md https://github.com/HLIntel/TheaSoftware/CHAT.md . Start: install/use https://github.com/HLIntel/TheaSoftware quickstart · architecture https://github.com/HLIntel/TheaSoftware where-it-fits · enterprise https://github.com/HLIntel/TheaSoftware enterprise-and-companion-runtimes · agents https://github.com/HLIntel/TheaSoftware/llms.txt · chat https://github.com/HLIntel/TheaSoftware/CHAT.md · adapters https://github.com/HLIntel/TheaSoftware/models/README.md An agent can report a change as done without running a single check. A check that never ran looks the same as a check that passed, and the change ships. Thea closes that gap: atlas.yaml SKIP , counted, and kept out of the pass count. Gates report… The premise is simple: Instead of treating an agent's summary as truth, verification should come from deterministic checks, recorded evidence, and an auditable outcome. The project positions itself as a "verdict layer for AI-written software" built around evidence-backed verification and enforceable checks. The biggest lesson wasn't technical. It was psychological. For decades, engineers learned: Don't trust user input. I think AI is teaching us a new version of the same lesson: Don't trust agent output. Verify it. Every time. Because eventually most software will involve AI-generated changes. When that happens, the question won't be "Which model wrote this?" The question will be: What evidence exists that it was verified? The organizations that answer that question well will move faster and safer than everyone else. And I suspect that's where the next generation of software tooling is headed. What edge cases did I miss here? If an AI coding assistant tells you, "Tests passed," what proof do you require before merging?