Matrix – Check whether your AI agent actually did what it claimed

A developer has launched Matrix, a verification tool that checks whether an AI agent actually performed the actions it claims by querying the authoritative external system — currently the Gmail mailbox — rather than trusting the agent's own trace. The tool returns confirmed, contradicted, or inconclusive verdicts with supporting evidence, and its creator notes it cannot catch a correct tool call made with the wrong argument, supports only Gmail and LangChain, and has zero users as of launch.

Agents report success for actions that never happened — no error, clean trace, and every observability tool reads it as a success, because they're all reading the agent's own account of itself. This doesn't read the trace differently. It queries the authoritative system instead — the actual Gmail mailbox — and returns confirmed / contradicted / inconclusive with the evidence attached: the account queried, the search window, and what was found in it. Two failure classes, different evidence. If there's no tool call in the trace at all, the absence is the evidence and nothing external is needed. If the call happened and returned cleanly but nothing landed, only the mailbox can tell you. The parts I'd most want criticised: Inconclusive is a first-class verdict. If the trace can't be shown complete, or the tool span has a name I don't recognise, or the mailbox can't be established — it refuses to judge rather than calling a working agent a liar. A false accusation costs more than a missed detection. I may have that balance wrong. It cannot catch a correct call with a wrong argument. Asked to mail one person, confidently mails another — the send is real, Gmail confirms it, verdict is confirmed, correctly. The instruction is recorded next to the arguments so a human can see it. No verdict catches it. Gmail only so far. LangChain via a callback handler, or the SDK by hand. No n8n, no CrewAI. Zero users — this is day one. matrixverify.dev