Show HN: AgentsProof – a small project for testing AI agents

A few SDK calls capture your agent's trace, grade it against your custom rules and approved test cases, and create a shareable proof report — in minutes.

You changed a prompt. Something broke. A user noticed first.

import { AgentsProof } from 'agentsproof';

const ap = new AgentsProof({ apiKey: process.env.AGENTSPROOF_API_KEY });
const run = ap.startRun({
  projectSlug: 'my-agent',
  input: { query },
  goal: 'Search the web and return a working code solution', // anchors grading
});

const result = await run.trace('llm_call', 'gpt-4o', () => llm(query));
const { publicUrl } = await run.complete({ answer: result });

console.log(publicUrl); // → https://agentsproof.dev/r/abc123

One package, zero config. Works in any Node or edge runtime.

Drop run.trace() around each LLM and tool call. That's the whole integration.

Write rules in plain English — the LLM checks every run against them automatically.

Turn a passing run into a live test spec. Set success criteria, trace assertions, and expected behavior — all evaluated on every future run.

One SDK call runs all approved Goldens through your agent. The report shows per-criterion pass/fail for every case — not just an overall score.

import { AgentsProof } from 'agentsproof';

const ap = new AgentsProof({ apiKey: process.env.AGENTSPROOF_API_KEY });
const run = ap.startRun({
  projectSlug: 'my-agent',
  input: { query },
  goal: 'Search the web and return a working code solution', // anchors grading
});

const result = await run.trace('llm_call', 'gpt-4o', () => llm(query));
const { publicUrl } = await run.complete({ answer: result });

console.log(publicUrl); // → https://agentsproof.dev/r/abc123

Define what 'good' means for your agent in plain English. Every run is automatically graded against your rules — pin specific graders to individual Goldens for targeted enforcement.

Turn any passing run into an executable test spec with success criteria, expected behavior, and trace assertions. Pass goldenId directly to startRun() to run against a Golden on demand — or batch them all in a proof suite.

Structured checks that run deterministically — no LLM needed. If your agent skips a required step or exceeds a step budget, the case fails immediately. Catches regressions the LLM judge can miss.

AgentsProof uses AI to generate edge-case variants from your Goldens, growing your test suite without writing tests by hand.

One SDK call runs all approved Goldens through your agent. The report shows per-criterion Golden checks alongside the 5-axis score — every pass and fail is explainable.

import { AgentsProof } from 'agentsproof';

const ap = new AgentsProof({ apiKey: process.env.AGENTSPROOF_API_KEY });

await ap.runProofSuite({
  projectSlug: 'my-agent',
  suiteSlug: 'core-behaviors',
  async handler(input, ctx) {
    // input comes from the approved Golden
    const run = ctx.startRun();
    const result = await myAgent(input);
    await run.complete({ answer: result });
  },
});
// → https://agentsproof.dev/p/suite-abc123

~ partial support · based on publicly available information