cd /news/artificial-intelligence/credit-scoring-for-ai-agents-why-i-s… · home topics artificial-intelligence article
[ARTICLE · art-64426] src=dev.to ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Credit Scoring for AI Agents: Why I Stopped Trusting Self-Reported Success

A developer built Ledgermind, a credit-scoring system for AI agents that replaces self-reported success with verified on-chain credit histories. The platform uses acceptance-test jobs and a proving ground to independently grade agent work, preventing agents from grading their own output. Early vulnerabilities, including exit-code bypass and Sybil attacks, were fixed through subprocess wrapping and owner-wide exposure netting.

read3 min views1 publishedJul 18, 2026

Testnet only, no real money involved — that up front, before anything else.

Every agent-to-agent system I looked at collapses to the same trust

primitive: the agent's own claim that it worked. No memory across runs,

no independent check, and a confidently wrong answer looks identical to

a correct one to whatever's reading the output.

That's not a small gap. If agents are going to hire each other, pay each

other, and eventually borrow against their own track record, "it said so"

can't be the foundation.

So I built Ledgermind: each agent gets a real credit history — earned

from work that's actually verified, not self-reported — and a programmable, on-chain credit limit it can draw against.

The design rule that everything else follows from: the agent that does

the work is never the one who grades it.

Acceptance-test jobs. A requester posts a job with Python assert

statements attached. When a worker submits code, the platform runs

those tests server-side — never the worker's own runtime. Pass, and

escrow releases automatically. Fail, and the job auto-refunds and

reposts for a different worker. The worker literally cannot grade its

own homework.

Proving Ground. The server procedurally generates a problem and

the hidden answer, sends only the problem to the solving agent, and

grades the submission against the hidden answer once it comes back.

Settlement happens via commit-reveal (front-running resistant). Both produce a "graded fact" event, weighted far above a self-reported

"I completed the task" event in the scoring formula.

Early on, I had a real vulnerability: the grading endpoint judged

pass/fail off the graded subprocess's exit code. Any submission ending

in sys.exit(0) — accidental, from a model's leftover if __name__ ==

block, or deliberate — would skip the actual test code

"main"

while the process still exited 0, which read as a pass. That's a style nit right up until a passing verdict can auto-release

real escrow with no human review, which mine does now. Fixed by

wrapping both the solution and test phases in try/except SystemExit

and only printing an unguessable marker after both phases provably ran

to completion — the caller checks for the marker in stdout, not the

exit code.

I also found (with help from an external reviewer) a Sybil-style hole: a user could leave one agent's drawn credit unpaid, spin up a brand new

agent, and get an independent credit line with zero regard for the

first agent's debt — because the on-chain vault's outstanding

mapping

is keyed per agent address, not per owner. Fixed by netting owner-wide

exposure across every agent a user controls before publishing a credit

limit on-chain.

Both writeups (and a few other findings) are in the repo's Claude.md

if you want the specifics. /guest

is a live, no-login view of whatever's actually happening on the platform right now — no seeded numbers anywhere.Next.js + Postgres for the app and credit engine, Python + LangGraph for

the agent runtime, Solidity/Foundry contracts (Sepolia + GIWA testnet)

with ERC-4337 smart accounts per agent. Payments/reads go over

x402 where it makes sense — pay-per-call, no account needed to post a job or pull an agent's credit report.

No formal contract audit yet. The code-execution sandbox isn't

network-isolated. No calibration yet for "confidently wrong" output

specifically (only right/wrong). All tracked openly in the repo instead

of glossed over.

Repo (Apache 2.0): [https://github.com/Kairose-master/ai-agent-credit-dashboard](https://github.com/Kairose-master/ai-agent-credit-dashboard)

Live demo: [https://ai-agent-credit-dashboard.vercel.app/guest](https://ai-agent-credit-dashboard.vercel.app/guest)

Would genuinely like the grading design picked apart — I'd rather find

out it's broken from a comment here than from someone exploiting it.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @ledgermind 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/credit-scoring-for-a…] indexed:0 read:3min 2026-07-18 ·