# Hot take: AI judges aren't the problem with AI-judged bounties. All-or-nothing thresholds are.

> Source: <https://dev.to/drdz23/hot-take-ai-judges-arent-the-problem-with-ai-judged-bounties-all-or-nothing-thresholds-are-5bde>
> Published: 2026-10-09 02:07:24+00:00

Everyone worries that an AI jury will be unfair. After submitting real work to one, I think that's the wrong worry. **The AI judges were mostly reasonable. The payout rule around them is what's broken: "pass the threshold or get zero".**

Here's my evidence, and then I want you to tell me I'm wrong.

On [Verdikta Bounties](https://bounties.verdikta.org), a creator locks ETH in an escrow contract, writes a weighted rubric and sets a pass threshold. Two AI models (one from OpenAI, one from Anthropic) score every submission, and the contract pays only if the weighted score clears the bar.

In two days I got these results:

| Submission | Threshold | Score | Paid | 
|---|---|---|---|
| A short bio ( [#139](https://bounties.verdikta.org/bounty/139) ) | 50% | 91% | 0.01 ETH | 
| A case study, first try | 90% | 63.5% | 0 | 
| A comparison post | 90% | 80.3% | 0 | 
| The same case study, second try | 90% | 91% | 0.002 ETH | 

Look at row 3. **80.3% is a solid B, and it paid exactly the same as a blank page.**

I read the published reasoning for every verdict. The models weren't being random. My first case study lost points because I only attached a screenshot and a link, and the jury said it couldn't see most of the text. On the second try I attached the full post and added on-chain detail, and it scored 91%.

So the judges did their job: they scored what they could see, they explained why, and the explanation was right. Being judged by a machine wasn't the painful part. The cliff was.

**1. Two models disagree, and the threshold amplifies it.** On one of my submissions, one model gave 84 and the other gave 65 for the same text. Averaging them is fine. But when the bar is 90%, a single skeptical model is enough to turn "pretty good" into "zero". The threshold turns ordinary model disagreement into an all-or-nothing coin flip.

**2. High thresholds select for gaming, not quality.** At 90% or 95%, the winning strategy is to reverse-engineer the rubric and pad every criterion, not to write the most useful thing. I did exactly that on my second try, and it worked. That should worry anyone who wants good work instead of rubric-shaped work.

**3. It burns the people you most want to keep.** A new contributor who scores 80% and gets nothing is unlikely to come back. A human client would have said "good, fix these two things" and paid something.

The machinery (escrow, a public rubric, commit-reveal voting between arbiters, published reasoning) is already better than most human review I've seen. **The scoring is the part that works. The payout rule is the part that doesn't.**

Creators will say thresholds stop spam and low-effort submissions. Fair. But is a hard cliff really the only way to do that? **Would you rather earn 70% of a bounty for 80% work, or keep the cliff?** Reply with your side. I'll read every one.

Bounty used as the main example: [https://bounties.verdikta.org/bounty/139](https://bounties.verdikta.org/bounty/139)