# My fact-checker said CONFIRMED about a group that doesn't exist

> Source: <https://dev.to/edycutjong/my-fact-checker-said-confirmed-about-a-group-that-doesnt-exist-2eo8>
> Published: 2026-09-23 00:13:27+00:00

I built a tool that fact-checks crypto claims. You paste a sentence like *"Whales are holding $XYZ"* and it plans the on-chain calls that claim needs, runs them against [Nansen](https://nansen.ai)'s API, and answers **CONFIRMED / OVERSTATED / CONTRADICTED / UNVERIFIABLE** with the numbers that decided it.

Last week it answered **CONFIRMED** about a group of wallets that does not exist.

Not "returned a weak signal." Not "was slightly off." It printed the word that means *this claim is true*, next to a cohort the data source tags exactly zero members of. Here is how a tool whose entire purpose is refusing to overstate ended up overstating, and why the README had already warned me.

Three limitations ship in the repo's `JUDGE.md`. The second one says this, verbatim:

**Nansen's Whale label is sparse.** A post's "whale" is often a big wallet Nansen does not tag; when no Whale-labelled wallet exists in the token the tool says UNVERIFIABLE rather than pretending a CONTRADICTED.

I wrote that line early, and I wrote it because it is *true of the data*. My API notes from the build (`docs/DX-REPORT.md`) record the measurement it came from:

`tgm/flow-intelligence` 1d on WETH: `whale_wallet_count = 0`. `tgm/holders label_type=whale` on the same token: 38 holders.

Zero labelled whales on wrapped Ether. Not an edge case — the label is genuinely sparse, and the flow columns that report it are DEX-only. "No wallets of this class" is an ordinary, frequent answer from this API, not a rare one.

So I knew. I documented it. And then I enforced it in exactly one of the two places it mattered.

Claims come in three shapes: *buying*, *selling*, *holding*. The first two go down one branch, holding goes down another. Both need the same question answered first — **does this class of wallet exist in this token at all?** That question has a function:

```
/** Does the subject class exist in this token at all (any window, any endpoint)? */
export function presence(e: Evidence, cls: "smart_trader" | "whale"): boolean {
  return (
    (e.flow1d?.[cls]?.wallets ?? 0) > 0 ||
    (e.flow7d?.[cls]?.wallets ?? 0) > 0 ||
    (e.holders?.count ?? 0) > 0 ||
    (cls === "smart_trader" && (e.table?.traders ?? 0) > 0) ||
    (e.named ? e.named.buyRows + e.named.sellRows > 0 : false)
  );
}
```

The flow branch calls it, and does the right thing:

```
if (!presence(e, cls))
  return R("UNVERIFIABLE", "U-NOCLASS", [
    `Nansen tags no wallet as ${who} in this token (24 h, 7 d, holders) — ` +
    `the wallet in the post is not one Nansen labels`,
  ]);
return R("CONTRADICTED", "C-NOBODY", [ /* ... */ ]);
```

The holding branch never called it. Not once. Here is what it did instead — read it as a ladder, because that is how it fails:

``` js
function decideHolding(claim, e, rules, T7, R) {
  const who = subjectName(claim.subject);   // "Whales" | "Smart Money"
  const cls = subjectClass(claim.subject);  // "whale"  | "smart_trader"
  const h = e.holders;
  const net7 = e.flow7d?.[cls]?.net ?? null;
  if (!h && net7 == null) return R("UNVERIFIABLE", "U-HOLD", [/* no data at all */]);

  // ↓ no presence() check here ↓

  if (h && h.count < rules.minHolders) return R("OVERSTATED", "O-HOLDERS", [/* ... */]);
  if (h && net7 != null && net7 <= -T7 && h.delta7d < 0) return R("CONTRADICTED", "C-EXIT", [/* ... */]);
  if ((h && h.delta7d < 0) || (net7 != null && net7 <= -T7)) return R("OVERSTATED", "O-TRIM", [/* ... */]);
  return R("CONFIRMED", "A-HOLD", [/* "balances are not shrinking" */]);
}
```

**Zero holders.** `tgm/holders` answers with an empty page, so `h.count === 0`. The first rung catches it — `0 < 5` — and returns `OVERSTATED`, whose UI copy means *partly true*. Partly true about nobody. Bad, but at least it is hedging.

**The holders call failed.** Now `h` is `null`, and the seven-day flow answered with `wallets: 0`. Look at the ladder again: every remaining rung is guarded by `h &&`. `O-HOLDERS` needs `h`. `C-EXIT` needs `h`. `O-TRIM` needs `h` or a net flow below the negative threshold — and the net flow is `0`, which is not below anything. So the claim falls all the way through and lands on the last line:

```
{ label: "CONFIRMED", rule: "A-HOLD", presence: false,
  reasons: [ "7 d Whales net flow $0 (threshold $5K)",
             "Whales balances are not shrinking" ] }
```

"Whales balances are not shrinking." Technically unfalsifiable and completely true, in the way that *"all the unicorns in my garage are healthy"* is true. There are no whales. Nothing is shrinking because there is nothing. And the tool rendered that as **CONFIRMED**, the strongest word it owns, on the one screen a reader actually looks at.

The path needs no exotic input. One failed HTTP call to a sparse endpoint, on a token whose class is empty — which, per my own notes, is WETH.

```
if (!presence(e, cls))
  return R("UNVERIFIABLE", "U-NOCLASS", [/* same message the flow path uses */]);
```

One line, plus a comment, plus 45 lines of tests. The whole commit is `+51`.

I found the first case because I ran the source past an outside model, which spotted the zero-holders rung. Verifying its claim is what turned up the second: I wrote a throwaway test with `holders: null` to see what the other branch did, and got `CONFIRMED` back. The review found the hedge; checking the review found the lie.

All 13 recorded fixtures still reproduce with byte-identical sha256 evidence hashes after the change, which is the thing that let me ship a one-line edit to the verdict engine four days before a deadline without flinching.

**A limitation stated in prose is not a constraint.** I had the right belief, written in the right file, in public, in the artifact judges read. It did nothing. Prose cannot fail a build.

**An invariant enforced in one branch is enforced in zero branches.** `presence()` existed. It was correct. It was even *called* — just not on every path that needed it. A shared helper that callers must remember to call is a convention, and conventions decay the moment someone adds a fourth claim type.

**The dangerous bug was in the fallback, not the happy path.** Every test I had written covered tokens *with* data, because that is what the demo uses and what the fixtures recorded. The failure mode lived where the data was absent — `h === null`, `wallets: 0`, a call that didn't answer — and absence is exactly what nobody writes fixtures for. If you have a decision function with a default return at the bottom, the question worth asking is not "is this the right default" but **"what is the emptiest input that can reach this line?"**

**Sparse-by-design data is a first-class input.** I treated "no whales" as degenerate. It is not. It is Wednesday.

`UNVERIFIABLE`, not an investigation of who that wallet is.` flow-intelligence` vs `smart-money/netflow`). I surface both rather than reconcile them; `docs/DX-REPORT.md` has the measurement.
For the numbers people ask for: 352 tests, 100% statement/branch/function/line coverage on the engine, 13 fixtures that replay offline with zero network calls, cold p50 4.0 s / p95 5.0 s, ~10 API credits per verdict, 0 of 154 live calls failed in the benchmark. Details in `docs/BENCH.md`.

The tool is at **[https://rebuttal.edycu.dev](https://rebuttal.edycu.dev)** and the code is at **[https://github.com/edycutjong/rebuttal](https://github.com/edycutjong/rebuttal)** — the commit in this post is `fedb50b`, and the test that pins both failure paths is `packages/core/test/review4.test.ts`.

If you have a verdict function with a `return` at the bottom of a ladder, go and check what the emptiest possible input does to it. That is the whole article.