cd /news/ai-agents/don-t-trust-the-score-a-fraud-invest… Β· home β€Ί topics β€Ί ai-agents β€Ί article
[ARTICLE Β· art-139193] src=dev.to β†— pub= topic=ai-agents verified=true sentiment=↑ positive

Don't Trust the Score: A Fraud Investigator That Argues Both Sides on TigerGraph

A developer built Fraud Investigator, a TigerGraph-backed fraud detection agent that treats bank risk scores as one weak witness rather than a verdict, after profiling five provided files with DuckDB showed all 900 cleared cases scored 0.81 or higher while 31% of confirmed fraud scored under 0.3. The system routes all reads through TigerGraph's MCP tool surface and all state-changing actions through a deterministic policy engine, using three schema-checked LLM calls per case for planning, explanation and SAR writing. Graph features reached an AUC of 0.91 versus 0.62 for the bank score among alerts at or above 0.8, and a score-blind sweep surfaced a hidden fraud ring of 60 transactions across 28 customers that no closed case contained.

by read12 min views1 publishedSep 24, 2026

πŸŽ₯ Demo video: Watch the Fraud Investigator in action

Most fraud-agent demos work the same way: a model produces a risk score, then an LLM writes a convincing story about why the score is right.

Before writing any code, We profiled all five provided files with DuckDB. The data showed that approach would fail in both directions: it blocks innocent people and misses real fraud.

What I measured Result Design consequence
Cleared (false-alarm) cases All 900 scored0.81 or higher A high score is where innocence lives, so the agent must be able to clear a 0.9
Confirmed fraud Mean score 0.47 , and31% scored under 0.3 The agent must be able to convict a 0.05
A hidden fraud ring 60 transactions, 28 customers, every score ≀ 0.44 , in no closed case A score-blind sweep is needed
Graph features vs bank score, among alerts β‰₯ 0.8 AUC 0.91 vs0.62 The graph decides where the score is confused
"Customers" 104 of them hold 48% of all transactions A customer is not a person, so baselines must be per card

Here is the same story as one picture. Fraud appears at every score. Innocence appears in only one zone:

bank score bin : 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
confirmed fraud: β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ
cleared (safe) : Β· Β· Β· Β· Β· Β· Β· Β· β–ˆβ–ˆβ–ˆ β–ˆβ–ˆβ–ˆ
                                                            β–²
                              the "decoy zone": the score cannot separate them here

Chart shows where each outcome appears; it is not a count.

The score tells you where to look. The graph tells you what happened.

So I built an agent that treats the score as one weak witness, not the judge.

Fraud Investigator takes an alert from any of three triggers (a bank risk flag, a customer saying "I never made this", or an analyst asking "look for related activity") and investigates it like a small legal team:

Five layers. Everything that reads goes through TigerGraph's MCP tool surface. Everything that can change the world goes through the policy engine.

flowchart TB
  subgraph ENTRY["ENTRY"]
    direction LR
    E1["Risk-score alert"]
    E2["Customer report"]
    E3["Analyst request"]
    E4["Analyst console"]
  end

  subgraph ORCH["ORCHESTRATOR - bounded state machine, typed evidence ledger"]
    direction LR
    O1["Intake"] --> O2["Recall"] --> O3["Advocates x3"] --> O4["Judge"] --> O5["Plan"] --> O6["Gate"] --> O7["Act"] --> O8["Learn"]
  end

  subgraph ENG["DETERMINISTIC ENGINES - no LLM inside"]
    direction LR
    G1["Signature library<br/>prosecution + defence"]
    G2["Contagion + rarity<br/>personalised PageRank"]
    G3["Calibrated judge<br/>log-odds ledger"]
    G4["Policy engine<br/>rules R1 to R10"]
    G5["Response simulator<br/>denies / confirms / silent"]
  end

  subgraph TG["TIGERGRAPH via MCP server"]
    direction LR
    T1[("Transaction graph")]
    T2[("Case memory")]
    T3[("Vector indexes")]
    T4[("Installed GSQL<br/>every query takes as_of")]
  end

  subgraph OFF["OFFLINE - build time"]
    direction LR
    F1["<br/>708 MB CSV to typed graph"]
    F2["Time-machine backtest"]
    F3["Calibration tables<br/>JSON, no ML at runtime"]
  end

  LLM["LLM - 3 schema-checked calls per case<br/>plan evidence, explain, write SAR"]

  ENTRY --> ORCH
  ORCH --> ENG
  ENG --> TG
  OFF -. "loads + calibrates".-> TG
  F3 -. "coefficients".-> G3
  ORCH -. "phrasing only".-> LLM

  classDef entry fill:#F3F4F6,stroke:#6B7280,color:#111827;
  classDef agent fill:#EEF0FF,stroke:#5B5BD6,color:#1B1B4B;
  classDef tgc fill:#E6FAF6,stroke:#0F9D8A,color:#063D36;
  classDef control fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class E1,E2,E3,E4 entry;
  class O1,O2,O3,O4,O5,O6,O7,O8,LLM agent;
  class T1,T2,T3,T4,F1,F2,F3,G1,G2 tgc;
  class G3,G4,G5 control;

indigo = agent and judge, teal = graph and graph-derived work, amber = control and side effects.

# Principle Why it matters
1 The LLM only phrases It cannot add an action, change a route, set exposure, or invent an ID. That is what makes a mid-sized open model safe here.
2 Read through MCP, act through policy One loggable tool contract, and no side effect without a rule citation.
3 Time is enforced in the data layer The alert's opened_at is 1 to 6 hoursafter the flagged transaction, and the file runs to Dec 31. Every query takesas_of , and a leakage test fails if any row is later than the trigger.
4 Graceful degradation Every LLM reply is validated, retried twice with the validator's error fed back, then replaced by a deterministic template. A valid case file is always written.
5 The graph is the source of truth The answer JSON is rendered from the graph and checked: IDs exist, exposure equals the sum of affected transactions, sar.file matchesFILE_REPORT .

TigerGraph is the substrate for the reasoning, not a place to park rows. The schema is shaped around the questions the agent asks, so each question becomes a one-hop lookup.

flowchart LR
  subgraph TXN["Transaction graph"]
    Customer -->|OWNS| Card
    Card -->|MADE| Transaction
    Transaction -->|NEXT| NextTxn["Next txn on card"]
    Transaction -->|FROM_DEVICE| DeviceProfile["DeviceProfile<br/>carries rarity weight"]
    Transaction -->|BILLED_IN| BillingRegion
    Transaction -->|PURCHASER_EMAIL| EmailDomain
    Transaction -->|IN_EPISODE| Episode["Episode<br/>derived burst window"]
    Card -->|MEMBER_OF| Ring["Ring<br/>derived component"]
  end

  subgraph MEM["Case memory"]
    ClosedCase -->|MATCHES| Pattern
    Case -->|MATCHES| Pattern
    Case -->|RETRIEVED| ClosedCase
    Case -->|HAS_EVIDENCE| Evidence
    Case -->|RECOMMENDS| Action
    Action -->|UNDER_RULE| PolicyChunk
    PolicyChunk -->|DESCRIBES| Pattern
  end

  ClosedCase -->|INVOLVES| Transaction
  Case -->|INVOLVES| Transaction
  Case -->|CONNECTED_TO| Card
  Case -->|SUSPECTS| Ring
  Evidence -->|CITES| Transaction

  classDef derived fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class Episode,Ring,DeviceProfile derived;

Amber vertices are the ones I added beyond the suggested schema. They are built offline so the agent's questions stay local to a card's neighbourhood.

Rarity-weighted devices. Each DeviceProfile stores how many customers use it, its share of New marks, its share behind an anonymous proxy, and an IDF weight. A generic Windows/Chrome profile shared by 1,000 customers gets a near-zero weight. The ring fingerprint gets a high one.

My first ring rule flagged 212 of 9,706 device profiles, mostly common iPhones. Requiring 5 to 200 users, New on 90% of uses, and an anonymous proxy on 80% left the real ring and very few others.

A score-blind contagion sweep. A personalised PageRank starts from confirmed-fraud entities and runs over the card, device, region and email graph. It ranks unflagged cards by proximity to known fraud and ignores the bank score. One query on the ring fingerprint recovers victims the model never flagged.

Case memory as GraphRAG. Closed cases attach to the same transactions, cards and devices that live traffic touches, and recall works three ways:

Recall mode What it does
Structural Closed cases on the same device or card neighbourhood, ranked by overlap, rarity and recency
Semantic Vector search over case notes, policy and regulatory text
Numeric Nearest neighbours in a 32-dimension PCA of the Vesta features

The three results are compiled into one brief for the LLM: what we know, what conflicts, what is missing.

Memory uses shape matches only, never "same customer". Closed history is 84% fraud, so anyone with history would look guilty.

Causal write-back. Every new case becomes a Case vertex linked to its transactions, cards, ring and pattern, with retrieved cases as RETRIEVED edges. Cases are processed in opened_at order, so case N may recall earlier cases and never a later one.

flowchart TD
  T["Trigger"] --> I["1. Intake<br/>pin as_of, resolve card"]
  I --> M["2. Recall<br/>prior cases, 3 ways"]
  M --> PR["Prosecution"]
  M --> DE["Defence"]
  M --> SW["Sweep"]
  PR --> J["Judge<br/>p, confidence, evidence classes"]
  DE --> J
  SW --> J
  J --> S{"Stop rule met?<br/>p >= 0.85 or p <= 0.15<br/>with 2 independent classes"}
  S -- "yes"--> A["Policy engine<br/>choose actions + routes"]
  S -- "no, budget left"--> V["Plan<br/>best value-of-information request"]
  V --> R["Reply<br/>customer / step-up / analyst"]
  R --> J
  S -- "no, nothing left to try"--> E["Escalate to analyst<br/>with gap list"]
  E --> A
  A --> X["Explain, SAR, write case to graph"]

  classDef agent fill:#EEF0FF,stroke:#5B5BD6,color:#1B1B4B;
  classDef tgc fill:#E6FAF6,stroke:#0F9D8A,color:#063D36;
  classDef control fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class PR,DE,J,V,R agent;
  class SW,M tgc;
  class A,E,X,S control;

Budgets: 3 evidence rounds, about 20 graph calls, 1 request of each type.

Prosecution and Defence run in parallel over the same as-of graph. An unanswered defence test is reported as uncertainty, never ignored. The bank score is not a signature. It enters as one evidence class through a learned, non-monotonic curve.

The meaning of a signal flips by zone. At score 0.8+, 98% of cleared online alerts show a New device, against 36% of fraud. So "New device" is defence evidence in that zone and prosecution evidence below it.

Prosecution signatures Defence signatures
Test then spend Trip continuity (a clone would keep spending at home)
Off-profile burst Device succession (new phone replaces old, same brand and OS)
New-device attribution Recurring cadence, with a coincidence test
Out-of-region while home continues Baseline-consistent spend
Threshold hugging (structuring) Hub hygiene (down-weight aggregate customers)
Shared rare fingerprint Novelty detector (flags patterns nothing explains)

Fraud probability is graded for calibration and drives every policy threshold, so it is computed, not guessed:

prior (reset for 84% fraud history) β†’ + each evidence item's log-likelihood ratio β†’ same class counts once (max, not sum) β†’ global damping tuned on backtest β†’ fraud_probability

Evidence is tagged by class (bank score, card behaviour, device, geography, network, memory, lookalike, customer reply). Correlated evidence inside a class does not stack, so the policy's "at least two independent pieces" rule is computed, not asserted.

If the stop rule isn't met, the agent scores candidate requests (customer validation, step-up auth, analyst info, one more graph hop) by expected shift in p, minus friction. It must state which two hypotheses the request is meant to separate, and that sentence becomes the recorded "why more evidence was requested".

Replies aren't provided, so before asking, the agent precomputes the plan for all three outcomes. The simulated reply is drawn from its own pre-reply posterior, and the assumption is recorded.

flowchart TD
  ASK["Ask the customer"] --> D["Denies"]
  ASK --> C["Confirms"]
  ASK --> Z["Silent for 24h"]
  D --> DR["Rule R2<br/>BLOCK_CARD, CREATE_CASE,<br/>FILE_REPORT if threshold met,<br/>MONITOR_CONNECTED_CARDS"]
  C --> CR["Rule R3<br/>CLOSE_NO_FRAUD"]
  Z --> ZR["Rule R4<br/>MONITOR_CARD, DECLINE_TRANSACTION,<br/>escalate if exposure over $500"]
  classDef bad fill:#FDECEC,stroke:#D1242F,color:#4A0A0F;
  classDef good fill:#E7F8EC,stroke:#2EA043,color:#0B3D1A;
  classDef mid fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class D,DR bad;
  class C,CR good;
  class Z,ZR mid;

All three branches are stored as a contingency object, so the before-and-after recommendation is a lookup, not an improvisation, and an analyst can override the simulated reply live.

Rules R1 to R10 are a decision table in code. Exposure is computed from transaction IDs, never by the LLM. The executor only runs auto actions:

Route Actions
Auto (executed) CREATE_CASE ,MONITOR_CARD ,VERIFY_WITH_CUSTOMER ,WARN_CUSTOMER ,ESCALATE_TO_ANALYST ,CLOSE_NO_FRAUD
L1 team lead DECLINE_TRANSACTION ,BLOCK_CARD if exposure up to $2,500
L2 fraud manager BLOCK_CARD above $2,500,BLOCK_ALL_CARDS ,FILE_REPORT

L1 and L2 actions appear as approval cards in the console, each showing the rule that required it.

Any closed case can be replayed as if it had just arrived, with the graph frozen at its as_of. That is how thresholds and likelihood ratios are calibrated, and how the UI can show a live calibration plot.

The screenshot at the top is this exact run. An analyst wrote: "several cards this month show purchases from the same unusual device profile."

sequenceDiagram
  autonumber
  participant A as Analyst
  participant AG as Agent
  participant TG as TigerGraph (MCP)
  participant PE as Policy engine
  A->>AG: Review $74.96 online purchase on card T9003-K1
  AG->>TG: Card history, as_of = alert time
  TG-->>AG: 47 earlier payments
  AG->>TG: Run fraud signatures (prosecution)
  TG-->>AG: 2 signs: device is New behind anonymous proxy, shared by 12 customers
  AG->>TG: Sweep from the shared device profile
  TG-->>AG: 5 other cards still live on it, 6 already handled
  AG->>TG: Run innocence signatures (defence)
  TG-->>AG: 1 reason it could be innocent: amount within card p95 ($97.20)
  AG->>PE: p = 92%, classes: device, network, geography
  PE-->>AG: Likely fraud, $187.33 at risk across 2 payments
  AG-->>A: Verdict, reasoning with "Why" lines, customer-friendly message
Step What the agent did Why it matters
1. Read the alert Found 47 earlier payments, used only what was known at alert time No look-ahead
2. Look for fraud 2 signs, including a rare New device behind an anonymous proxy used by 12 customers Graph-derived evidence, not the score
3. Look for innocence Found 1 reason: amount is normal for this card The defence gets a real hearing
4. Verdict 92%, very likely fraud ,$187.33 at risk across 2 payments Computed by the judge and policy engine, not by the LLM
5. Explain Customer-friendly message, next steps Plain language, not a risk-model dump

Each step has an italic "Why" line, and a "slow down so I can watch" toggle lets a human follow the reasoning live.

Typical fraud agent Fraud Investigator
Trusts the bank score Treats the score as one witness on a learned, non-monotonic curve
Hunts only for fraud signals Prosecution and defence in parallel
"Same customer" memory Shape-matched memory, aware that customers are aggregates
LLM decides the action LLM only phrases ; a policy engine decides
Probability from vibes Log-odds ledger with independence classes
Sees the future by accident as_of enforced in thedata layer , with a leakage test
Only finds flagged cards Score-blind sweep finds the ring at score 0.05
Area Status Next step
Full time-machine backtest Harness and replay built Publish per-pattern accuracy, per-signature precision and the calibration curve [insert results]
Account-takeover signature Not built Add to the prosecution set
Numeric lookalike channel Not built PCA-32 vectors and kNN restricted to labelled exemplars
Full PageRank sweep One-hop version today Run personalised PageRank in-database with the GDS library
Vector search in TigerGraph Planned Move semantic recall in-database
Monitor mode Optional Let the agent raise alerts on its own from graph contagion
Learn from analysts Candidate patterns proposed Feed analyst-named patterns back into signatures and calibration
Narration model comparison Planned Compare models on schema pass rate, citation accuracy and SAR completeness

A fraud agent should not be a persuasive storyteller. It should be a disciplined investigator that argues both sides, states its uncertainty, cites everything, and leaves the final decision to auditable policy. TigerGraph is what makes that possible: it holds the transactions, the memory and the time-bounded queries in one place, and the agent reaches all of it through a single MCP tool surface.

TigerGraph Β· TigerGraph MCP server Β· GSQL Β· Ollama (gemma4:31b-cloud, nomic-embed-text) Β· Next.js + TypeScript Β· Zod Β· DuckDB Β· Python

https://www.youtube.com/watch?v=kXDzmKNrhIE https://github.com/shreesha345/hhg-fraud-agent https://hhg-goa-frontend-hc8k.vercel.app/ #TigerGraph @TigerGraphDB

── more in #ai-agents 4 stories Β· sorted by recency
── more on @tigergraph 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/don-t-trust-the-scor…] indexed:0 read:12min 2026-09-24 Β· β€”