# Don't Trust the Score: A Fraud Investigator That Argues Both Sides on TigerGraph

> Source: <https://dev.to/pranav_ssalian_be1687de7/dont-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph-299n>
> Published: 2026-09-24 17:06:44+00:00

**🎥 Demo video:** [Watch the Fraud Investigator in action](https://www.youtube.com/watch?v=kXDzmKNrhIE)

Most fraud-agent demos work the same way: a model produces a risk score, then an LLM writes a convincing story about why the score is right.

Before writing any code, We profiled all five provided files with DuckDB. The data showed that approach would **fail in both directions**: it blocks innocent people and misses real fraud.

| What I measured | Result | Design consequence | 
|---|---|---|
| Cleared (false-alarm) cases | **All 900** scored**0.81 or higher** | A high score is where innocence lives, so the agent must be able to *clear a 0.9* | 
| Confirmed fraud | Mean score **0.47** , and**31% scored under 0.3** | The agent must be able to *convict a 0.05* | 
| A hidden fraud ring | 60 transactions, 28 customers, every score **≤ 0.44** , in no closed case | A **score-blind** sweep is needed | 
| Graph features vs bank score, among alerts ≥ 0.8 | AUC **0.91** vs**0.62** | The **graph** decides where the score is confused | 
| "Customers" | 104 of them hold **48%** of all transactions | A customer is not a person, so baselines must be **per card** | 

Here is the same story as one picture. Fraud appears at every score. Innocence appears in only one zone:

```
bank score bin : 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
confirmed fraud: ███ ███ ███ ███ ███ ███ ███ ███ ███ ███
cleared (safe) : · · · · · · · · ███ ███
                                                            ▲
                              the "decoy zone": the score cannot separate them here
```

*Chart shows where each outcome appears; it is not a count.*

**The score tells you where to look. The graph tells you what happened.**

So I built an agent that treats the score as one weak witness, not the judge.

**Fraud Investigator** takes an alert from any of three triggers (a bank risk flag, a customer saying *"I never made this"*, or an analyst asking *"look for related activity"*) and investigates it like a small legal team:

Five layers. **Everything that reads goes through TigerGraph's MCP tool surface. Everything that can change the world goes through the policy engine.**

```
flowchart TB
  subgraph ENTRY["ENTRY"]
    direction LR
    E1["Risk-score alert"]
    E2["Customer report"]
    E3["Analyst request"]
    E4["Analyst console"]
  end

  subgraph ORCH["ORCHESTRATOR - bounded state machine, typed evidence ledger"]
    direction LR
    O1["Intake"] --> O2["Recall"] --> O3["Advocates x3"] --> O4["Judge"] --> O5["Plan"] --> O6["Gate"] --> O7["Act"] --> O8["Learn"]
  end

  subgraph ENG["DETERMINISTIC ENGINES - no LLM inside"]
    direction LR
    G1["Signature library<br/>prosecution + defence"]
    G2["Contagion + rarity<br/>personalised PageRank"]
    G3["Calibrated judge<br/>log-odds ledger"]
    G4["Policy engine<br/>rules R1 to R10"]
    G5["Response simulator<br/>denies / confirms / silent"]
  end

  subgraph TG["TIGERGRAPH via MCP server"]
    direction LR
    T1[("Transaction graph")]
    T2[("Case memory")]
    T3[("Vector indexes")]
    T4[("Installed GSQL<br/>every query takes as_of")]
  end

  subgraph OFF["OFFLINE - build time"]
    direction LR
    F1["Loader<br/>708 MB CSV to typed graph"]
    F2["Time-machine backtest"]
    F3["Calibration tables<br/>JSON, no ML at runtime"]
  end

  LLM["LLM - 3 schema-checked calls per case<br/>plan evidence, explain, write SAR"]

  ENTRY --> ORCH
  ORCH --> ENG
  ENG --> TG
  OFF -. "loads + calibrates".-> TG
  F3 -. "coefficients".-> G3
  ORCH -. "phrasing only".-> LLM

  classDef entry fill:#F3F4F6,stroke:#6B7280,color:#111827;
  classDef agent fill:#EEF0FF,stroke:#5B5BD6,color:#1B1B4B;
  classDef tgc fill:#E6FAF6,stroke:#0F9D8A,color:#063D36;
  classDef control fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class E1,E2,E3,E4 entry;
  class O1,O2,O3,O4,O5,O6,O7,O8,LLM agent;
  class T1,T2,T3,T4,F1,F2,F3,G1,G2 tgc;
  class G3,G4,G5 control;
```

*indigo = agent and judge, teal = graph and graph-derived work, amber = control and side effects.*

| # | Principle | Why it matters | 
|---|---|---|
| 1 | **The LLM only phrases** | It cannot add an action, change a route, set exposure, or invent an ID. That is what makes a mid-sized open model safe here. | 
| 2 | **Read through MCP, act through policy** | One loggable tool contract, and no side effect without a rule citation. | 
| 3 | **Time is enforced in the data layer** | The alert's `opened_at` is 1 to 6 hours*after* the flagged transaction, and the file runs to Dec 31. Every query takes`as_of` , and a leakage test fails if any row is later than the trigger. | 
| 4 | **Graceful degradation** | Every LLM reply is validated, retried twice with the validator's error fed back, then replaced by a deterministic template. A valid case file is always written. | 
| 5 | **The graph is the source of truth** | The answer JSON is rendered from the graph and checked: IDs exist, exposure equals the sum of affected transactions, `sar.file` matches`FILE_REPORT` . | 

TigerGraph is the substrate for the reasoning, not a place to park rows. The schema is shaped around the *questions the agent asks*, so each question becomes a one-hop lookup.

``` php
flowchart LR
  subgraph TXN["Transaction graph"]
    Customer -->|OWNS| Card
    Card -->|MADE| Transaction
    Transaction -->|NEXT| NextTxn["Next txn on card"]
    Transaction -->|FROM_DEVICE| DeviceProfile["DeviceProfile<br/>carries rarity weight"]
    Transaction -->|BILLED_IN| BillingRegion
    Transaction -->|PURCHASER_EMAIL| EmailDomain
    Transaction -->|IN_EPISODE| Episode["Episode<br/>derived burst window"]
    Card -->|MEMBER_OF| Ring["Ring<br/>derived component"]
  end

  subgraph MEM["Case memory"]
    ClosedCase -->|MATCHES| Pattern
    Case -->|MATCHES| Pattern
    Case -->|RETRIEVED| ClosedCase
    Case -->|HAS_EVIDENCE| Evidence
    Case -->|RECOMMENDS| Action
    Action -->|UNDER_RULE| PolicyChunk
    PolicyChunk -->|DESCRIBES| Pattern
  end

  ClosedCase -->|INVOLVES| Transaction
  Case -->|INVOLVES| Transaction
  Case -->|CONNECTED_TO| Card
  Case -->|SUSPECTS| Ring
  Evidence -->|CITES| Transaction

  classDef derived fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class Episode,Ring,DeviceProfile derived;
```

*Amber vertices are the ones I added beyond the suggested schema. They are built offline so the agent's questions stay local to a card's neighbourhood.*

**Rarity-weighted devices.** Each `DeviceProfile` stores how many customers use it, its share of *New* marks, its share behind an anonymous proxy, and an IDF weight. A generic Windows/Chrome profile shared by 1,000 customers gets a near-zero weight. The ring fingerprint gets a high one.

My first ring rule flagged **212 of 9,706** device profiles, mostly common iPhones. Requiring **5 to 200 users, *New* on 90% of uses, and an anonymous proxy on 80%** left the real ring and very few others.

**A score-blind contagion sweep.** A personalised PageRank starts from confirmed-fraud entities and runs over the card, device, region and email graph. It ranks *unflagged* cards by proximity to known fraud and ignores the bank score. One query on the ring fingerprint recovers victims the model never flagged.

**Case memory as GraphRAG.** Closed cases attach to the same transactions, cards and devices that live traffic touches, and recall works three ways:

| Recall mode | What it does | 
|---|---|
| **Structural** | Closed cases on the same device or card neighbourhood, ranked by overlap, rarity and recency | 
| **Semantic** | Vector search over case notes, policy and regulatory text | 
| **Numeric** | Nearest neighbours in a 32-dimension PCA of the Vesta features | 

The three results are compiled into one brief for the LLM: *what we know, what conflicts, what is missing*.

Memory uses **shape matches only**, never "same customer". Closed history is 84% fraud, so anyone with history would look guilty.

**Causal write-back.** Every new case becomes a `Case` vertex linked to its transactions, cards, ring and pattern, with retrieved cases as `RETRIEVED` edges. Cases are processed in `opened_at` order, so case N may recall earlier cases and **never a later one**.

``` php
flowchart TD
  T["Trigger"] --> I["1. Intake<br/>pin as_of, resolve card"]
  I --> M["2. Recall<br/>prior cases, 3 ways"]
  M --> PR["Prosecution"]
  M --> DE["Defence"]
  M --> SW["Sweep"]
  PR --> J["Judge<br/>p, confidence, evidence classes"]
  DE --> J
  SW --> J
  J --> S{"Stop rule met?<br/>p >= 0.85 or p <= 0.15<br/>with 2 independent classes"}
  S -- "yes"--> A["Policy engine<br/>choose actions + routes"]
  S -- "no, budget left"--> V["Plan<br/>best value-of-information request"]
  V --> R["Reply<br/>customer / step-up / analyst"]
  R --> J
  S -- "no, nothing left to try"--> E["Escalate to analyst<br/>with gap list"]
  E --> A
  A --> X["Explain, SAR, write case to graph"]

  classDef agent fill:#EEF0FF,stroke:#5B5BD6,color:#1B1B4B;
  classDef tgc fill:#E6FAF6,stroke:#0F9D8A,color:#063D36;
  classDef control fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class PR,DE,J,V,R agent;
  class SW,M tgc;
  class A,E,X,S control;
```

*Budgets: 3 evidence rounds, about 20 graph calls, 1 request of each type.*

Prosecution and Defence run **in parallel over the same as-of graph**. An unanswered defence test is reported as uncertainty, never ignored. The bank score is not a signature. It enters as one evidence class through a learned, **non-monotonic** curve.

**The meaning of a signal flips by zone.** At score 0.8+, **98%** of cleared online alerts show a *New* device, against **36%** of fraud. So "New device" is **defence** evidence in that zone and **prosecution** evidence below it.

| Prosecution signatures | Defence signatures | 
|---|---|
| Test then spend | Trip continuity (a clone would keep spending at home) | 
| Off-profile burst | Device succession (new phone replaces old, same brand and OS) | 
| New-device attribution | Recurring cadence, with a *coincidence test* | 
| Out-of-region while home continues | Baseline-consistent spend | 
| Threshold hugging (structuring) | Hub hygiene (down-weight aggregate customers) | 
| Shared rare fingerprint | Novelty detector (flags patterns nothing explains) | 

Fraud probability is graded for calibration and drives every policy threshold, so it is **computed, not guessed**:

`prior (reset for 84% fraud history)` → `+ each evidence item's log-likelihood ratio` → `same class counts once (max, not sum)` → `global damping tuned on backtest` → `fraud_probability`

Evidence is tagged by class (bank score, card behaviour, device, geography, network, memory, lookalike, customer reply). Correlated evidence inside a class does not stack, so the policy's *"at least two independent pieces"* rule is **computed, not asserted**.

If the stop rule isn't met, the agent scores candidate requests (customer validation, step-up auth, analyst info, one more graph hop) by *expected shift in p, minus friction*. It must state **which two hypotheses the request is meant to separate**, and that sentence becomes the recorded "why more evidence was requested".

Replies aren't provided, so before asking, the agent **precomputes the plan for all three outcomes**. The simulated reply is drawn from its own pre-reply posterior, and the assumption is recorded.

``` php
flowchart TD
  ASK["Ask the customer"] --> D["Denies"]
  ASK --> C["Confirms"]
  ASK --> Z["Silent for 24h"]
  D --> DR["Rule R2<br/>BLOCK_CARD, CREATE_CASE,<br/>FILE_REPORT if threshold met,<br/>MONITOR_CONNECTED_CARDS"]
  C --> CR["Rule R3<br/>CLOSE_NO_FRAUD"]
  Z --> ZR["Rule R4<br/>MONITOR_CARD, DECLINE_TRANSACTION,<br/>escalate if exposure over $500"]
  classDef bad fill:#FDECEC,stroke:#D1242F,color:#4A0A0F;
  classDef good fill:#E7F8EC,stroke:#2EA043,color:#0B3D1A;
  classDef mid fill:#FFF4DC,stroke:#D98E04,color:#4A3000;
  class D,DR bad;
  class C,CR good;
  class Z,ZR mid;
```

All three branches are stored as a `contingency` object, so the before-and-after recommendation is a **lookup, not an improvisation**, and an analyst can override the simulated reply live.

Rules R1 to R10 are a decision table in code. Exposure is computed from transaction IDs, never by the LLM. The executor only runs `auto` actions:

| Route | Actions | 
|---|---|
| **Auto** (executed) | `CREATE_CASE` ,`MONITOR_CARD` ,`VERIFY_WITH_CUSTOMER` ,`WARN_CUSTOMER` ,`ESCALATE_TO_ANALYST` ,`CLOSE_NO_FRAUD` | 
| **L1 team lead** | `DECLINE_TRANSACTION` ,`BLOCK_CARD` if exposure up to $2,500 | 
| **L2 fraud manager** | `BLOCK_CARD` above $2,500,`BLOCK_ALL_CARDS` ,`FILE_REPORT` | 

L1 and L2 actions appear as approval cards in the console, each showing the rule that required it.

Any closed case can be replayed *as if it had just arrived*, with the graph frozen at its `as_of`. That is how thresholds and likelihood ratios are calibrated, and how the UI can show a live calibration plot.

The screenshot at the top is this exact run. An analyst wrote: *"several cards this month show purchases from the same unusual device profile."*

```
sequenceDiagram
  autonumber
  participant A as Analyst
  participant AG as Agent
  participant TG as TigerGraph (MCP)
  participant PE as Policy engine
  A->>AG: Review $74.96 online purchase on card T9003-K1
  AG->>TG: Card history, as_of = alert time
  TG-->>AG: 47 earlier payments
  AG->>TG: Run fraud signatures (prosecution)
  TG-->>AG: 2 signs: device is New behind anonymous proxy, shared by 12 customers
  AG->>TG: Sweep from the shared device profile
  TG-->>AG: 5 other cards still live on it, 6 already handled
  AG->>TG: Run innocence signatures (defence)
  TG-->>AG: 1 reason it could be innocent: amount within card p95 ($97.20)
  AG->>PE: p = 92%, classes: device, network, geography
  PE-->>AG: Likely fraud, $187.33 at risk across 2 payments
  AG-->>A: Verdict, reasoning with "Why" lines, customer-friendly message
```

| Step | What the agent did | Why it matters | 
|---|---|---|
| **1. Read the alert** | Found 47 earlier payments, used only what was known at alert time | No look-ahead | 
| **2. Look for fraud** | 2 signs, including a rare *New* device behind an anonymous proxy used by 12 customers | Graph-derived evidence, not the score | 
| **3. Look for innocence** | Found 1 reason: amount is normal for this card | The defence gets a real hearing | 
| **4. Verdict** | **92%, very likely fraud** ,**$187.33 at risk across 2 payments** | Computed by the judge and policy engine, not by the LLM | 
| **5. Explain** | Customer-friendly message, next steps | Plain language, not a risk-model dump | 

Each step has an italic **"Why"** line, and a *"slow down so I can watch"* toggle lets a human follow the reasoning live.

| Typical fraud agent | Fraud Investigator | 
|---|---|
| Trusts the bank score | Treats the score as one witness on a learned, **non-monotonic** curve | 
| Hunts only for fraud signals | **Prosecution *and* defence** in parallel | 
| "Same customer" memory | **Shape-matched** memory, aware that customers are aggregates | 
| LLM decides the action | LLM **only phrases** ; a policy engine decides | 
| Probability from vibes | **Log-odds ledger** with independence classes | 
| Sees the future by accident | `as_of` enforced in the**data layer** , with a leakage test | 
| Only finds flagged cards | **Score-blind sweep** finds the ring at score 0.05 | 

| Area | Status | Next step | 
|---|---|---|
| Full time-machine backtest | Harness and replay built | Publish per-pattern accuracy, per-signature precision and the calibration curve `[insert results]` | 
| Account-takeover signature | Not built | Add to the prosecution set | 
| Numeric lookalike channel | Not built | PCA-32 vectors and kNN restricted to labelled exemplars | 
| Full PageRank sweep | One-hop version today | Run personalised PageRank in-database with the GDS library | 
| Vector search in TigerGraph | Planned | Move semantic recall in-database | 
| Monitor mode | Optional | Let the agent raise alerts on its own from graph contagion | 
| Learn from analysts | Candidate patterns proposed | Feed analyst-named patterns back into signatures and calibration | 
| Narration model comparison | Planned | Compare models on schema pass rate, citation accuracy and SAR completeness | 

A fraud agent should not be a persuasive storyteller. It should be a **disciplined investigator** that argues both sides, states its uncertainty, cites everything, and leaves the final decision to **auditable policy**. TigerGraph is what makes that possible: it holds the transactions, the memory and the time-bounded queries in one place, and the agent reaches all of it through a single MCP tool surface.

`TigerGraph` · `TigerGraph MCP server` · `GSQL` · `Ollama (gemma4:31b-cloud, nomic-embed-text)` · `Next.js + TypeScript` · `Zod` · `DuckDB` · `Python`

`https://www.youtube.com/watch?v=kXDzmKNrhIE`
`https://github.com/shreesha345/hhg-fraud-agent`
`https://hhg-goa-frontend-hc8k.vercel.app/`
`#TigerGraph @TigerGraphDB`