{"slug": "don-t-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph", "title": "Don't Trust the Score: A Fraud Investigator That Argues Both Sides on TigerGraph", "summary": "A developer built Fraud Investigator, a TigerGraph-backed fraud detection agent that treats bank risk scores as one weak witness rather than a verdict, after profiling five provided files with DuckDB showed all 900 cleared cases scored 0.81 or higher while 31% of confirmed fraud scored under 0.3. The system routes all reads through TigerGraph's MCP tool surface and all state-changing actions through a deterministic policy engine, using three schema-checked LLM calls per case for planning, explanation and SAR writing. Graph features reached an AUC of 0.91 versus 0.62 for the bank score among alerts at or above 0.8, and a score-blind sweep surfaced a hidden fraud ring of 60 transactions across 28 customers that no closed case contained.", "body_md": "**๐ŸŽฅ Demo video:** [Watch the Fraud Investigator in action](https://www.youtube.com/watch?v=kXDzmKNrhIE)\n\nMost fraud-agent demos work the same way: a model produces a risk score, then an LLM writes a convincing story about why the score is right.\n\nBefore writing any code, We profiled all five provided files with DuckDB. The data showed that approach would **fail in both directions**: it blocks innocent people and misses real fraud.\n\n| What I measured | Result | Design consequence | \n|---|---|---|\n| Cleared (false-alarm) cases | **All 900** scored**0.81 or higher** | A high score is where innocence lives, so the agent must be able to *clear a 0.9* | \n| Confirmed fraud | Mean score **0.47** , and**31% scored under 0.3** | The agent must be able to *convict a 0.05* | \n| A hidden fraud ring | 60 transactions, 28 customers, every score **โ‰ค 0.44** , in no closed case | A **score-blind** sweep is needed | \n| Graph features vs bank score, among alerts โ‰ฅ 0.8 | AUC **0.91** vs**0.62** | The **graph** decides where the score is confused | \n| \"Customers\" | 104 of them hold **48%** of all transactions | A customer is not a person, so baselines must be **per card** | \n\nHere is the same story as one picture. Fraud appears at every score. Innocence appears in only one zone:\n\n```\nbank score bin : 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9\nconfirmed fraud: โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ\ncleared (safe) : ยท ยท ยท ยท ยท ยท ยท ยท โ–ˆโ–ˆโ–ˆ โ–ˆโ–ˆโ–ˆ\n โ–ฒ\n the \"decoy zone\": the score cannot separate them here\n```\n\n*Chart shows where each outcome appears; it is not a count.*\n\n**The score tells you where to look. The graph tells you what happened.**\n\nSo I built an agent that treats the score as one weak witness, not the judge.\n\n**Fraud Investigator** takes an alert from any of three triggers (a bank risk flag, a customer saying *\"I never made this\"*, or an analyst asking *\"look for related activity\"*) and investigates it like a small legal team:\n\nFive layers. **Everything that reads goes through TigerGraph's MCP tool surface. Everything that can change the world goes through the policy engine.**\n\n```\nflowchart TB\n subgraph ENTRY[\"ENTRY\"]\n direction LR\n E1[\"Risk-score alert\"]\n E2[\"Customer report\"]\n E3[\"Analyst request\"]\n E4[\"Analyst console\"]\n end\n\n subgraph ORCH[\"ORCHESTRATOR - bounded state machine, typed evidence ledger\"]\n direction LR\n O1[\"Intake\"] --> O2[\"Recall\"] --> O3[\"Advocates x3\"] --> O4[\"Judge\"] --> O5[\"Plan\"] --> O6[\"Gate\"] --> O7[\"Act\"] --> O8[\"Learn\"]\n end\n\n subgraph ENG[\"DETERMINISTIC ENGINES - no LLM inside\"]\n direction LR\n G1[\"Signature library
prosecution + defence\"]\n G2[\"Contagion + rarity
personalised PageRank\"]\n G3[\"Calibrated judge
log-odds ledger\"]\n G4[\"Policy engine
rules R1 to R10\"]\n G5[\"Response simulator
denies / confirms / silent\"]\n end\n\n subgraph TG[\"TIGERGRAPH via MCP server\"]\n direction LR\n T1[(\"Transaction graph\")]\n T2[(\"Case memory\")]\n T3[(\"Vector indexes\")]\n T4[(\"Installed GSQL
every query takes as_of\")]\n end\n\n subgraph OFF[\"OFFLINE - build time\"]\n direction LR\n F1[\"Loader
708 MB CSV to typed graph\"]\n F2[\"Time-machine backtest\"]\n F3[\"Calibration tables
JSON, no ML at runtime\"]\n end\n\n LLM[\"LLM - 3 schema-checked calls per case
plan evidence, explain, write SAR\"]\n\n ENTRY --> ORCH\n ORCH --> ENG\n ENG --> TG\n OFF -. \"loads + calibrates\".-> TG\n F3 -. \"coefficients\".-> G3\n ORCH -. \"phrasing only\".-> LLM\n\n classDef entry fill:#F3F4F6,stroke:#6B7280,color:#111827;\n classDef agent fill:#EEF0FF,stroke:#5B5BD6,color:#1B1B4B;\n classDef tgc fill:#E6FAF6,stroke:#0F9D8A,color:#063D36;\n classDef control fill:#FFF4DC,stroke:#D98E04,color:#4A3000;\n class E1,E2,E3,E4 entry;\n class O1,O2,O3,O4,O5,O6,O7,O8,LLM agent;\n class T1,T2,T3,T4,F1,F2,F3,G1,G2 tgc;\n class G3,G4,G5 control;\n```\n\n*indigo = agent and judge, teal = graph and graph-derived work, amber = control and side effects.*\n\n| # | Principle | Why it matters | \n|---|---|---|\n| 1 | **The LLM only phrases** | It cannot add an action, change a route, set exposure, or invent an ID. That is what makes a mid-sized open model safe here. | \n| 2 | **Read through MCP, act through policy** | One loggable tool contract, and no side effect without a rule citation. | \n| 3 | **Time is enforced in the data layer** | The alert's `opened_at` is 1 to 6 hours*after* the flagged transaction, and the file runs to Dec 31. Every query takes`as_of` , and a leakage test fails if any row is later than the trigger. | \n| 4 | **Graceful degradation** | Every LLM reply is validated, retried twice with the validator's error fed back, then replaced by a deterministic template. A valid case file is always written. | \n| 5 | **The graph is the source of truth** | The answer JSON is rendered from the graph and checked: IDs exist, exposure equals the sum of affected transactions, `sar.file` matches`FILE_REPORT` . | \n\nTigerGraph is the substrate for the reasoning, not a place to park rows. The schema is shaped around the *questions the agent asks*, so each question becomes a one-hop lookup.\n\n``` php\nflowchart LR\n subgraph TXN[\"Transaction graph\"]\n Customer -->|OWNS| Card\n Card -->|MADE| Transaction\n Transaction -->|NEXT| NextTxn[\"Next txn on card\"]\n Transaction -->|FROM_DEVICE| DeviceProfile[\"DeviceProfile
carries rarity weight\"]\n Transaction -->|BILLED_IN| BillingRegion\n Transaction -->|PURCHASER_EMAIL| EmailDomain\n Transaction -->|IN_EPISODE| Episode[\"Episode
derived burst window\"]\n Card -->|MEMBER_OF| Ring[\"Ring
derived component\"]\n end\n\n subgraph MEM[\"Case memory\"]\n ClosedCase -->|MATCHES| Pattern\n Case -->|MATCHES| Pattern\n Case -->|RETRIEVED| ClosedCase\n Case -->|HAS_EVIDENCE| Evidence\n Case -->|RECOMMENDS| Action\n Action -->|UNDER_RULE| PolicyChunk\n PolicyChunk -->|DESCRIBES| Pattern\n end\n\n ClosedCase -->|INVOLVES| Transaction\n Case -->|INVOLVES| Transaction\n Case -->|CONNECTED_TO| Card\n Case -->|SUSPECTS| Ring\n Evidence -->|CITES| Transaction\n\n classDef derived fill:#FFF4DC,stroke:#D98E04,color:#4A3000;\n class Episode,Ring,DeviceProfile derived;\n```\n\n*Amber vertices are the ones I added beyond the suggested schema. They are built offline so the agent's questions stay local to a card's neighbourhood.*\n\n**Rarity-weighted devices.** Each `DeviceProfile` stores how many customers use it, its share of *New* marks, its share behind an anonymous proxy, and an IDF weight. A generic Windows/Chrome profile shared by 1,000 customers gets a near-zero weight. The ring fingerprint gets a high one.\n\nMy first ring rule flagged **212 of 9,706** device profiles, mostly common iPhones. Requiring **5 to 200 users, *New* on 90% of uses, and an anonymous proxy on 80%** left the real ring and very few others.\n\n**A score-blind contagion sweep.** A personalised PageRank starts from confirmed-fraud entities and runs over the card, device, region and email graph. It ranks *unflagged* cards by proximity to known fraud and ignores the bank score. One query on the ring fingerprint recovers victims the model never flagged.\n\n**Case memory as GraphRAG.** Closed cases attach to the same transactions, cards and devices that live traffic touches, and recall works three ways:\n\n| Recall mode | What it does | \n|---|---|\n| **Structural** | Closed cases on the same device or card neighbourhood, ranked by overlap, rarity and recency | \n| **Semantic** | Vector search over case notes, policy and regulatory text | \n| **Numeric** | Nearest neighbours in a 32-dimension PCA of the Vesta features | \n\nThe three results are compiled into one brief for the LLM: *what we know, what conflicts, what is missing*.\n\nMemory uses **shape matches only**, never \"same customer\". Closed history is 84% fraud, so anyone with history would look guilty.\n\n**Causal write-back.** Every new case becomes a `Case` vertex linked to its transactions, cards, ring and pattern, with retrieved cases as `RETRIEVED` edges. Cases are processed in `opened_at` order, so case N may recall earlier cases and **never a later one**.\n\n``` php\nflowchart TD\n T[\"Trigger\"] --> I[\"1. Intake
pin as_of, resolve card\"]\n I --> M[\"2. Recall
prior cases, 3 ways\"]\n M --> PR[\"Prosecution\"]\n M --> DE[\"Defence\"]\n M --> SW[\"Sweep\"]\n PR --> J[\"Judge
p, confidence, evidence classes\"]\n DE --> J\n SW --> J\n J --> S{\"Stop rule met?
p >= 0.85 or p <= 0.15
with 2 independent classes\"}\n S -- \"yes\"--> A[\"Policy engine
choose actions + routes\"]\n S -- \"no, budget left\"--> V[\"Plan
best value-of-information request\"]\n V --> R[\"Reply
customer / step-up / analyst\"]\n R --> J\n S -- \"no, nothing left to try\"--> E[\"Escalate to analyst
with gap list\"]\n E --> A\n A --> X[\"Explain, SAR, write case to graph\"]\n\n classDef agent fill:#EEF0FF,stroke:#5B5BD6,color:#1B1B4B;\n classDef tgc fill:#E6FAF6,stroke:#0F9D8A,color:#063D36;\n classDef control fill:#FFF4DC,stroke:#D98E04,color:#4A3000;\n class PR,DE,J,V,R agent;\n class SW,M tgc;\n class A,E,X,S control;\n```\n\n*Budgets: 3 evidence rounds, about 20 graph calls, 1 request of each type.*\n\nProsecution and Defence run **in parallel over the same as-of graph**. An unanswered defence test is reported as uncertainty, never ignored. The bank score is not a signature. It enters as one evidence class through a learned, **non-monotonic** curve.\n\n**The meaning of a signal flips by zone.** At score 0.8+, **98%** of cleared online alerts show a *New* device, against **36%** of fraud. So \"New device\" is **defence** evidence in that zone and **prosecution** evidence below it.\n\n| Prosecution signatures | Defence signatures | \n|---|---|\n| Test then spend | Trip continuity (a clone would keep spending at home) | \n| Off-profile burst | Device succession (new phone replaces old, same brand and OS) | \n| New-device attribution | Recurring cadence, with a *coincidence test* | \n| Out-of-region while home continues | Baseline-consistent spend | \n| Threshold hugging (structuring) | Hub hygiene (down-weight aggregate customers) | \n| Shared rare fingerprint | Novelty detector (flags patterns nothing explains) | \n\nFraud probability is graded for calibration and drives every policy threshold, so it is **computed, not guessed**:\n\n`prior (reset for 84% fraud history)` โ†’ `+ each evidence item's log-likelihood ratio` โ†’ `same class counts once (max, not sum)` โ†’ `global damping tuned on backtest` โ†’ `fraud_probability`\n\nEvidence is tagged by class (bank score, card behaviour, device, geography, network, memory, lookalike, customer reply). Correlated evidence inside a class does not stack, so the policy's *\"at least two independent pieces\"* rule is **computed, not asserted**.\n\nIf the stop rule isn't met, the agent scores candidate requests (customer validation, step-up auth, analyst info, one more graph hop) by *expected shift in p, minus friction*. It must state **which two hypotheses the request is meant to separate**, and that sentence becomes the recorded \"why more evidence was requested\".\n\nReplies aren't provided, so before asking, the agent **precomputes the plan for all three outcomes**. The simulated reply is drawn from its own pre-reply posterior, and the assumption is recorded.\n\n``` php\nflowchart TD\n ASK[\"Ask the customer\"] --> D[\"Denies\"]\n ASK --> C[\"Confirms\"]\n ASK --> Z[\"Silent for 24h\"]\n D --> DR[\"Rule R2
BLOCK_CARD, CREATE_CASE,
FILE_REPORT if threshold met,
MONITOR_CONNECTED_CARDS\"]\n C --> CR[\"Rule R3
CLOSE_NO_FRAUD\"]\n Z --> ZR[\"Rule R4
MONITOR_CARD, DECLINE_TRANSACTION,
escalate if exposure over $500\"]\n classDef bad fill:#FDECEC,stroke:#D1242F,color:#4A0A0F;\n classDef good fill:#E7F8EC,stroke:#2EA043,color:#0B3D1A;\n classDef mid fill:#FFF4DC,stroke:#D98E04,color:#4A3000;\n class D,DR bad;\n class C,CR good;\n class Z,ZR mid;\n```\n\nAll three branches are stored as a `contingency` object, so the before-and-after recommendation is a **lookup, not an improvisation**, and an analyst can override the simulated reply live.\n\nRules R1 to R10 are a decision table in code. Exposure is computed from transaction IDs, never by the LLM. The executor only runs `auto` actions:\n\n| Route | Actions | \n|---|---|\n| **Auto** (executed) | `CREATE_CASE` ,`MONITOR_CARD` ,`VERIFY_WITH_CUSTOMER` ,`WARN_CUSTOMER` ,`ESCALATE_TO_ANALYST` ,`CLOSE_NO_FRAUD` | \n| **L1 team lead** | `DECLINE_TRANSACTION` ,`BLOCK_CARD` if exposure up to $2,500 | \n| **L2 fraud manager** | `BLOCK_CARD` above $2,500,`BLOCK_ALL_CARDS` ,`FILE_REPORT` | \n\nL1 and L2 actions appear as approval cards in the console, each showing the rule that required it.\n\nAny closed case can be replayed *as if it had just arrived*, with the graph frozen at its `as_of`. That is how thresholds and likelihood ratios are calibrated, and how the UI can show a live calibration plot.\n\nThe screenshot at the top is this exact run. An analyst wrote: *\"several cards this month show purchases from the same unusual device profile.\"*\n\n```\nsequenceDiagram\n autonumber\n participant A as Analyst\n participant AG as Agent\n participant TG as TigerGraph (MCP)\n participant PE as Policy engine\n A->>AG: Review $74.96 online purchase on card T9003-K1\n AG->>TG: Card history, as_of = alert time\n TG-->>AG: 47 earlier payments\n AG->>TG: Run fraud signatures (prosecution)\n TG-->>AG: 2 signs: device is New behind anonymous proxy, shared by 12 customers\n AG->>TG: Sweep from the shared device profile\n TG-->>AG: 5 other cards still live on it, 6 already handled\n AG->>TG: Run innocence signatures (defence)\n TG-->>AG: 1 reason it could be innocent: amount within card p95 ($97.20)\n AG->>PE: p = 92%, classes: device, network, geography\n PE-->>AG: Likely fraud, $187.33 at risk across 2 payments\n AG-->>A: Verdict, reasoning with \"Why\" lines, customer-friendly message\n```\n\n| Step | What the agent did | Why it matters | \n|---|---|---|\n| **1. Read the alert** | Found 47 earlier payments, used only what was known at alert time | No look-ahead | \n| **2. Look for fraud** | 2 signs, including a rare *New* device behind an anonymous proxy used by 12 customers | Graph-derived evidence, not the score | \n| **3. Look for innocence** | Found 1 reason: amount is normal for this card | The defence gets a real hearing | \n| **4. Verdict** | **92%, very likely fraud** ,**$187.33 at risk across 2 payments** | Computed by the judge and policy engine, not by the LLM | \n| **5. Explain** | Customer-friendly message, next steps | Plain language, not a risk-model dump | \n\nEach step has an italic **\"Why\"** line, and a *\"slow down so I can watch\"* toggle lets a human follow the reasoning live.\n\n| Typical fraud agent | Fraud Investigator | \n|---|---|\n| Trusts the bank score | Treats the score as one witness on a learned, **non-monotonic** curve | \n| Hunts only for fraud signals | **Prosecution *and* defence** in parallel | \n| \"Same customer\" memory | **Shape-matched** memory, aware that customers are aggregates | \n| LLM decides the action | LLM **only phrases** ; a policy engine decides | \n| Probability from vibes | **Log-odds ledger** with independence classes | \n| Sees the future by accident | `as_of` enforced in the**data layer** , with a leakage test | \n| Only finds flagged cards | **Score-blind sweep** finds the ring at score 0.05 | \n\n| Area | Status | Next step | \n|---|---|---|\n| Full time-machine backtest | Harness and replay built | Publish per-pattern accuracy, per-signature precision and the calibration curve `[insert results]` | \n| Account-takeover signature | Not built | Add to the prosecution set | \n| Numeric lookalike channel | Not built | PCA-32 vectors and kNN restricted to labelled exemplars | \n| Full PageRank sweep | One-hop version today | Run personalised PageRank in-database with the GDS library | \n| Vector search in TigerGraph | Planned | Move semantic recall in-database | \n| Monitor mode | Optional | Let the agent raise alerts on its own from graph contagion | \n| Learn from analysts | Candidate patterns proposed | Feed analyst-named patterns back into signatures and calibration | \n| Narration model comparison | Planned | Compare models on schema pass rate, citation accuracy and SAR completeness | \n\nA fraud agent should not be a persuasive storyteller. It should be a **disciplined investigator** that argues both sides, states its uncertainty, cites everything, and leaves the final decision to **auditable policy**. TigerGraph is what makes that possible: it holds the transactions, the memory and the time-bounded queries in one place, and the agent reaches all of it through a single MCP tool surface.\n\n`TigerGraph` ยท `TigerGraph MCP server` ยท `GSQL` ยท `Ollama (gemma4:31b-cloud, nomic-embed-text)` ยท `Next.js + TypeScript` ยท `Zod` ยท `DuckDB` ยท `Python`\n\n`https://www.youtube.com/watch?v=kXDzmKNrhIE`\n`https://github.com/shreesha345/hhg-fraud-agent`\n`https://hhg-goa-frontend-hc8k.vercel.app/`\n`#TigerGraph @TigerGraphDB`", "url": "https://wpnews.pro/news/don-t-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph", "canonical_source": "https://dev.to/pranav_ssalian_be1687de7/dont-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph-299n", "published_at": "2026-09-24 17:06:44+00:00", "updated_at": "2026-09-24 17:29:59.708082+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "mlops", "agent-protocols", "artificial-intelligence"], "entities": ["TigerGraph", "DuckDB", "Fraud Investigator", "MCP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/don-t-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph", "markdown": "https://wpnews.pro/news/don-t-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph.md", "text": "https://wpnews.pro/news/don-t-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph.txt", "jsonld": "https://wpnews.pro/news/don-t-trust-the-score-a-fraud-investigator-that-argues-both-sides-on-tigergraph.jsonld"}}