Sentinel-IR: Stop Making Your Agents Read Human Code. Give Them a Fact Layer

A developer built Sentinel-IR, a deterministic machine-oriented intermediate representation that extracts security-relevant facts from source code — HTTP routes, environment variable reads, file writes, spawned processes, and exports — so coding agents can answer questions without reading entire files. Benchmarked against gpt-6-astra across 12 files and 87 questions (267 LLM calls), IR with raw-source fallback cut input tokens by 71.3% while scoring 87/87 (100%) versus 84/87 (96.6%) for raw source alone, with break-even at roughly 303 source tokens (~34 lines).

Live-benchmarked against gpt-6-astra . Raw scorecard and log included. Every coding agent today does the same wasteful thing: to answer a simple question like "does this merge request touch the network?" , it reads the entire source file — hundreds of lines of human-oriented code — and burns thousands of tokens on it. Source code is written for humans. Comments, formatting, naming style — all of it is noise to an agent that only needs to know what the code does . So we built Sentinel-IR : a compact, machine-oriented intermediate representation that keeps the meaning and drops the noise. Sentinel-IR is not a new programming language. Nobody writes in it. It's a fact layer : a deterministic extraction of the security-relevant things in a file — the HTTP routes it exposes, the environment variables it reads, the files it writes, the processes it spawns, the surface it exports. Think of it as the difference between handing your agent a 500-page novel and a one-page factual brief written by a parser that never gets tired and never guesses. Key properties, in plain language: content.includes "axios" . A call is a call expression ; a route is a call on app / router / server with a string path; an env read is a member access on process.env . That's how a false positive on regexp.exec got eliminated — name matching alone can't tell it from child process.exec . We ran the benchmark for real: 12 files, 87 questions, 267 actual LLM calls against gpt-6-astra . Same files, same questions, same ground truth for every variant. | Variant | Input tokens | Accuracy | Unresolved | |---|---|---|---| | Raw source | 279,476 | 84/87 96.6% | 0 | | IR only | 58,549 −79.1% | 82/87 94.3% | 5 | | IR + raw fallback | 80,340 −71.3% | 87/87 100% | 0 | The headline: IR with fallback saves 71.3% of input tokens and is more accurate than reading the raw source — 100% vs 96.6%. Two findings worth more than the headline: git-probe , token-signer , report-worker , the model answered the IR has near-constant size, so savings scale with file size. Fitted break-even: ~303 source tokens ~34 lines . | File | Lines | Raw tokens | IR tokens | Savings | |---|---|---|---|---| | 12-billing-platform | 1,366 | 11,635 | 1,332 | 88.6% | | 10-analytics-kernel | 951 | 7,186 | 498 | 93.1% | | 11-gateway-service | 952 | 7,364 | 1,192 | 83.8% | | 09-report-worker | 511 | 3,819 | 778 | 79.6% | | 08-inventory-api | 393 | 2,861 | 734 | 74.3% | | 07-order-service | 138 | 959 | 527 | 45.0% | | 04-polynomial | 32 | 223 | 134 | 39.9% | | 02-cache-writer | 33 | 209 | 287 | −37.3% | | 05-git-probe | 23 | 149 | 300 | −101.3% | | 01-http-api-server | 27 | 163 | 272 | −66.9% | | 06-token-signer | 23 | 163 | 224 | −37.4% | | 03-status-client | 26 | 172 | 199 | −15.7% | Below ~34 lines, IR costs more than the source. We print that in our own output rather than hiding it — the per-file table is part of the scorecard. The full live run: 263 requests, 395,847 input / 9,239 output tokens, $4.93 total on the org account. Two honest notes: chars/4 token estimate predicted ~418k input tokens; reality was 396k | | Orbit Local | Sentinel-IR | |---|---|---| | Answers correct | 29/87 33.3% | 87/87 100% | | Context completeness | 41.4% | 100% | | Confidently wrong | 7 | 0 | The gap is expression-level: Orbit's graph knows file structure, but not "this line spawns a child process" or "this MR adds a POST route reading an env secret". That is exactly the layer IR fills. Orbit Remote is unmeasured — it needs a Premium group and a Knowledge Graph: Read token; we don't claim it. JavaScript ↓ tree-sitter parser ↓ AstFacts — routes / exports / imports / env / calls / risk ↓ Sentinel-IR — compact, flat, self-describing projection ↓ LLM your agent fallback: raw source on unresolved ↓ Validator → Simulation → Commit Everything below the parser is deterministic and local: no network, no LLM, no I/O. libs/core/ast-facts.js walks the AST and classifies real nodes — exec / fork only count as process-spawning when the callee resolves to child process / execa / zx . compressFacts in libs/core/sentinel-ir.js — deliberately boring: function compressFacts facts { if facts?.ast { return { ast: false, reason: facts?.error || "source did not parse; IR fell back to text heuristics" }; } const compressed = { ast: true }; const put = key, value = { if Array.isArray value && value.length 0 compressed key = value; }; put "routes", facts.routes ; put "exports", facts.exports ; put "imports", facts.imports ; put "env", facts.env ; put "operations", Object.entries facts.operations || {} .filter , enabled = enabled .map name = name ; const calls = {}; for const bucket, entries of Object.entries facts.calls || {} { if Array.isArray entries && entries.length 0 calls bucket = entries; } if Object.keys calls .length 0 compressed.calls = calls; put "dangerous", facts.dangerous ; put "riskSignals", facts.riskSignals || .map s = ${s.signal}:${s.evidence}@${s.line} ; return compressed; } Three design choices worth stealing: signal:evidence@line , traceable back to the syntax that produced it. ast: true is a completeness contract ast: false with a reason, never silently wrong data. The known gap this creates is the interesting part: explicitly-empty categories are omitted , so "are there env vars?" currently resolves to unresolved → escalate rather than provenly no . The fix — emitting explicit empty facts when ast: true — is the single change that would have turned 5 of our live misses into correct answers without touching the fallback . It's on the list. The full compressed shape, field-for-field faithful to SentinelIR.compress : { "mission": { "target": "libs/api/server.js", "objective": "network stability" }, "world": { "pressure": 0.5, "confidence": 0.78, "budget": 0.015, "risk": "high" }, "constraints": "preserve api", "avoid breaking changes" , "forbidden": "eval", "child process" , "summary": { "size": 14203, "lines": 389, "hasCrypto": false, "hasFilesystem": true, "hasNetwork": true }, "facts": { "ast": true, "routes": "GET /health", "POST /orders" , "env": "DATABASE URL", "STRIPE SECRET KEY" , "operations": "inboundHttp", "diskWrite" , "calls": { "process": "spawn node:child process @214" }, "riskSignals": "process spawn:spawn@214" } } Illustrative values; schema is exact. libs/ir-benchmark/runner.js defines "savings at retained accuracy" : the best variant that is at least as accurate as reading raw source. If none is, the honest answer is 0% — not a smaller lie. js const candidates = { variant: "ir", stats: ir }, { variant: "ir+raw", stats: hybrid } .filter c = c.stats.correct = raw.correct ; const best = candidates.sort a, b = a.stats.inputTokens - b.stats.inputTokens 0 || null; // → savingsAtRetainedAccuracyPct: best ? savings best.stats : 0 This run: ir+raw was the only variant at ≥ raw accuracy, so the claimed figure is 71.3% — not the prettier 79.1% that lost accuracy. A diff shows what changed in text . IR answers what the change does : routes added/removed, env values newly read, fs/network/process operations appeared, exported surface changed, risk taxonomy movement. We dogfood it as a per-MR CI report across our own 44 merged MRs: 35 touched JS, 31 produced facts, median 13 facts/MR, 18 raised a file's risk level — and the job gates : an MR pushing a file to critical fails until acknowledged in .sentinel-gate.json . npm run ir-benchmark offline: info content, upper bound node scripts/ir-benchmark.js --live --model gpt-6-astra what we ran: 267 calls, ~$4.9 npm run orbit-ab IR vs GitLab Orbit Local npm run ir-pipeline-ab 0 invariant drift across input modes npm run mr-report per-MR fact report over your own history All of it is libs/core/sentinel-ir.js + libs/core/ast-facts.js + libs/ir-benchmark/ . Tree-sitter is the only runtime dependency. No source leaves your runner. ir-benchmark-live.json 212.26 kb Download here: https://app.devin.ai/attachments/b44c2c3c-9619-4914-a509-02a4e6c59a27/ir-benchmark-live.json https://app.devin.ai/attachments/b44c2c3c-9619-4914-a509-02a4e6c59a27/ir-benchmark-live.json