{"slug": "sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer", "title": "Sentinel-IR: Stop Making Your Agents Read Human Code. Give Them a Fact Layer", "summary": "A developer built Sentinel-IR, a deterministic machine-oriented intermediate representation that extracts security-relevant facts from source code — HTTP routes, environment variable reads, file writes, spawned processes, and exports — so coding agents can answer questions without reading entire files. Benchmarked against gpt-6-astra across 12 files and 87 questions (267 LLM calls), IR with raw-source fallback cut input tokens by 71.3% while scoring 87/87 (100%) versus 84/87 (96.6%) for raw source alone, with break-even at roughly 303 source tokens (~34 lines).", "body_md": "*Live-benchmarked against `gpt-6-astra`. Raw scorecard and log included.*\n\nEvery coding agent today does the same wasteful thing: to answer a simple question like *\"does this merge request touch the network?\"*, it reads the entire source file — hundreds of lines of human-oriented code — and burns thousands of tokens on it.\n\nSource code is written for humans. Comments, formatting, naming style — all of it is noise to an agent that only needs to know *what the code does*. So we built **Sentinel-IR**: a compact, machine-oriented intermediate representation that keeps the meaning and drops the noise.\n\nSentinel-IR is **not** a new programming language. Nobody writes in it. It's a *fact layer*: a deterministic extraction of the security-relevant things in a file — the HTTP routes it exposes, the environment variables it reads, the files it writes, the processes it spawns, the surface it exports.\n\nThink of it as the difference between handing your agent a 500-page novel and a one-page factual brief written by a parser that never gets tired and never guesses.\n\nKey properties, in plain language:\n\n`content.includes(\"axios\")`. A call is a `call_expression`; a route is a call on `app`/` router`/` server` with a string path; an env read is a member access on `process.env`. That's how a false positive on `regexp.exec` got eliminated — name matching alone can't tell it from `child_process.exec`.\nWe ran the benchmark for real: **12 files, 87 questions, 267 actual LLM calls** against `gpt-6-astra`. Same files, same questions, same ground truth for every variant.\n\n| Variant | Input tokens | Accuracy | Unresolved | \n|---|---|---|---|\n| Raw source | 279,476 | 84/87 ( **96.6%** ) | 0 | \n| IR only | 58,549 ( **−79.1%** ) | 82/87 ( **94.3%** ) | 5 | \n| **IR + raw fallback** | 80,340 ( **−71.3%** ) | **87/87 (100%)** | 0 | \n\nThe headline: **IR with fallback saves 71.3% of input tokens and is *more* accurate than reading the raw source — 100% vs 96.6%.**\n\nTwo findings worth more than the headline:\n\n`git-probe`, `token-signer`, `report-worker`), the model answered the IR has near-constant size, so savings scale with file size. Fitted break-even: **~303 source tokens (~34 lines)**.\n\n| File | Lines | Raw tokens | IR tokens | Savings | \n|---|---|---|---|---|\n| 12-billing-platform | 1,366 | 11,635 | 1,332 | **88.6%** | \n| 10-analytics-kernel | 951 | 7,186 | 498 | **93.1%** | \n| 11-gateway-service | 952 | 7,364 | 1,192 | 83.8% | \n| 09-report-worker | 511 | 3,819 | 778 | 79.6% | \n| 08-inventory-api | 393 | 2,861 | 734 | 74.3% | \n| 07-order-service | 138 | 959 | 527 | 45.0% | \n| 04-polynomial | 32 | 223 | 134 | 39.9% | \n| 02-cache-writer | 33 | 209 | 287 | −37.3% | \n| 05-git-probe | 23 | 149 | 300 | −101.3% | \n| 01-http-api-server | 27 | 163 | 272 | −66.9% | \n| 06-token-signer | 23 | 163 | 224 | −37.4% | \n| 03-status-client | 26 | 172 | 199 | −15.7% | \n\nBelow ~34 lines, IR costs more than the source. We print that in our own output rather than hiding it — the per-file table is part of the scorecard.\n\nThe full live run: **263 requests, 395,847 input / 9,239 output tokens, $4.93 total** on the org account. Two honest notes:\n\n`chars/4` token estimate predicted ~418k input tokens; reality was 396k (\n|  | Orbit Local | Sentinel-IR | \n|---|---|---|\n| Answers correct | 29/87 (33.3%) | **87/87 (100%)** | \n| Context completeness | 41.4% | **100%** | \n| Confidently wrong | 7 | **0** | \n\nThe gap is expression-level: Orbit's graph knows file structure, but not \"this line spawns a child process\" or \"this MR adds a POST route reading an env secret\". That is exactly the layer IR fills. *(Orbit Remote is unmeasured — it needs a Premium group and a `Knowledge Graph: Read` token; we don't claim it.)*\n\n```\nJavaScript\n    ↓\ntree-sitter parser\n    ↓\nAstFacts    — routes / exports / imports / env / calls / risk\n    ↓\nSentinel-IR — compact, flat, self-describing projection\n    ↓\nLLM (your agent)   [fallback: raw source on unresolved]\n    ↓\nValidator → Simulation → Commit\n```\n\nEverything below the parser is deterministic and local: no network, no LLM, no I/O. `libs/core/ast-facts.js` walks the AST and classifies real nodes — `exec`/` fork` only count as process-spawning when the callee resolves to `child_process`/` execa`/` zx`.\n\n`compressFacts` in `libs/core/sentinel-ir.js` — deliberately boring:\n\n```\nfunction compressFacts(facts) {\n    if (!facts?.ast) {\n        return { ast: false, reason: facts?.error || \"source did not parse; IR fell back to text heuristics\" };\n    }\n\n    const compressed = { ast: true };\n    const put = (key, value) => {\n        if (Array.isArray(value) && value.length > 0) compressed[key] = value;\n    };\n\n    put(\"routes\", facts.routes);\n    put(\"exports\", facts.exports);\n    put(\"imports\", facts.imports);\n    put(\"env\", facts.env);\n    put(\"operations\", Object.entries(facts.operations || {})\n        .filter(([, enabled]) => enabled)\n        .map(([name]) => name));\n\n    const calls = {};\n    for (const [bucket, entries] of Object.entries(facts.calls || {})) {\n        if (Array.isArray(entries) && entries.length > 0) calls[bucket] = entries;\n    }\n    if (Object.keys(calls).length > 0) compressed.calls = calls;\n\n    put(\"dangerous\", facts.dangerous);\n    put(\"riskSignals\", (facts.riskSignals || []).map(s => `${s.signal}:${s.evidence}@${s.line}`));\n\n    return compressed;\n}\n```\n\nThree design choices worth stealing:\n\n`signal:evidence@line`, traceable back to the syntax that produced it.` ast: true` is a completeness contract`ast: false` with a reason, never silently wrong data.\nThe known gap this creates is the interesting part: **explicitly-empty categories are omitted**, so \"are there env vars?\" currently resolves to *unresolved → escalate* rather than *provenly no*. The fix — emitting explicit empty facts when `ast: true` — is the single change that would have turned 5 of our live misses into correct answers *without touching the fallback*. It's on the list.\n\nThe full compressed shape, field-for-field faithful to `SentinelIR.compress()`:\n\n```\n{\n  \"mission\":     { \"target\": \"libs/api/server.js\", \"objective\": \"network_stability\" },\n  \"world\":       { \"pressure\": 0.5, \"confidence\": 0.78, \"budget\": 0.015, \"risk\": \"high\" },\n  \"constraints\": [\"preserve_api\", \"avoid_breaking_changes\"],\n  \"forbidden\":   [\"eval\", \"child_process\"],\n  \"summary\":     { \"size\": 14203, \"lines\": 389, \"hasCrypto\": false,\n                   \"hasFilesystem\": true, \"hasNetwork\": true },\n  \"facts\": {\n    \"ast\": true,\n    \"routes\":  [\"GET /health\", \"POST /orders\"],\n    \"env\":     [\"DATABASE_URL\", \"STRIPE_SECRET_KEY\"],\n    \"operations\": [\"inboundHttp\", \"diskWrite\"],\n    \"calls\":   { \"process\": [\"spawn(node:child_process)@214\"] },\n    \"riskSignals\": [\"process_spawn:spawn@214\"]\n  }\n}\n```\n\n*(Illustrative values; schema is exact.)*\n\n`libs/ir-benchmark/runner.js` defines **\"savings at retained accuracy\"**: the best variant that is *at least as accurate* as reading raw source. If none is, the honest answer is 0% — not a smaller lie.\n\n``` js\nconst candidates = [\n    { variant: \"ir\", stats: ir },\n    { variant: \"ir+raw\", stats: hybrid }\n].filter(c => c.stats.correct >= raw.correct);\n\nconst best = candidates.sort((a, b) => a.stats.inputTokens - b.stats.inputTokens)[0] || null;\n// → savingsAtRetainedAccuracyPct: best ? savings(best.stats) : 0\n```\n\nThis run: `ir+raw` was the only variant at ≥ raw accuracy, so the claimed figure is **71.3%** — not the prettier 79.1% that lost accuracy.\n\nA diff shows *what changed in text*. IR answers *what the change does*: routes added/removed, env values newly read, fs/network/process operations appeared, exported surface changed, risk taxonomy movement. We dogfood it as a per-MR CI report across our own 44 merged MRs: 35 touched JS, 31 produced facts, median 13 facts/MR, 18 raised a file's risk level — and the job **gates**: an MR pushing a file to `critical` fails until acknowledged in `.sentinel-gate.json`.\n\n```\nnpm run ir-benchmark        # offline: info content, upper bound\nnode scripts/ir-benchmark.js --live --model gpt-6-astra   # what we ran: 267 calls, ~$4.9\nnpm run orbit-ab            # IR vs GitLab Orbit Local\nnpm run ir-pipeline-ab      # 0 invariant drift across input modes\nnpm run mr-report           # per-MR fact report over your own history\n```\n\nAll of it is `libs/core/sentinel-ir.js` + `libs/core/ast-facts.js` + `libs/ir-benchmark/`. Tree-sitter is the only runtime dependency. No source leaves your runner.\n\nir-benchmark-live.json\n\n212.26 kb\n\nDownload here:\n\n[https://app.devin.ai/attachments/b44c2c3c-9619-4914-a509-02a4e6c59a27/ir-benchmark-live.json](https://app.devin.ai/attachments/b44c2c3c-9619-4914-a509-02a4e6c59a27/ir-benchmark-live.json)", "url": "https://wpnews.pro/news/sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer", "canonical_source": "https://dev.to/jackymencz/sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer-1a6i", "published_at": "2026-09-25 05:01:27+00:00", "updated_at": "2026-09-25 05:59:04.215412+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "large-language-models"], "entities": ["Sentinel-IR", "gpt-6-astra", "Orbit Local", "tree-sitter"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer", "markdown": "https://wpnews.pro/news/sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer.md", "text": "https://wpnews.pro/news/sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer.txt", "jsonld": "https://wpnews.pro/news/sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer.jsonld"}}