# Sentinel-IR: Stop Making Your Agents Read Human Code. Give Them a Fact Layer

> Source: <https://dev.to/jackymencz/sentinel-ir-stop-making-your-agents-read-human-code-give-them-a-fact-layer-1a6i>
> Published: 2026-09-25 05:01:27+00:00

*Live-benchmarked against `gpt-6-astra`. Raw scorecard and log included.*

Every coding agent today does the same wasteful thing: to answer a simple question like *"does this merge request touch the network?"*, it reads the entire source file — hundreds of lines of human-oriented code — and burns thousands of tokens on it.

Source code is written for humans. Comments, formatting, naming style — all of it is noise to an agent that only needs to know *what the code does*. So we built **Sentinel-IR**: a compact, machine-oriented intermediate representation that keeps the meaning and drops the noise.

Sentinel-IR is **not** a new programming language. Nobody writes in it. It's a *fact layer*: a deterministic extraction of the security-relevant things in a file — the HTTP routes it exposes, the environment variables it reads, the files it writes, the processes it spawns, the surface it exports.

Think of it as the difference between handing your agent a 500-page novel and a one-page factual brief written by a parser that never gets tired and never guesses.

Key properties, in plain language:

`content.includes("axios")`. A call is a `call_expression`; a route is a call on `app`/` router`/` server` with a string path; an env read is a member access on `process.env`. That's how a false positive on `regexp.exec` got eliminated — name matching alone can't tell it from `child_process.exec`.
We ran the benchmark for real: **12 files, 87 questions, 267 actual LLM calls** against `gpt-6-astra`. Same files, same questions, same ground truth for every variant.

| Variant | Input tokens | Accuracy | Unresolved | 
|---|---|---|---|
| Raw source | 279,476 | 84/87 ( **96.6%** ) | 0 | 
| IR only | 58,549 ( **−79.1%** ) | 82/87 ( **94.3%** ) | 5 | 
| **IR + raw fallback** | 80,340 ( **−71.3%** ) | **87/87 (100%)** | 0 | 

The headline: **IR with fallback saves 71.3% of input tokens and is *more* accurate than reading the raw source — 100% vs 96.6%.**

Two findings worth more than the headline:

`git-probe`, `token-signer`, `report-worker`), the model answered the IR has near-constant size, so savings scale with file size. Fitted break-even: **~303 source tokens (~34 lines)**.

| File | Lines | Raw tokens | IR tokens | Savings | 
|---|---|---|---|---|
| 12-billing-platform | 1,366 | 11,635 | 1,332 | **88.6%** | 
| 10-analytics-kernel | 951 | 7,186 | 498 | **93.1%** | 
| 11-gateway-service | 952 | 7,364 | 1,192 | 83.8% | 
| 09-report-worker | 511 | 3,819 | 778 | 79.6% | 
| 08-inventory-api | 393 | 2,861 | 734 | 74.3% | 
| 07-order-service | 138 | 959 | 527 | 45.0% | 
| 04-polynomial | 32 | 223 | 134 | 39.9% | 
| 02-cache-writer | 33 | 209 | 287 | −37.3% | 
| 05-git-probe | 23 | 149 | 300 | −101.3% | 
| 01-http-api-server | 27 | 163 | 272 | −66.9% | 
| 06-token-signer | 23 | 163 | 224 | −37.4% | 
| 03-status-client | 26 | 172 | 199 | −15.7% | 

Below ~34 lines, IR costs more than the source. We print that in our own output rather than hiding it — the per-file table is part of the scorecard.

The full live run: **263 requests, 395,847 input / 9,239 output tokens, $4.93 total** on the org account. Two honest notes:

`chars/4` token estimate predicted ~418k input tokens; reality was 396k (
|  | Orbit Local | Sentinel-IR | 
|---|---|---|
| Answers correct | 29/87 (33.3%) | **87/87 (100%)** | 
| Context completeness | 41.4% | **100%** | 
| Confidently wrong | 7 | **0** | 

The gap is expression-level: Orbit's graph knows file structure, but not "this line spawns a child process" or "this MR adds a POST route reading an env secret". That is exactly the layer IR fills. *(Orbit Remote is unmeasured — it needs a Premium group and a `Knowledge Graph: Read` token; we don't claim it.)*

```
JavaScript
    ↓
tree-sitter parser
    ↓
AstFacts    — routes / exports / imports / env / calls / risk
    ↓
Sentinel-IR — compact, flat, self-describing projection
    ↓
LLM (your agent)   [fallback: raw source on unresolved]
    ↓
Validator → Simulation → Commit
```

Everything below the parser is deterministic and local: no network, no LLM, no I/O. `libs/core/ast-facts.js` walks the AST and classifies real nodes — `exec`/` fork` only count as process-spawning when the callee resolves to `child_process`/` execa`/` zx`.

`compressFacts` in `libs/core/sentinel-ir.js` — deliberately boring:

```
function compressFacts(facts) {
    if (!facts?.ast) {
        return { ast: false, reason: facts?.error || "source did not parse; IR fell back to text heuristics" };
    }

    const compressed = { ast: true };
    const put = (key, value) => {
        if (Array.isArray(value) && value.length > 0) compressed[key] = value;
    };

    put("routes", facts.routes);
    put("exports", facts.exports);
    put("imports", facts.imports);
    put("env", facts.env);
    put("operations", Object.entries(facts.operations || {})
        .filter(([, enabled]) => enabled)
        .map(([name]) => name));

    const calls = {};
    for (const [bucket, entries] of Object.entries(facts.calls || {})) {
        if (Array.isArray(entries) && entries.length > 0) calls[bucket] = entries;
    }
    if (Object.keys(calls).length > 0) compressed.calls = calls;

    put("dangerous", facts.dangerous);
    put("riskSignals", (facts.riskSignals || []).map(s => `${s.signal}:${s.evidence}@${s.line}`));

    return compressed;
}
```

Three design choices worth stealing:

`signal:evidence@line`, traceable back to the syntax that produced it.` ast: true` is a completeness contract`ast: false` with a reason, never silently wrong data.
The known gap this creates is the interesting part: **explicitly-empty categories are omitted**, so "are there env vars?" currently resolves to *unresolved → escalate* rather than *provenly no*. The fix — emitting explicit empty facts when `ast: true` — is the single change that would have turned 5 of our live misses into correct answers *without touching the fallback*. It's on the list.

The full compressed shape, field-for-field faithful to `SentinelIR.compress()`:

```
{
  "mission":     { "target": "libs/api/server.js", "objective": "network_stability" },
  "world":       { "pressure": 0.5, "confidence": 0.78, "budget": 0.015, "risk": "high" },
  "constraints": ["preserve_api", "avoid_breaking_changes"],
  "forbidden":   ["eval", "child_process"],
  "summary":     { "size": 14203, "lines": 389, "hasCrypto": false,
                   "hasFilesystem": true, "hasNetwork": true },
  "facts": {
    "ast": true,
    "routes":  ["GET /health", "POST /orders"],
    "env":     ["DATABASE_URL", "STRIPE_SECRET_KEY"],
    "operations": ["inboundHttp", "diskWrite"],
    "calls":   { "process": ["spawn(node:child_process)@214"] },
    "riskSignals": ["process_spawn:spawn@214"]
  }
}
```

*(Illustrative values; schema is exact.)*

`libs/ir-benchmark/runner.js` defines **"savings at retained accuracy"**: the best variant that is *at least as accurate* as reading raw source. If none is, the honest answer is 0% — not a smaller lie.

``` js
const candidates = [
    { variant: "ir", stats: ir },
    { variant: "ir+raw", stats: hybrid }
].filter(c => c.stats.correct >= raw.correct);

const best = candidates.sort((a, b) => a.stats.inputTokens - b.stats.inputTokens)[0] || null;
// → savingsAtRetainedAccuracyPct: best ? savings(best.stats) : 0
```

This run: `ir+raw` was the only variant at ≥ raw accuracy, so the claimed figure is **71.3%** — not the prettier 79.1% that lost accuracy.

A diff shows *what changed in text*. IR answers *what the change does*: routes added/removed, env values newly read, fs/network/process operations appeared, exported surface changed, risk taxonomy movement. We dogfood it as a per-MR CI report across our own 44 merged MRs: 35 touched JS, 31 produced facts, median 13 facts/MR, 18 raised a file's risk level — and the job **gates**: an MR pushing a file to `critical` fails until acknowledged in `.sentinel-gate.json`.

```
npm run ir-benchmark        # offline: info content, upper bound
node scripts/ir-benchmark.js --live --model gpt-6-astra   # what we ran: 267 calls, ~$4.9
npm run orbit-ab            # IR vs GitLab Orbit Local
npm run ir-pipeline-ab      # 0 invariant drift across input modes
npm run mr-report           # per-MR fact report over your own history
```

All of it is `libs/core/sentinel-ir.js` + `libs/core/ast-facts.js` + `libs/ir-benchmark/`. Tree-sitter is the only runtime dependency. No source leaves your runner.

ir-benchmark-live.json

212.26 kb

Download here:

[https://app.devin.ai/attachments/b44c2c3c-9619-4914-a509-02a4e6c59a27/ir-benchmark-live.json](https://app.devin.ai/attachments/b44c2c3c-9619-4914-a509-02a4e6c59a27/ir-benchmark-live.json)