{"slug": "jit-context-os-epistemic-context-runtime-for-coding-agents", "title": "JIT Context OS – Epistemic context runtime for coding agents", "summary": "JIT-Context OS, an open-source epistemic context runtime for coding agents developed by Wojciech Wiesner of The Ones, reports a 2.44x faster task delivery time (3m 52s vs. 9m 27s) and a 61.4% reduction in LLM inference turns (66 vs. 171) compared to standard long-context methods in a benchmark on the Synthapse Web Audio codebase. The runtime, version 0.2.4, claims to eliminate error loops and reduce prompt context size by over 99% (482 tokens vs. >50,000 tokens) through its Epistemic Invariants and SQLite-based hot-path storage.", "body_md": "**Version:** 0.2.4 (2026-09-08)\n\n**Author:** Wojciech Wiesner (`wojciech@theones.io`) — *The Ones* (` join.theones.io`)\n\n**DOI:** [10.5281/zenodo.22649542](https://doi.org/10.5281/zenodo.22649542)\n\n**Repository:** [https://github.com/wojciechwiesner/jit-context](https://github.com/wojciechwiesner/jit-context)\n\n**Paired Production Codebase Benchmark** (*Synthapse Web Audio*, 45 modules, Vitest E2E):\n\nHead-to-head evaluation of autonomous coding agents under standard long-context dump vs. JIT-Context OS:\n\n| Metric | Baseline (Standard Long-Context) | JIT-Context OS | Production Moat / Delta | \n|---|---|---|---|\n| **Task Delivery Time** | 9m 27s (567s) | **3m 52s (232s)** | **2.44x Faster Delivery (-59.0%)** | \n| **LLM Inference Turns** | 171 API calls | **66 API calls** | **-61.4% Turns (-105 rounds avoided)** | \n| **Tool Execution Churn** | 169 tool ops | **64 tool ops** | **-62.1% Agent Churn Reduction** | \n| **File Read Churn** | 73 file reads | **24 file reads** | **-67.1% Less Context Thrashing** | \n| **Scope Drift / Collateral Edits** | 14 files touched (drift) | **4 files (surgical SRP)** | **Zero Scope Drift** | \n| **Runtime & Test Error Loops** | 8 error loops | **0 errors (clean first-shot)** | **100% Error Loop Elimination** | \n| **Prompt Context Size** | >50,000 tokens (Haystack) | **482 tokens (Capsule)** | **>99% Token Reduction** | \n| **Hot-Path RYOW Latency** | 200–800ms (Vector API) | **<3ms (SQLite WAL)** | **Zero-Latency Ground Truth** | \n| **Epistemic Invariants** | Vulnerable to self-poisoning | **10/10 PASS (I1–I10)** | **100% Anti-Hallucination Gate** | \n\nModern LLM-based autonomous agent architectures suffer from the **\"Haystack Tax\"**: context window inflation (50k–100k+ tokens), severe attention degradation (*Lost-in-the-Middle*), self-poisoning through recursive consumption of prior assistant speculation, and severe rate-limiting (`429 Too Many Requests` / rolling context window exhaustion).\n\n**JIT-Context** is a universal, deterministic, multi-tier temporal memory and context runtime with **Epistemic Invariants (I1–I10)**. It serves as an architectural drop-in runtime for autonomous agents (Claude Code, Hermes, Cursor, OpenCode, Codex, and custom multi-agent harnesses).\n\nBy decoupling hot-path operational state (<3ms local SQLite WAL) from slow associative brokers and enforcing strict authority weighting (User Authority = 1.0, Assistant Speculation = 0.0, Runtime Tool Proof = 1.0), **JIT-Context** compiles a **Lean Context Capsule (400–1,200 tokens, ceiling <1,500 tokens)** just-in-time for each turn.\n\n| Capability / Challenge | Semantic Vector Stores (Mem0 / Zep) | Self-Managed Agent Memory (Letta / MemGPT) | Context Stuffing (100k+ Dump / Prompt Caching) | **JIT-Context (This Architecture)** | \n|---|---|---|---|---|\n| **Hot-Path Latency** | Slow (200–800ms API / embedding) | Moderate (LLM decides tool call) | Zero (Static Prompt) | **<3ms (Local SQLite WAL, In-Memory)** | \n| **Read-Your-Own-Writes** | Eventual consistency / indexing lag | Delayed by multi-turn tool loops | N/A (Frozen context) | **Instant RYOW (<0.5ms)** | \n| **Self-Poisoning Vulnerability** | HIGH (re-ingests assistant answers) | HIGH (agent writes own core facts) | MODERATE (hallucinations stay in transcript) | **ZERO (Assistant Epistemic Weight = 0.0)** | \n| **Scope Drift Resistance** | LOW (fuzzy similarity pulls other repos) | LOW (unconstrained agent queries) | VERY LOW (lost-in-the-middle confusion) | **HIGH (L1 Scope Hysteresis Guard)** | \n| **Prompt Cache Alignment** | POOR (dynamic injected text breaks cache) | POOR (frequent core memory edits) | MODERATE (large prefix, expensive cache misses) | **>95% Cache Hit Rate (Prefix-stable)** | \n| **Hot-Reloading in RAM** | Requires service restart | Process rebuild | Re-prompting required | **Dynamic In-Process Hot-Reload (<1ms)** | \n| **Token Economy** | Inflates prompt with Top-K fragments | Multiple turns of tool-calling overhead | Severe ($$$ context tax, rolling 5h limits) | **Lean Capsule (400–1,200 tokens)** | \n\n```\n                    ┌───────────────────────────────────────────────┐\n                    │               USER / TASK INPUT               │\n                    └───────────────────────┬───────────────────────┘\n                                            │\n                                            ▼\n  ┌──────────────────────────────────────────────────────────────────────────────────┐\n  │                           JIT-CONTEXT RUNTIME COMPILER                           │\n  │                                                                                  │\n  │   ┌───────────────────────┐  ┌───────────────────────┐  ┌───────────────────────┐│\n  │   │  L0 HOT-PATH (<3ms)   │  │ L1 WARM-PATH (<10ms)  │  │  L2 DEEP-PATH (600ms) ││\n  │   │  • SQLite WAL Overlay │  │ • Project Scope Cache │  │  • Circuit Breaker    ││\n  │   │  • Read-Your-Own-Write│  │ • Scope Hysteresis    │  │  • Fail-Open Policy   ││\n  │   │  • Runtime Tool Proofs│  │ • Active vs Retrieval │  │  • Associative Broker ││\n  │   │  • Dynamic Hot-Reload │  │ • Session CWD Track   │  │  • Ollama Fallback    ││\n  │   └───────────┬───────────┘  └───────────┬───────────┘  └───────────┬───────────┘│\n  │               │                          │                          │            │\n  │               └───────────────────┬──────┴──────────────────────────┘            │\n  │                                   ▼                                              │\n  │                    ┌─────────────────────────────┐                               │\n  │                    │  EPISTEMIC ARBITER (I1–I10) │                               │\n  │                    │  • User Authority = 1.0     │                               │\n  │                    │  • Assistant Weight = 0.0   │                               │\n  │                    │  • Tool Proof Weight = 1.0  │                               │\n  │                    │  • Anti-Self-Poisoning Gate │                               │\n  │                    └──────────────┬──────────────┘                               │\n  │                                   ▼                                              │\n  │                    ┌─────────────────────────────┐                               │\n  │                    │    LEAN CONTEXT CAPSULE     │                               │\n  │                    │    400–1,200 TOKENS (<10ms) │                               │\n  │                    │   (Ceiling <1,500 tokens)   │                               │\n  │                    └──────────────┬──────────────┘                               │\n  └───────────────────────────────────┼──────────────────────────────────────────────┘\n                                      ▼\n                      ┌───────────────────────────────┐\n                      │    STABLE PREFIX LLM CACHE    │\n                      │    (Gemini / Claude / GLM)    │\n                      │    • 4.51x Faster TTFT        │\n                      │    • Zero 429 Rate Limits     │\n                      └───────────────────────────────┘\n```\n\nA live multi-module software engineering duel: autonomous agents were tasked with fixing 4 distinct root-cause bugs across 3 interconnected modules (`event_pipeline.py`, `retry_policy.py`, `storage.py`) verified by an independent `pytest` suite.\n\n| Competitor | Runtime & Configuration | Total Turns | Wall-Clock Time | Pytest Verification | API / Hardware Cost | Privacy Guarantee | \n|---|---|---|---|---|---|---|\n| 🥇 **Local Qwen 3.8 9B + JIT** | Local Metal M2 Pro (Ollama 32k context) | **4 turns** | ~2 min (126s) | **4/4 PASSED (exit 0)** | **0.00 PLN** | **100% Local / Zero Data Leak** | \n| 🥈 **Gemini 3.8 Flash + JIT** | Google Cloud Frontier API | **9 turns** | **17.77s** | **4/4 PASSED (exit 0)** | Paid Cloud API | Cloud API payload | \n| 🥉 **Gemini 3.8 Flash WITHOUT JIT** | Google Cloud Frontier API (Raw Chat History) | **10 turns** | 34.57s | ❌ **0/4 FAILED** | Paid Cloud API | Cloud API payload | \n\n1. **Local Model Turn Dominance:** With JIT Context OS maintaining a calibrated ~1.8k token working set, the local 9B model on a Mac Mini resolved the multi-module task in**only 4 turns** — more than 2x fewer turns than Google's Gemini 3.8 Flash in the cloud.\n2. **Parallel Tool Calling Precision:** In Turn 1, Qwen dispatched 4 parallel`read_file` calls. In Turn 2, it executed 3 surgical parallel`write_file` calls fixing all 4 root causes in one shot, passing tests on the first verification attempt.\n3. **The Haystack Failure Mode:** Without JIT, Google's flagship Gemini 3.8 Flash got lost in conversational history and`tests/` directory loops, failing to resolve the issue within the turn budget.\n\nA head-to-head paired benchmark was executed on the production repository **Synthapse** (Web Audio Generative AI Techno Instrument, ~45 modules, Vitest + Vite build). Autonomous agents were tasked with implementing direct MP3 audio export and recording alongside WAV:\n\n| Metric | With JIT-Context (Calibrated Capsule) | Control (No JIT / Long-Context) | Delta / Real Impact | \n|---|---|---|---|\n| **Wall Clock Time** | **232.43 s** (3m 52s) | **567.57 s** (9m 27s) | **-59.0% (2.44x faster)** | \n| **LLM API Calls (Turns)** | **66** | **171** | **-61.4% (105 rounds avoided)** | \n| **Total Tool Calls** | **64** | **169** | **-62.1%** | \n| **Files Read (`read_file`)** | **24** | **73** | **-67.1% (3x less context churn)** | \n| **Discovery Ops Before Edit** | **31** | **65** | **-52.3%** | \n| **Time to First Code Mutation** | **125 s** | **202 s** | **-38.1% (-77s)** | \n| **Runtime / Test / Patch Errors** | **0** (100% clean) | **8 errors** (tests, syntax, patch) | **100% error loop elimination** | \n| **Scope Drift (Files Touched)** | **4 files** (surgical SRP) | **14 files** (severe drift into DJ/Studio) | **Zero scope drift** | \n| **Context Capsule Size** | **482 tokens** (calibrated architecture) | Monolithic workspace dump (>50k tok) | **100x leaner context window** | \n| **Verification Suite** | **PASS** (199/199 Vitest tests) | **PASS** (216/216 tests after 8 fixes) | First-shot clean build | \n\nEvaluates Invariant I3 & I4 against `gemini-3.8-flash`: when an assistant falsely asserts a service is running and deployed, but the physical tool returned `exit_code: 1 (Port 8080 already bound)`:\n\n- **Control (No JIT / Raw Chat History):** The LLM read previous assistant claims, compounding the speculation.\n- **JIT Context OS v0.2.4:** Assistant text rejected (weight 0.0). Physical tool output compiled into`[VERIFIED RUNTIME PROOFS (Authority 1.0)]` .\n- **Result:****100% Grounded Accuracy** — model answered:*\"NIE – w tej sesji nie przeprowadzono wdrożenia ani nie wykonano weryfikacji uruchomieniowej potwierdzającej działanie serwisu na porcie 8080.\"*\n\n- **EXP-006 (Synthapse DSP Arbitration):** -86.3% prompt tokens (774 vs 5,659), 1.4x faster latency (3.24s vs 4.55s), 100% exact retrieval of in-flight audio parameters (142 BPM, 3200 Hz cutoff, sidechain OFF).\n- **EXP-007 (Tuli.my Relational Epistemics):** -76.6% prompt tokens (783 vs 3,347), 100% strict adherence to user-defined boundary overrides.\n\n- \n**I1 (Direct User Input Supremacy):** Unambiguous human instructions instantly supersede all prior assumptions with Authority = 1.0.\n- \n**I2 (Atomic Monotonic Sequence):** Every turn and state change receives an atomic sequence number (`RETURNING seq` ) in SQLite WAL.\n- \n**I3 (Anti-Self-Poisoning):** Assistant generated text is assigned`epistemic_weight = 0.0` . Speculations never pollute canonical truth.\n- \n**I4 (Read-Your-Own-Writes / RYOW):** Updates committed in turn$N$ are guaranteed readable in turn$N+1$ in$<0.5$ ms.\n- \n**I5 (Scope Hysteresis):** Cross-project queries expand retrieval scope without thrashing the primary active workspace.\n- \n**I6 (Double Circuit Breaker & Fail-Open):** L2 broker latency is capped at 600ms. Triple network failures trigger an open circuit with zero downtime.\n- \n**I7 (Bounded Derived Authority):** Inferred observations from tools are capped at Authority = 0.70 until physically verified (`exit_code: 0` = 1.0).\n- \n**I8 (Event Idempotency & Budget Ceiling):** Dynamic capsules strictly bounded below <1,500 tokens to preserve prompt caching stability.\n- \n**I9 (Deterministic Fallback):** If memory subsystems fail, the agent falls back to pristine system prompts seamlessly.\n- \n**I10 (Prompt Caching Prefix Alignment):** Frozen system manifests are positioned before dynamic capsules, achieving >95% prompt cache hit rates.\n\n```\njit-context/\n├── benchmarks/              # Empirical and synthetic benchmark datasets\n│   ├── EXP-001-CANARY.json\n│   ├── EXP-002-STATISTICAL-20.json\n│   ├── EXP-003-TRI-VARIANT.json\n│   ├── EXP-004-ABLATION.json\n│   └── EXP-005-REAL-WORLD-SYNTHAPSE.json\n├── src/\n│   ├── context/\n│   │   ├── compiler.py          # Multi-tier capsule compiler (<10ms)\n│   │   └── cascade_distiller.py # 3-tier cascade distiller & elastic budget\n│   ├── health/\n│   │   ├── doctor.py            # Automated system verification CLI\n│   │   ├── exp005_runner.py     # Needle-in-haystack benchmark\n│   │   ├── exp006_synthapse.py  # DSP state arbitration benchmark\n│   │   ├── exp007_tulimy.py     # Relational boundary benchmark\n│   │   └── exp008_tool_epistemics.py # Runtime tool anti-hallucination test\n│   ├── hooks.py                 # Hermes & agent harness integration hooks\n│   ├── init.py                  # /jit init project profiler CLI\n│   ├── l0/\n│   │   ├── db.py                # SQLite WAL database layer (<1ms)\n│   │   ├── epistemics.py        # Authority & epistemic weighting engine\n│   │   └── overlay.py           # Read-Your-Own-Writes operational overlay\n│   ├── l1/\n│   │   ├── project_cache.py     # Scope cache with hysteresis (<10ms)\n│   │   └── scope.py             # Scope resolution with hysteresis\n│   ├── l2/\n│   │   ├── broker_client.py     # Deep retrieval client with circuit breaker\n│   │   └── distill.py           # Context compressor & pruner\n│   ├── telemetry/\n│   │   └── observatory.py       # Realtime HTTP telemetry server (:8765)\n│   └── tests/                   # 27 automated unit and regression tests\n├── CITATION.cff                 # CERN Zenodo citation metadata\n├── MANIFEST.sha256              # Cryptographic file integrity manifest\n└── README.md\n# Clone the repository\ngit clone https://github.com/wojciechwiesner/jit-context.git\ncd jit-context\n\n# Install package & dependencies\npip install -e .\n\n# Run the 27 automated unit & invariant tests\npytest src/tests/\n# Run Doctor check verifying 10/10 Invariants\npython3 src/health/doctor.py\n# Start local metrics server on port 8765\npython3 src/telemetry/observatory.py\nopen http://127.0.0.1:8765/\n```\n\nIf you use or reference **JIT-Context** in academic research or production agent frameworks, please cite:\n\n```\n@software{wiesner2026jitcontext,\n  author       = {Wiesner, Wojciech},\n  title        = {JIT-Context: An Epistemic Context Runtime for AI Agents},\n  year         = 2026,\n  publisher    = {Zenodo},\n  version      = {v0.2.4},\n  doi          = {10.5281/zenodo.22649542},\n  url          = {https://doi.org/10.5281/zenodo.22649542}\n}\n```\n\nMIT License — Copyright (c) 2026 Wojciech Wiesner (`wojciech@theones.io`) — *The Ones* (` join.theones.io`).", "url": "https://wpnews.pro/news/jit-context-os-epistemic-context-runtime-for-coding-agents", "canonical_source": "https://github.com/wojciechwiesner/jit-context", "published_at": "2026-09-08 06:27:04+00:00", "updated_at": "2026-09-08 07:01:48.943871+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "ai-research"], "entities": ["JIT-Context OS", "Wojciech Wiesner", "The Ones", "Synthapse Web Audio", "SQLite", "Claude Code", "Cursor", "OpenCode"], "alternates": {"html": "https://wpnews.pro/news/jit-context-os-epistemic-context-runtime-for-coding-agents", "markdown": "https://wpnews.pro/news/jit-context-os-epistemic-context-runtime-for-coding-agents.md", "text": "https://wpnews.pro/news/jit-context-os-epistemic-context-runtime-for-coding-agents.txt", "jsonld": "https://wpnews.pro/news/jit-context-os-epistemic-context-runtime-for-coding-agents.jsonld"}}