Show HN: AgentJIT – Compile dynamic LLM agent workflows into 0.1ms Python

AgentJIT, a new open-source Python library, compiles dynamic LLM agent workflows into deterministic Python code that runs in under 0.1 milliseconds with zero LLM tokens consumed on repeat executions, according to its Show HN launch. The tool traces an agent's first run, analyzes the resulting DAG, and generates type-safe Python with runtime guards that fall back to the dynamic LLM agent on unexpected inputs, and it is compatible with LangChain, CrewAI, AutoGen, and OpenAI tool calls. AgentJIT reports hot-path speedups of up to 100,000x over the 15-to-45-second latency of a standard 4-step agent workflow.

Compile flaky, 30-second multi-step AI Agent workflows into 5-millisecond deterministic code. Quickstart quickstart • Why AgentJIT? the-problem-in-2026-why-agentjit • Architecture architecture • Benchmarks benchmarks • Speculative Execution In 2026, autonomous AI agents solve real-world workflows across business, DevOps, and data analysis. However, running stochastic LLM loops in production faces four critical barriers: 1. Massive Latency: A standard 4-step agent workflow think - tool - observe - think takes 15 to 45 seconds . 2. Exponential Costs: Running the loop 10,000 times/day costs thousands of dollars in redundant API tokens. 3. Flakiness & Hallucinations: Even 98% reliability per step leads to compounding errors across multi-turn trajectories. 4. Redundant Reasoning: Most agent invocations execute the exact same structural trajectory with slightly different input parameters e.g. different user IDs or dates . Just like V8 compiles hot JavaScript into machine code, and PyTorch torch.compile traces dynamic tensors into optimized CUDA kernels, AgentJIT traces dynamic agent trajectories and compiles them into pure, type-safe, ultra-fast Python code. Dynamic Agent Task │ 1st run / warmup ▼ ┌──────────────────────┐ │ AgentJIT Tracer │ ── Captures tool calls, data flow, variables └──────────────────────┘ │ ▼ ┌──────────────────────┐ │ DAG Flow Analyzer │ ── Parameter generalization, dependency graph └──────────────────────┘ │ ▼ ┌──────────────────────┐ │ AST Code Generator │ ── Synthesizes pure Python pipeline + Guards └──────────────────────┘ │ ▼ ┌────────────────────────────┐ │ Compiled JIT Pipeline │ ──► Subsequent runs: <1ms, $0 tokens └────────────────────────────┘ │ Guard failure? Deopt ▼ Fall back to LLM Agent - 🏎️ Up to 100,000x Speedup: Hot paths drop from ~20,000 ms to < 0.1 ms . - 💸 100% Token Savings: Once compiled, recurring workflows run completely locally with 0 LLM tokens consumed . - 🛡️ Speculative De-Optimization Bailout : Automatically generates runtime input guards. If unexpected data formats or divergent branches appear, AgentJIT transparently falls back to the dynamic LLM agent. - 🔍 Transparent & Inspectable: Inspect the exact Python code generated by the JIT with agent.source code . - 🧵 Free-Threaded / No-GIL PEP 703 Ready: Thread-safe runtime fully tested on Python 3.13t and 3.14t for true multi-core parallel agent execution without GIL contention. - 🔌 Framework Agnostic: Seamlessly wraps LangChain, CrewAI, AutoGen, OpenAI Tool calls, or native Python functions. pip install agentjit or with uv uv add agentjit Decorate your agent with @jit and mark your tools with @trace tool : python from agentjit import jit, trace tool 1. Define your tools @trace tool def search product name: str : return {"name": name, "price": 49.99, "stock": 120} @trace tool def apply tax price: float, tax rate: float : return round price 1.0 + tax rate , 2 2. Decorate your agent with @jit @jit def checkout agent product name: str, tax rate: float : This dynamic workflow could call an LLM Claude, GPT, Gemini item = search product name=product name total = apply tax price=item "price" , tax rate=tax rate return {"item": item "name" , "total": total} --- Run 1: Warmup & Tracing runs dynamic agent, compiles to Python --- order1 = checkout agent "Mechanical Keyboard", 0.19 --- Run 2+: Instant compiled execution ZERO tokens, sub-millisecond --- order2 = checkout agent "Wireless Mouse", 0.19 Takes 0.05 ms You can view the exact synthesized Python code generated by the JIT at any time: print checkout agent.source code Synthesized Output: python def compiled checkout agent product name, tax rate : """JIT-compiled trajectory pipeline generated by AgentJIT. Executes deterministically in sub-millisecond time with zero token cost. """ --- Speculative Guards --- if not product name is not None : raise GuardViolation "Argument 'product name' must not be None", param="product name" if not isinstance product name, str : raise GuardViolation "Argument 'product name' must be of type str", param="product name" --- Execution Steps --- step 1 out = tools 'search product' name=product name step 2 out = tools 'apply tax' price=step 1 out 'price' , tax rate=tax rate --- Return Final Result --- return {'item': step 1 out 'name' , 'total': step 2 out} Benchmark comparing a simulated 3-step reasoning agent 15s latency, 2,500 tokens vs AgentJIT compiled execution over 100 runs: | Execution Mode | Mean Latency | 99th Percentile | Cost per 1k runs | Token Usage | Determinism | |---|---|---|---|---|---| | Standard LLM Agent | 14,820 ms | 22,400 ms | $75.00 | 2,500,000 | ~94% | | AgentJIT Warm Path | 0.08 ms | 0.12 ms | $0.00 | 0 | 100% | | Improvement | 185,000x faster | 186,000x faster | 100% savings | Zero tokens | Rock-solid | What happens when an input is unusual or triggers an unexpected branch? AgentJIT uses Speculative De-Optimization : 1. Input variables are validated against synthesized guards. 2. If any guard fails e.g. wrong type, missing required key or a tool raises an unhandled exception, AgentJIT catches GuardViolation . 3. It seamlessly bails out to the dynamic LLM agent to handle the edge case. 4. Telemetry records the bailout for future multi-branch specialization. Normal input: runs compiled pipeline in 0.08ms checkout agent "Monitor", 0.19 Divergent input e.g. invalid type : automatically bails out to dynamic agent checkout agent 12345, None Transparently de-optimizes, no crash Monitor your compiled agents in real time: print checkout agent.stats Output: { "total calls": 1500, "compiled hits": 1492, "bailouts": 8, "compiled hit rate": 99.47, "total time saved ms": 22380000.0, "total tokens saved": 3730000 } - Core Tracer & DAG Flow Analyzer - AST Code Generation with Speculative Guards - De-optimization / Bailout Runtime - @jit Decorator with Auto-Warmup - Multi-Branch Polyhedral JIT: Merge multiple execution paths into a unified control-flow graph if/else branching synthesis . - eBPF-Isolated Micro-Sandbox: Ultra-fast sub-millisecond process sandbox for compiled shell actions. - WebAssembly Wasm Export: Compile agent trajectories into standalone Wasm binaries for browser and edge runtime. AgentJIT is open-source software licensed under the Apache 2.0 License /eminsk/agentjit/blob/main/LICENSE .