Show HN: AgentJIT – Compile dynamic LLM agent workflows into 0.1ms Python

Compile flaky, 30-second multi-step AI Agent workflows into 5-millisecond deterministic code.

Quickstart • ** Why AgentJIT?** • ** Architecture** • ** Benchmarks** • Speculative Execution

In 2026, autonomous AI agents solve real-world workflows across business, DevOps, and data analysis. However, running stochastic LLM loops in production faces four critical barriers:

1. Massive Latency: A standard 4-step agent workflow (think -> tool -> observe -> think ) takes15 to 45 seconds .
2. Exponential Costs: Running the loop 10,000 times/day costs thousands of dollars in redundant API tokens.
3. Flakiness & Hallucinations: Even 98% reliability per step leads to compounding errors across multi-turn trajectories.
4. Redundant Reasoning: Most agent invocations execute theexact same structural trajectory with slightly different input parameters (e.g. different user IDs or dates).

Just like V8 compiles hot JavaScript into machine code, and PyTorch torch.compile traces dynamic tensors into optimized CUDA kernels, AgentJIT traces dynamic agent trajectories and compiles them into pure, type-safe, ultra-fast Python code.

       [Dynamic Agent Task]
                │
         (1st run / warmup)
                ▼
     ┌──────────────────────┐
     │   AgentJIT Tracer    │ ── (Captures tool calls, data flow, variables)
     └──────────────────────┘
                │
                ▼
     ┌──────────────────────┐
     │ DAG Flow Analyzer    │ ── (Parameter generalization, dependency graph)
     └──────────────────────┘
                │
                ▼
     ┌──────────────────────┐
     │ AST Code Generator   │ ── (Synthesizes pure Python pipeline + Guards)
     └──────────────────────┘
                │
                ▼
  ┌────────────────────────────┐
  │   Compiled JIT Pipeline    │ ──► Subsequent runs: <1ms, $0 tokens!
  └────────────────────────────┘
                │
       (Guard failure? Deopt!)
                ▼
     [Fall back to LLM Agent]

• 🏎️ Up to 100,000x Speedup: Hot paths drop from ~20,000 ms to**< 0.1 ms** .
• 💸 100% Token Savings: Once compiled, recurring workflows run completely locally with0 LLM tokens consumed .
• 🛡️ Speculative De-Optimization (Bailout): Automatically generates runtime input guards. If unexpected data formats or divergent branches appear, AgentJIT transparently falls back to the dynamic LLM agent.
• 🔍 Transparent & Inspectable: Inspect the exact Python code generated by the JIT withagent.source_code .
• 🧵 Free-Threaded / No-GIL (PEP 703) Ready: Thread-safe runtime fully tested on Python 3.13t and 3.14t for true multi-core parallel agent execution without GIL contention.
• 🔌 Framework Agnostic: Seamlessly wraps LangChain, CrewAI, AutoGen, OpenAI Tool calls, or native Python functions.

pip install agentjit
uv add agentjit

Decorate your agent with @jit and mark your tools with @trace_tool:

from agentjit import jit, trace_tool

@trace_tool()
def search_product(name: str):
    return {"name": name, "price": 49.99, "stock": 120}

@trace_tool()
def apply_tax(price: float, tax_rate: float):
    return round(price * (1.0 + tax_rate), 2)

@jit
def checkout_agent(product_name: str, tax_rate: float):
    item = search_product(name=product_name)
    total = apply_tax(price=item["price"], tax_rate=tax_rate)
    return {"item": item["name"], "total": total}

order1 = checkout_agent("Mechanical Keyboard", 0.19)

order2 = checkout_agent("Wireless Mouse", 0.19)  # Takes 0.05 ms!

You can view the exact synthesized Python code generated by the JIT at any time:

print(checkout_agent.source_code)

Synthesized Output:

def compiled_checkout_agent(product_name, tax_rate):
    """JIT-compiled trajectory pipeline generated by AgentJIT.
    Executes deterministically in sub-millisecond time with zero token cost.
    """
    if not (product_name is not None):
        raise GuardViolation("Argument 'product_name' must not be None", param="product_name")
    if not (isinstance(product_name, str)):
        raise GuardViolation("Argument 'product_name' must be of type str", param="product_name")

    step_1_out = _tools['search_product'](name=product_name)
    step_2_out = _tools['apply_tax'](price=step_1_out['price'], tax_rate=tax_rate)

    return {'item': step_1_out['name'], 'total': step_2_out}

Benchmark comparing a simulated 3-step reasoning agent (15s latency, 2,500 tokens) vs AgentJIT compiled execution over 100 runs:

What happens when an input is unusual or triggers an unexpected branch?

AgentJIT uses Speculative De-Optimization:

1. Input variables are validated against synthesized guards.
2. If any guard fails (e.g. wrong type, missing required key) or a tool raises an unhandled exception, AgentJIT catches GuardViolation .
3. It seamlessly bails out to the dynamic LLM agent to handle the edge case.
4. Telemetry records the bailout for future multi-branch specialization.

checkout_agent("Monitor", 0.19)

checkout_agent(12345, None)  # Transparently de-optimizes, no crash!

Monitor your compiled agents in real time:

print(checkout_agent.stats)

• Core Tracer & DAG Flow Analyzer
• AST Code Generation with Speculative Guards
• De-optimization / Bailout Runtime
• @jit Decorator with Auto-Warmup
• Multi-Branch Polyhedral JIT: Merge multiple execution paths into a unified control-flow graph (if/else branching synthesis).
• eBPF-Isolated Micro-Sandbox: Ultra-fast sub-millisecond process sandbox for compiled shell actions.
• WebAssembly (Wasm) Export: Compile agent trajectories into standalone Wasm binaries for browser and edge runtime.

AgentJIT is open-source software licensed under the Apache 2.0 License.