How Loopers-OSS Saves 76% of Input Tokens and 68% of LLM Spend

Loopers-OSS released an open-source, wire-level AI firewall built on a Dual-Regime Architecture that it says cuts input tokens by 76% and LLM spend by 68% by compressing historical tool outputs in multi-turn agent conversations. In benchmark testing across 60 production-grade agent turns executing enterprise tool operations, the project measured savings against the quadratic context expansion that occurs when agent frameworks re-send megabytes of prior tool results on every API request. The system pairs inline context acceleration and fast-tier arbitrage for normal operations with deterministic circuit breakers, OPA policy gating and hard budgets for runaway loops and tool-execution storms.

Autonomous AI agents are transforming enterprise operations, from automated code generation and incident remediation to business intelligence. However, as organizations transition these agents from simple prototypes to production workflows, they encounter an unexpected financial hurdle: the multi-turn context baggage tax . When an agent executes a multi-step task involving tools such as querying a database, searching logs, or inspecting files , every piece of data returned by a tool is appended to the agent conversation history. Over a 10 to 20 turn trajectory, those historical tool outputs are re-sent to frontier Large Language Models LLMs on every single API request. What began as a lightweight prompt expands into a multi-megabyte payload. To address this challenge, Loopers-OSS introduces an open-source, wire-level AI firewall built on a Dual-Regime Architecture : In empirical benchmark testing across 60 production-grade agent turns executing enterprise tool operations, Loopers demonstrated: This article details the mechanics behind these results, the experimental methodology used to measure them, the architectural components that make it possible, and the practical caveats enterprises must evaluate before deployment. To understand how Loopers works, it is helpful to look at how enterprise teams experience AI agents on two very different kinds of days. ┌──────────────────────────────────────────────────────────────────────────┐ │ THE DUAL-REGIME REALITY │ ├────────────────────────────────────┬─────────────────────────────────────┤ │ BAD DAYS Security Events │ GOOD DAYS Normal Operations │ ├────────────────────────────────────┼─────────────────────────────────────┤ │ • Runaway recursive loops │ • Silent context bloat │ │ • Tool-execution storms │ • Re-sending megabytes of old tools │ │ • Semantic drift and hallucinations│ • Quadratic token cost escalation │ │ • Unexpected budget spikes │ • 60%+ wasted cloud spend │ │ │ │ │ Loopers Action: │ Loopers Action: │ │ Deterministic Circuit Breakers, │ Inline Context Acceleration, │ │ OPA Policy Gating, Hard Budgets │ Ghost Tokens, Fast-Tier Arbitrage │ └────────────────────────────────────┴─────────────────────────────────────┘ Every engineering leader deploying autonomous agents worries about "the bad day": To guard against these bad days, the industry created AI security gateways. These tools act like insurance policies: they monitor traffic, enforce rate limits, and block requests when anomalous behavior occurs. However, pure security gateways suffer from a major adoption barrier: they only provide value on bad days . On normal days, they feel like an added tax, introducing latency and infrastructure overhead without saving a dollar. Now consider "good days." The agent is behaving exactly as designed. It is not looping, it is not hallucinating, and no attacker is injecting prompts. Yet at the end of the month, the cloud bill arrives at five times the projected budget. Why does this happen? In multi-turn autonomous agents, the interaction pattern looks like this: Notice what happens to the network payload on Turn 12: the 30 KB log dump from Turn 2 and the 20 KB database dump from Turn 3 are still included in the prompt , along with every other turn in the conversation. This is quadratic context expansion . The model does not need the complete raw text of the Turn 2 logs to decide which tool to call on Turn 8; it only needed to know that the logs showed a connection timeout error. Yet because standard agent frameworks append everything to the message array, the customer pays for those identical 12,500 input tokens on Turn 4, Turn 5, Turn 6, all the way to the end. On good days, enterprises are not losing money to rogue loops; they are losing money to context baggage . Loopers-OSS is deployed as a drop-in reverse proxy between your agent application and your LLM and MCP providers. It requires zero changes to your agent codebase . You simply configure your agent to route HTTP requests through the Loopers proxy endpoint. ┌─────────────────┐ HTTP / JSON-RPC ┌──────────────────────────────────────────┐ │ │ ────────────────────────── │ LOOPERS-OSS │ │ Autonomous │ │ │ │ Agent Runtime │ │ ┌────────────────────────────────────┐ │ │ Cursor, Claude,│ <────────────────────────── │ │ 1. Schema Minifier & Read Memoizer │ │ │ ZeroClaw, etc. │ Accelerated Wire │ ├────────────────────────────────────┤ │ └─────────────────┘ │ │ 2. Ghost Token Engine Redis/AES │ │ │ ├────────────────────────────────────┤ │ │ │ 3. Prefix Canonicalization │ │ │ ├────────────────────────────────────┤ │ │ │ 4. Smart Fast-Tier Model Router │ │ │ └────────────────────────────────────┘ │ └────────────────────┬─────────────────────┘ │ ┌───────────────────────┴───────────────────────┐ ▼ ▼ ┌─────────────────────────────┐ ┌─────────────────────────────┐ │ Upstream LLM Provider │ │ Upstream MCP Servers │ │ Anthropic, OpenAI, DeepSeek│ │ Databases, Git, Filesystem,│ │ OpenRouter, etc. │ │ Cloud APIs, etc. │ └─────────────────────────────┘ └─────────────────────────────┘ The system operates across four coordinated architectural pillars matching the diagram above: internal/mcp/ When an agent starts a session, it queries registered MCP servers tools/list to discover available functions and then dispatches tool calls tools/call . internal/mcp/sanitizer.go : $schema metadata, title tags, and formatting examples . Loopers truncates tool descriptions to their first sentence, strips non-functional JSON Schema metadata keys, and sorts tools alphabetically. In our benchmark, this compressed MCP schema definitions by 3,200 tokens down to 680 tokens per turn while preserving 100% tool-selection accuracy. internal/mcp/handler.go : blastradius.Calculate <= 15 , confirming pure read-only verbs with zero mutation signals and caches the sanitized response in Redis loopers:mcp:memo:{keyHash}:{sessionID}:{callHash} . Repeat read requests are served directly from Redis in internal/proxy/ghost.go This is the core mechanism for eliminating multi-turn context baggage. 8 KB in production, or 4 KB in our benchmark , Loopers intercepts the response before it enters the agent's conversation history. Authorization: Bearer lp-... and session ID DeriveGhostKey , encrypts the payload with AES-256-GCM, and stores the ciphertext in Redis loopers:blob:{keyHash}:{sessionID}:{blobID} with an automatic TTL default 30 minutes . Because the raw proxy key is never persisted in Redis, a Redis snapshot cannot decrypt offloaded tool outputs. loopers ref , structural metadata row count and sorted keys , and a truncated preview: { " loopers ref": "loopers:blob:9f8e7d6c5b4a3210", "tool": "read audit log", "original bytes": 24800, "est tokens saved": 6200, "structure": {"keys": "binary", "pid", "status", "user" , "rows": 142}, "preview": {"binary": "/bin/sh", "pid": 4102, "status": "elevated", "user": "root"} } isSimpleTurn , HydratePromptTail fetches the referenced blob from Redis, decrypts it in memory, and appends it to the <loopers hydrated context ref="loopers:blob:..." . Crucially, historical messages messages 0..len-2 remain untouched as 80-token Ghost Stubs - preserving upstream byte-level KV-cache prefixes while giving the frontier model 100% of the raw forensic data for final synthesis. internal/proxy/canonicalizer.go Modern LLM providers Anthropic, OpenAI, DeepSeek, OpenRouter discount cached prompt prefixes by 50% to 90%, but only when the prefix is byte-for-byte identical across turns. tools arrays in non-deterministic order or injecting per-request Timestamp: / Request ID: lines into the system prompt. CanonicalizeRequest deterministically sorts system prompt to the tail of the final user message, and injects provider-native cache directives dual cache control: {"type": "ephemeral"} breakpoints on Anthropic prompt cache key pinning on OpenAI/OpenRouter . In Cohort 1, this lifted effective KV-cache hit rates from internal/server/router.go In a typical agent trajectory, not every turn requires a frontier model like deepseek-v4-pro , claude-sonnet-4-6 , or gpt-5.4 . userMsgCount <= 1 , before any upstream KV cache has been established, Loopers routes simple discovery and tool-dispatch turns to a configured fast tier model for example, deepseek/deepseek-v4-pro - deepseek/deepseek-v4.1-flash-1017 on OpenRouter, or claude-sonnet-4-6 - claude-haiku-4-5 on Anthropic . In our benchmark, this delivered an immediate userMsgCount 1 , switching models mid-session would invalidate the 90% KV-cache read discount. Loopers keeps the model pinned to the frontier tier on warm-cache turns while capping max tokens: 1024 on simple intermediate checks and crediting counterfactual savings in the X-Loopers-Saved-USD response header. To verify these savings with scientific rigor, we designed an empirical benchmark using a real autonomous agent executing production-grade tasks. loopers-proxy:8080 , backed by Redis loopers-redis:6379 and an enterprise MCP server loopers-mock-mcp:8081 . To avoid conflating input context compression with multi-tier model routing, the benchmark was split into two isolated cohorts: ┌─────────────────────────────────────────┐ │ 60-Turn Empirical Experiment Design │ └────────────────────┬────────────────────┘ │ ┌────────────────────────────┴────────────────────────────┐ ▼ ▼ ┌─────────────────────────────────────┐ ┌─────────────────────────────────────┐ │ COHORT 1: Pure Fast-Tier Stress │ │ COHORT 2: Enterprise Frontier Stack │ │ Model: deepseek-v4.1-flash-1017 │ │ Model: deepseek-v4-pro - Flash │ ├─────────────────────────────────────┤ ├─────────────────────────────────────┤ │ • Rates: $0.0157/1M in, $0.6439/1M │ │ • Rates: $0.22/1M in, $0.44/1M out │ │ • Price Ratio: 41x Inverted │ │ • Price Ratio: 2x Balanced │ │ • Isolates: Ghost Tokens & Schemas │ │ • Tests: All 4 Pillars + Tier Route │ │ • Tier Routing: Disabled │ │ • Tier Routing: Active │ └─────────────────────────────────────┘ └─────────────────────────────────────┘ deepseek-v4.1-flash-1017 . Because this model has an extreme $0.01569/1M input vs. $0.6439/1M output , output tokens represent ~92% of the bill - providing a harsh stress test for whether input token reduction alone produces net bill savings. deepseek-v4-pro $0.22/1M input, $0.44/1M output with Fast-Tier routing to The table below presents the verified empirical measurements across all 60 turns: | Telemetry Metric | Cohort 1: Baseline | Cohort 1: Treatment Loopers | Cohort 1 Delta | Cohort 2: Baseline | Cohort 2: Treatment Loopers | Cohort 2 Delta | Combined Total Delta | |---|---|---|---|---|---|---|---| | Active Model | v4.1-flash-1017 | v4.1-flash-1017 | Same Model | v4-pro | v4-pro - Flash | Multi-Tier | All Cohorts | | Input Price / 1M | $0.01569 | $0.01569 | -- | $0.22 | Dynamic $0.016-$0.22 | -- | -- | | Output Price / 1M | $0.6439 | $0.6439 | -- | $0.44 | Dynamic $0.44-$0.64 | -- | -- | | Total Input Tokens | 116,098 | 40,724 | -64.92% | 263,796 | 63,876 | -75.79% | -275,294 tokens -72.47% | | Total Output Tokens | 13,860 | 12,999 | -6.21% | 19,051 | 16,016 | -15.93% | Unconstrained | | Input Dollar Spend | $0.001822 | $0.000639 | -64.92% | $0.058035 | $0.014053 | -75.79% | -$0.045165 -75.45% | | Total Turn Spend $ | $0.010745 | $0.009011 | -16.14% | $0.066418 | $0.021021 | -68.35% | -$0.047131 -61.08% | | Read Tool Latency | Network 250ms+ | 3.8 ms Redis | 98.5% faster | Network 250ms+ | 3.8 ms Redis | 98.5% faster | Instant Retrieval | | Quality Preservation | 100% | 100% | Verified | 100% | 100% | Verified | Zero Degradation | In Cohort 2 the enterprise frontier workload , the baseline agent consumed 263,796 input tokens across its investigation steps. With Loopers active, total input tokens dropped to 63,876 : Total customer expenditure in Cohort 2 fell from $0.066418 USD in baseline to $0.021021 USD under Loopers: T1 148,581 to 4,304 $0.034770 to $0.001889 Cohort 1 illustrates a fundamental law of AI FinOps. Even though Loopers compressed input tokens by 64.92% , the net customer bill dropped by 16.14% : $0.001822 input vs. $0.008923 output . gpt-4o , During initial Cohort 2 testing, our telemetry revealed a subtle protocol interaction: when an agent invokes an MCP tool on Turn 1 and returns the tool output to the LLM to complete that turn, the OpenAI/Anthropic messages array already contains 4 items system , user , assistant tool call , tool result . len messages 2 misclassifies the second half of Turn 1 Turn 1b as a warm multi-turn conversation, snapping back to the frontier model mid-turn and hydrating the Ghost Stub onto the raw role: "tool" message. shouldHydrate := isFrontier && isSimpleTurn , and blocking hydration onto No engineering solution is a silver bullet. To evaluate Loopers-OSS responsibly, technical decision-makers must understand where these savings apply and where they do not. As demonstrated in Cohort 1, if your application runs on an ultra-lightweight model where output tokens represent 90%+ of the cost, slashing input tokens by 65% - 75% will reduce your total bill by ~16%. The 68%+ net bill savings occur on frontier reasoning stacks where input context and cold discovery turns dominate total spend. Ghost Tokens, atomic budget leases, and tool memoization require a running Redis 7+ instance local or managed, such as AWS ElastiCache . While Redis operations add sub-4ms overhead, organizations must provision this state store and tune mcp.ghost tokens.ttl seconds default 1800 seconds to match their maximum session duration. Note that because Loopers is fail-closed by design , an unreachable Redis instance will block proxy traffic to guarantee zero unshielded spend. If a state-mutating tool such as execute sql insert or restart service were ever cached, an agent would receive a stale response instead of executing the action. Loopers prevents this deterministically in internal/blastradius/scorer.go : before consulting the Redis memoization cache, Loopers runs blastradius.Calculate toolName, toolArgs and restricts memoization strictly to calls scoring <= 15 pure read-only verbs such as get , list , read , fetch , search with zero write, exec, or destructive argument patterns . Synthesis-gated hydration relies on turn complexity heuristics isSimpleTurn : prompt length, reasoning markers such as "root cause" , "step by step" , "architect" , "synthesize" , and session drift score plus message-role validation role == "user" . For deterministic control in custom pipelines, applications can explicitly override this behavior per request via the X-Loopers-Skip-Hydration: true and X-Loopers-Force-Frontier: true HTTP headers. Loopers-OSS provides a practical solution to the agent adoption dilemma: ┌────────────────────────────────────────────────────────────────────────────┐ │ THE LOOPERS-OSS VALUE PROPOSITION │ ├────────────────────────────────────┬───────────────────────────────────────┤ │ 1. Zero Code Changes: Drop-in reverse proxy on standard HTTP / JSON-RPC. │ │ 2. Dual-Regime Protection: Active safety on bad days; FinOps on good days. │ │ 3. 76% Input Token Reduction: Eliminates multi-turn context baggage. │ │ 4. 68% Net Bill Reduction: Validated across enterprise frontier workflows. │ │ 5. Fully Open Source: MIT licensed, auditable, and self-hostable. │ └────────────────────────────────────────────────────────────────────────────┘ You can launch Loopers-OSS and Redis locally using Docker: 1. Create a shared Docker network and start Redis 7+ docker network create loopers-net docker run -d --name loopers-redis --network loopers-net -p 6379:6379 redis:7-alpine 2. Pull and start the Loopers AI Firewall connected to Redis docker pull ghcr.io/try-loopers/loopers:latest docker run -d --name loopers-firewall --network loopers-net \ -p 8080:8080 \ -e REDIS ADDR=loopers-redis:6379 \ -e SERVER INSECURE DEV=true \ ghcr.io/try-loopers/loopers:latest Alternatively, install the standalone binary via Homebrew macOS / Linux or go install : Via Homebrew brew install try-loopers/tap/loopers Initialize configuration, create an agent proxy key, and start the firewall loopers init loopers keys create --name dev-agent --provider openai SERVER INSECURE DEV=true loopers serve --presets safety,pci,mcp sandbox Once running, you can wrap any agent CLI transparently with loopers exec by setting the generated proxy key in your environment: Export the agent proxy key created above export LOOPERS PROXY KEY="lp-your-proxy-key" Wrap any agent CLI transparently loopers exec -- claude or loopers exec -- aider Alternatively, point your client's base URL directly to http://localhost:8080/lp-your-proxy-key/openai/v1 . Your multi-turn workflows will automatically receive MCP schema minification, Ghost Token payload offloading, read-only memoization, prefix canonicalization, and active runaway protection. Security and efficiency should not be opposing forces in AI engineering. By positioning the firewall directly on the wire between agents, tools, and LLMs, Loopers-OSS transforms an insurance policy for "bad days" into an active cost accelerator for "good days." Cutting 76% of input tokens and 68% of multi-turn LLM spend makes autonomous agent swarms economically viable for production enterprise deployment. Loopers-OSS is an open-source project distributed under the MIT License https://github.com/try-loopers/loopers-oss/blob/main/LICENSE . To inspect the codebase, review the architecture, or contribute, visit github.com/try-loopers/loopers-oss https://github.com/try-loopers/loopers-oss .