Local firewall for AI Agents that cuts tokens usage and cost by 40–70%

Guardian Runtime, a local-first security middleware and FinOps firewall for AI agents, now intercepts every prompt and response locally to stop data leaks and runaway token costs. The tool tracks token usage, enforces daily budget limits, and reduces output tokens by 40–70% through aggressive context optimization while scanning for secrets and PII before they reach cloud LLM providers.

A Zero-Latency FinOps & Security Firewall for AI Applications. Intercept every prompt and response locally. Stop data leaks and runaway token costs. 🌐 Website & Docs: https://ashp15205.github.io/guardian-runtime/ https://ashp15205.github.io/guardian-runtime/ 📦 Available on PyPI: https://pypi.org/project/guardian-runtime/ https://pypi.org/project/guardian-runtime/ 🛑 The Core Problem: Why You Need Guardian -the-core-problem-why-you-need-guardian 🟢 The Solution: What is Guardian Runtime? -the-solution-what-is-guardian-runtime 🏗 Architecture -architecture 🚀 Quickstart & Installation -quickstart--installation 🎯 Comprehensive Use Cases Where & How to Use -comprehensive-use-cases-where--how-to-use 💻 Complete CLI Command Reference -complete-cli-command-reference ⚙️ Advanced Configuration Policy YAML %EF%B8%8F-advanced-configuration-policy-yaml 📜 License -license As AI coding agents Claude Code, Cursor, Aider become standard developer tools, they introduce two massive, hidden risks, and one regulatory headache: Autonomous agents operate in loops. If an agent gets stuck retrying a bug fix or accidentally dumps a massive 1GB log file into its context window, you can wake up to a $100 API bill overnight . The Problem: You have zero visibility or control over session costs until the provider's bill arrives at the end of the month. Coding agents require full local codebase access to be useful. However, if you accidentally leave an AWS SECRET KEY or a database password in a .env file, the agent will silently upload it to a third-party LLM provider OpenAI, Anthropic . The Problem: Current observability tools like Langfuse only log the leak after the credentials have already reached the cloud. Sending unauthorized PII like SSNs or emails in a test database to foreign LLM APIs violates GDPR and DPDP regulations. Guardian Runtime is a local-first security middleware and FinOps firewall . It runs entirely on your local machine and intercepts LLM traffic before it leaves your infrastructure. | The Problem | How Guardian Solves It | |---|---| Cost Runaways | Hard FinOps Budgets & Optimization: Tracks every token you spend locally. You can set a strict "$5.00 per day" limit. Advanced Terse Mode aggressively optimizes input context and provides output brevity enforcement via system prompt injection. In benchmarks across real developer prompts, it reduces output tokens by 40–70% while maintaining full technical accuracy. | Data Exfiltration | Zero-Latency Secret Scanners: Scans every prompt for API keys, AWS credentials, and secrets locally. If it detects a secret, it instantly drops the request before it reaches the internet. | Compliance | Local PII Blocking: Regex and ML scanners prevent PII from leaving your machine. | Guardian intercepts traffic at the network layer or via SDK, passing it through a strict verification pipeline before it ever reaches the cloud. Agent / Dev Guardian Runtime Cloud LLM │ │ │ │ 1. Prompt + Context │ │ │ ──────────────────────────▶ │ │ │ │ │ │ │ Security Firewall │ │ │ ├─ Scan AWS Keys / Secrets │ │ │ └─ Block if Threat Detected ──┼─ Drops Request │ │ │ │ │ Token Optimizer │ │ │ ├─ Compress Whitespace │ │ │ └─ Terse Mode Output Trim │ │ │ │ │ │ FinOps Budget │ │ │ ├─ Check Daily Spend Limit │ │ │ └─ Block if $5 Limit Hit ─────┼─ Drops Request │ │ │ │ │ 2. Sanitized Prompt │ │ │ ────────────────────────────▶ │ │ │ │ │ │ 3. LLM Response │ │ │ ◀──────────────────────────── │ │ │ │ │ │ Output Guard │ │ │ Audit for Leaked PII/Secrets │ │ │ │ │ 4. Safe Response │ │ │ ◀────────────────────────── │ │ │ │ │ Guardian Runtime acts as an HTTP proxy or a native Python SDK, meaning it integrates effortlessly with almost any modern AI tool without modifying their internal code. Visual IDEs: Cursor, Windsurf, VS Code via Cline/RooCode Terminal Agents: Claude Code, Aider, GitHub Copilot CLI Frameworks: LangChain, AutoGen, LlamaIndex, CrewAI LLM Providers: OpenAI, Anthropic, Google Gemini via OpenAI compatibility layer Supported Models: Claude Fable 5, Claude Opus 4.8, GPT-4o, Gemini Core framework only pip install guardian runtime Or install with specific LLM providers: pip install "guardian runtime openai " pip install "guardian runtime anthropic " pip install "guardian runtime gemini " Or install everything Providers, ML Scanner, Document Converter : pip install "guardian runtime all " Done. No signup, no keys, zero configuration required. All monitoring data stays on your local machine in ~/.guardian runtime/. Guardian is designed to be universal. Here are the exact ways to deploy it based on your workflow. Why use it here? CLI agents operate autonomously. They can accidentally read a .env file containing your production AWS keys and send it to Anthropic/OpenAI as context. Guardian prevents this and ensures the agent doesn't blow your budget. How to use: - Start the proxy in a background terminal: guardian runtime proxy --port 8080 - Tell your agent to route traffic through the proxy using environment variables: In PowerShell: $env:ANTHROPIC BASE URL="http://localhost:8080" claude In Mac/Linux/Git Bash: export ANTHROPIC BASE URL=http://localhost:8080 claude Why use it here? Modern GUI editors like Cursor have deep codebase access. While coding, you might highlight a file containing a secret and ask "explain this file". Guardian stops Cursor from sending that secret to the cloud. How to use Cursor Example : - Start the proxy in your terminal: guardian runtime proxy --port 8080 - Open Cursor Settings Cmd/Ctrl + , - Navigate to Models Override Base URL - Set the Base URL to: http://localhost:8080 Now all of Cursor's traffic is protected and tracked locally Why use it here? If you are building a production chatbot or RAG pipeline, you must ensure your users cannot perform "jailbreak" prompt injections or trick the LLM into leaking internal system prompts. How to use: Use Guardian as a drop-in replacement for the OpenAI/Anthropic SDK. python import os from guardian runtime import GuardianRuntime, GuardianRuntimeBlockedError os.environ "OPENAI API KEY" = "sk-proj-..." gr = GuardianRuntime Zero-config initialization try: Protects user input before sending to OpenAI response = gr.complete messages= {"role": "user", "content": "My AWS Key is AKIAIOSFODNN7EXAMPLE"} , raise on block=True print response.content except GuardianRuntimeBlockedError as e: Fails cleanly in your app instead of leaking the secret print f"Blocked Locally: {e.response.violations 0 .detail}" Why use it here? Frameworks that spawn multiple communicating agents can rapidly consume tokens. Guardian acts as a central cost-tracking hub for all agent nodes. How to use: Point your framework's base url to the local proxy. python from langchain openai import ChatOpenAI llm = ChatOpenAI model="gpt-4o", base url="http://localhost:8080", Traffic routes through Guardian api key="sk-proj-..." response = llm.invoke "Hello, Guardian " Why use it here? If you use the standard ChatGPT or Claude Web UI, uploading large PDFs eats up your context window quickly because PDFs contain massive amounts of hidden formatting bloat. How to use: Use the built-in CLI to strip out formatting bloat and compress documents into pure Markdown before manually uploading them. guardian runtime convert <path/to/input.pdf --out <path/to/output.md You can now upload cleaned report.md to ChatGPT, saving huge amounts of context space and preventing hallucination. Guardian ships with a powerful suite of offline CLI tools. All data is stored purely locally in ~/.guardian runtime/ . Below is a detailed dive into every command, its flags, and exactly how and why to use it. Starts the local HTTP interception server. This is the core engine for protecting tools that you cannot edit the source code for like Cursor or Claude Code . Flags & Options: --port, -p <int : Port to listen on Default: 8080 . --host <str : Host to bind to. Use 0.0.0.0 to expose on your local network Default: 127.0.0.1 . --policy <path : Path to a custom policy.yaml file. If omitted, uses the default Zero-Config policy $10 budget . --reload : Enables auto-reload if the policy file changes useful for dev mode . Example Usage: bash $ guardian runtime proxy --port 8080 ⛨ GuardianRuntime Runtime Proxy ───────────────────────────────────────── Listening on : http://127.0.0.1:8080 Policy : Default Zero-Config Dashboard : guardian runtime dashboard run in another terminal Agent setup: Claude Code → ANTHROPIC BASE URL=http://localhost:8080 claude Aider → OPENAI BASE URL=http://localhost:8080 aider Cursor → Settings → API Base → http://localhost:8080 Converts massive PDF, DOCX, and XLSX files into highly compressed, token-optimized Markdown. Why use this? If you upload a raw PDF to a Web UI like ChatGPT or parse it in an agent, you waste thousands of tokens on hidden formatting bloat. This command strips the bloat before it hits the LLM context window. Arguments & Flags: <path : The absolute or relative path to the document you want to compress. --out, -o <path : Output file path for the converted Markdown. If omitted, prints a preview to the terminal. Example Usage: bash $ guardian runtime convert <path/to/input file --out <path/to/output file.md ⛨ GuardianRuntime Document Converter Processing: input file... ✓ Conversion Complete • Original File: input file • Token Count: 14,205 • Saved to: output file.md Performs a local security scan on a specific text string using the ML InputGuard and Regex scanners. Why use this? Use this to verify exactly what the firewall will catch before you send a massive codebase to an agent, or if you want to test how sensitive the PII/Secret detection is. Example Usage: bash $ guardian runtime scan "My AWS key is AKIAIOSFODNN7EXAMPLE" 🛑 Scan failed Threats detected: - HIGH secret detected: AWS Access Key ID found. Prints a beautiful terminal summary of today's API costs, token usage, and intercepted threats broken down by tool. Flags: --all : Shows all-time historical analytics instead of just today. Example Usage: bash $ guardian runtime analytics ⛨ GuardianRuntime Session Analytics Today ────────────────────────────────────────────── Claude Code Cost: $2.3100 Requests: 54 Blocked: 3 3 secret detected Tokens: 82,000 : Prints the global help menu listing all available commands and flags. guardian runtime --help : Launches a beautiful React-based local Web UI tracking costs and threats on port 3000. It visualizes the analytics data with charts. guardian runtime dashboard : Tails the local JSONL event stream in real-time guardian runtime logs tail -f ~/.guardian runtime/logs/events.jsonl . Perfect for debugging exactly why a specific prompt was blocked.: Generates a boilerplate guardian runtime init policy.yaml file in your current directory. Use this if you want to customize budgets, disable ML scanners, or enforce strict enterprise PII blocking.: Checks your guardian runtime validate policy.yaml for syntax errors before you restart the proxy.: Shows the health of the local installation, ML models, and storage directory. guardian runtime status : Deletes your entire guardian runtime clean ~/.guardian runtime directory. Use this if you want to permanently delete all local analytics, logs, and custom policies. Guardian Runtime is perfectly tuned out of the box with a $10 daily budget and strict secret scanning. If you need custom rules, run guardian runtime init to create a policy.yaml : version: "1.0" agents: default: llm: provider: openai default model: gpt-4o input guard: scanner enabled: true jailbreak detection: true scanner action: block cost: daily budget: 5.00 Instantly block if daily spend exceeds $5.00 max input tokens: 20000 Block massive context windows to save money optimizer: enabled: true terse mode: true Slashes output tokens by forcing terse shorthand Where will I see the block? If using the Proxy: You will see the block in the terminal running guardian runtime proxy , AND inside the UI of the tool you are using e.g., Claude Code or Aider . If using the Python SDK: It surfaces instantly in your standard Python server logs or terminal. How is it blocked? Proxy Mode: Guardian returns a graceful error with a clear message. This ensures CLI agents display a clean error message in their chat interface instead of crashing or freezing your session. SDK Mode: Guardian raises a GuardianRuntimeBlockedError exception that can be cleanly caught. Example Block Message: BadRequestError: 🚨 SECRET DETECTED AWS key AKIAIOS... found. Request blocked locally. Released under the MIT License . Zero tracking, zero cloud dependencies. Your code is yours.