{"slug": "local-firewall-for-ai-agents-that-cuts-tokens-usage-and-cost-by-40-70", "title": "Local firewall for AI Agents that cuts tokens usage and cost by 40–70%", "summary": "Guardian Runtime, a local-first security middleware and FinOps firewall for AI agents, now intercepts every prompt and response locally to stop data leaks and runaway token costs. The tool tracks token usage, enforces daily budget limits, and reduces output tokens by 40–70% through aggressive context optimization while scanning for secrets and PII before they reach cloud LLM providers.", "body_md": "**A Zero-Latency FinOps & Security Firewall for AI Applications.\nIntercept every prompt and response locally. Stop data leaks and runaway token costs.**\n\n🌐 **Website & Docs:** [https://ashp15205.github.io/guardian-runtime/](https://ashp15205.github.io/guardian-runtime/)\n\n📦 **Available on PyPI:** [https://pypi.org/project/guardian-runtime/](https://pypi.org/project/guardian-runtime/)\n\n[🛑 The Core Problem: Why You Need Guardian](#-the-core-problem-why-you-need-guardian)[🟢 The Solution: What is Guardian Runtime?](#-the-solution-what-is-guardian-runtime)[🏗 Architecture](#-architecture)[🚀 Quickstart & Installation](#-quickstart--installation)[🎯 Comprehensive Use Cases (Where & How to Use)](#-comprehensive-use-cases-where--how-to-use)[💻 Complete CLI Command Reference](#-complete-cli-command-reference)[⚙️ Advanced Configuration (Policy YAML)](#%EF%B8%8F-advanced-configuration-policy-yaml)[📜 License](#-license)\n\nAs AI coding agents (Claude Code, Cursor, Aider) become standard developer tools, they introduce two massive, hidden risks, and one regulatory headache:\n\nAutonomous agents operate in loops. If an agent gets stuck retrying a bug fix or accidentally dumps a massive 1GB log file into its context window, you can wake up to a **$100 API bill overnight**.\n**The Problem:** You have zero visibility or control over session costs until the provider's bill arrives at the end of the month.\n\nCoding agents require full local codebase access to be useful. However, if you accidentally leave an `AWS_SECRET_KEY`\n\nor a database password in a `.env`\n\nfile, the agent will silently upload it to a third-party LLM provider (OpenAI, Anthropic).\n**The Problem:** Current observability tools (like Langfuse) only log the leak *after* the credentials have already reached the cloud.\n\nSending unauthorized PII (like SSNs or emails in a test database) to foreign LLM APIs violates GDPR and DPDP regulations.\n\nGuardian Runtime is a **local-first security middleware and FinOps firewall**. It runs entirely on your local machine and intercepts LLM traffic *before* it leaves your infrastructure.\n\n| The Problem | How Guardian Solves It |\n|---|---|\nCost Runaways |\nHard FinOps Budgets & Optimization: Tracks every token you spend locally. You can set a strict \"$5.00 per day\" limit. Advanced Terse Mode aggressively optimizes input context and provides output brevity enforcement via system prompt injection. In benchmarks across real developer prompts, it reduces output tokens by 40–70% while maintaining full technical accuracy. |\nData Exfiltration |\nZero-Latency Secret Scanners: Scans every prompt for API keys, AWS credentials, and secrets locally. If it detects a secret, it instantly drops the request before it reaches the internet. |\nCompliance |\nLocal PII Blocking: Regex and ML scanners prevent PII from leaving your machine. |\n\nGuardian intercepts traffic at the network layer or via SDK, passing it through a strict verification pipeline before it ever reaches the cloud.\n\n```\n  Agent / Dev                 Guardian Runtime                   Cloud LLM\n       │                             │                               │\n       │  1. Prompt + Context        │                               │\n       │ ──────────────────────────▶ │                               │\n       │                             │                               │\n       │                             │ [Security Firewall]           │\n       │                             │ ├─ Scan AWS Keys / Secrets    │\n       │                             │ └─ Block if Threat Detected ──┼─ (Drops Request)\n       │                             │                               │\n       │                             │ [Token Optimizer]             │\n       │                             │ ├─ Compress Whitespace        │\n       │                             │ └─ Terse Mode (Output Trim)   │\n       │                             │                               │\n       │                             │ [FinOps Budget]               │\n       │                             │ ├─ Check Daily Spend Limit    │\n       │                             │ └─ Block if $5 Limit Hit ─────┼─ (Drops Request)\n       │                             │                               │\n       │                             │  2. Sanitized Prompt          │\n       │                             │ ────────────────────────────▶ │\n       │                             │                               │\n       │                             │  3. LLM Response              │\n       │                             │ ◀──────────────────────────── │\n       │                             │                               │\n       │                             │ [Output Guard]                │\n       │                             │  Audit for Leaked PII/Secrets │\n       │                             │                               │\n       │  4. Safe Response           │                               │\n       │ ◀────────────────────────── │                               │\n       │                             │                               │\n```\n\nGuardian Runtime acts as an HTTP proxy or a native Python SDK, meaning it integrates effortlessly with almost any modern AI tool without modifying their internal code.\n\n**Visual IDEs:** Cursor, Windsurf, VS Code (via Cline/RooCode)**Terminal Agents:** Claude Code, Aider, GitHub Copilot CLI**Frameworks:** LangChain, AutoGen, LlamaIndex, CrewAI**LLM Providers:** OpenAI, Anthropic, Google Gemini (via OpenAI compatibility layer)**Supported Models:** Claude Fable 5, Claude Opus 4.8, GPT-4o, Gemini\n\n```\n# Core framework only\npip install guardian_runtime\n\n# Or install with specific LLM providers:\npip install \"guardian_runtime[openai]\"\npip install \"guardian_runtime[anthropic]\"\npip install \"guardian_runtime[gemini]\"\n\n# Or install everything (Providers, ML Scanner, Document Converter):\npip install \"guardian_runtime[all]\"\n```\n\n*Done. No signup, no keys, zero configuration required. All monitoring data stays on your local machine in ~/.guardian_runtime/.*\n\nGuardian is designed to be universal. Here are the exact ways to deploy it based on your workflow.\n\n**Why use it here?** CLI agents operate autonomously. They can accidentally read a `.env`\n\nfile containing your production AWS keys and send it to Anthropic/OpenAI as context. Guardian prevents this and ensures the agent doesn't blow your budget.\n\n**How to use:**\n\n- Start the proxy in a background terminal:\n\n```\nguardian_runtime proxy --port 8080\n```\n\n- Tell your agent to route traffic through the proxy using environment variables:\n*In PowerShell:*\n\n```\n$env:ANTHROPIC_BASE_URL=\"http://localhost:8080\"\nclaude\n```\n\n*In Mac/Linux/Git Bash:*\n\n```\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nclaude\n```\n\n**Why use it here?** Modern GUI editors like Cursor have deep codebase access. While coding, you might highlight a file containing a secret and ask \"explain this file\". Guardian stops Cursor from sending that secret to the cloud.\n\n**How to use (Cursor Example):**\n\n- Start the proxy in your terminal:\n`guardian_runtime proxy --port 8080`\n\n- Open Cursor Settings (\n`Cmd/Ctrl + ,`\n\n) - Navigate to\n**Models > Override Base URL** - Set the Base URL to:\n`http://localhost:8080`\n\n*(Now all of Cursor's traffic is protected and tracked locally!)*\n\n**Why use it here?** If you are building a production chatbot or RAG pipeline, you must ensure your users cannot perform \"jailbreak\" prompt injections or trick the LLM into leaking internal system prompts.\n\n**How to use:**\nUse Guardian as a drop-in replacement for the OpenAI/Anthropic SDK.\n\n``` python\nimport os\nfrom guardian_runtime import GuardianRuntime, GuardianRuntimeBlockedError\n\nos.environ[\"OPENAI_API_KEY\"] = \"sk-proj-...\"\ngr = GuardianRuntime() # Zero-config initialization\n\ntry:\n    # Protects user input before sending to OpenAI\n    response = gr.complete(\n        messages=[{\"role\": \"user\", \"content\": \"My AWS Key is AKIAIOSFODNN7EXAMPLE\"}],\n        raise_on_block=True\n    )\n    print(response.content)\nexcept GuardianRuntimeBlockedError as e:\n    # Fails cleanly in your app instead of leaking the secret!\n    print(f\"Blocked Locally: {e.response.violations[0].detail}\")\n```\n\n**Why use it here?** Frameworks that spawn multiple communicating agents can rapidly consume tokens. Guardian acts as a central cost-tracking hub for all agent nodes.\n\n**How to use:**\nPoint your framework's `base_url`\n\nto the local proxy.\n\n``` python\nfrom langchain_openai import ChatOpenAI\n\nllm = ChatOpenAI(\n    model=\"gpt-4o\",\n    base_url=\"http://localhost:8080\", # Traffic routes through Guardian\n    api_key=\"sk-proj-...\"\n)\nresponse = llm.invoke(\"Hello, Guardian!\")\n```\n\n**Why use it here?** If you use the standard ChatGPT or Claude Web UI, uploading large PDFs eats up your context window quickly because PDFs contain massive amounts of hidden formatting bloat.\n\n**How to use:**\nUse the built-in CLI to strip out formatting bloat and compress documents into pure Markdown *before* manually uploading them.\n\n```\nguardian_runtime convert <path/to/input.pdf> --out <path/to/output.md>\n```\n\n*You can now upload cleaned_report.md to ChatGPT, saving huge amounts of context space and preventing hallucination.*\n\nGuardian ships with a powerful suite of offline CLI tools. All data is stored purely locally in `~/.guardian_runtime/`\n\n.\nBelow is a detailed dive into every command, its flags, and exactly how and why to use it.\n\nStarts the local HTTP interception server. This is the core engine for protecting tools that you cannot edit the source code for (like Cursor or Claude Code).\n\n**Flags & Options:**\n\n`--port, -p <int>`\n\n: Port to listen on (Default:`8080`\n\n).`--host <str>`\n\n: Host to bind to. Use`0.0.0.0`\n\nto expose on your local network (Default:`127.0.0.1`\n\n).`--policy <path>`\n\n: Path to a custom`policy.yaml`\n\nfile. If omitted, uses the default Zero-Config policy ($10 budget).`--reload`\n\n: Enables auto-reload if the policy file changes (useful for dev mode).\n\n**Example Usage:**\n\n``` bash\n$ guardian_runtime proxy --port 8080\n  ⛨  GuardianRuntime Runtime Proxy\n  ─────────────────────────────────────────\n  Listening on : http://127.0.0.1:8080\n  Policy       : Default (Zero-Config)\n  Dashboard    : guardian_runtime dashboard (run in another terminal)\n\n  Agent setup:\n    Claude Code  →  ANTHROPIC_BASE_URL=http://localhost:8080 claude\n    Aider        →  OPENAI_BASE_URL=http://localhost:8080 aider\n    Cursor       →  Settings → API Base → http://localhost:8080\n```\n\nConverts massive PDF, DOCX, and XLSX files into highly compressed, token-optimized Markdown.\n\n**Why use this?** If you upload a raw PDF to a Web UI (like ChatGPT) or parse it in an agent, you waste thousands of tokens on hidden formatting bloat. This command strips the bloat *before* it hits the LLM context window.\n\n**Arguments & Flags:**\n\n`<path>`\n\n: The absolute or relative path to the document you want to compress.`--out, -o <path>`\n\n: Output file path for the converted Markdown. If omitted, prints a preview to the terminal.\n\n**Example Usage:**\n\n``` bash\n$ guardian_runtime convert <path/to/input_file> --out <path/to/output_file.md>\n⛨ GuardianRuntime Document Converter\nProcessing: input_file...\n\n✓ Conversion Complete!\n  • Original File:  input_file\n  • Token Count:    14,205\n  • Saved to:       output_file.md\n```\n\nPerforms a local security scan on a specific text string using the ML InputGuard and Regex scanners.\n\n**Why use this?** Use this to verify exactly what the firewall will catch before you send a massive codebase to an agent, or if you want to test how sensitive the PII/Secret detection is.\n\n**Example Usage:**\n\n``` bash\n$ guardian_runtime scan \"My AWS key is AKIAIOSFODNN7EXAMPLE\"\n🛑 Scan failed! Threats detected:\n  - [HIGH] secret_detected: AWS Access Key ID found.\n```\n\nPrints a beautiful terminal summary of today's API costs, token usage, and intercepted threats broken down by tool.\n\n**Flags:**\n\n`--all`\n\n: Shows all-time historical analytics instead of just today.\n\n**Example Usage:**\n\n``` bash\n$ guardian_runtime analytics\n  ⛨  GuardianRuntime Session Analytics (Today)\n  ──────────────────────────────────────────────\n\n  Claude Code\n  Cost:       $2.3100\n  Requests:   54\n  Blocked:    3 (3 secret_detected)\n  Tokens:     82,000\n```\n\n: Prints the global help menu listing all available commands and flags.`guardian_runtime --help`\n\n: Launches a beautiful React-based local Web UI tracking costs and threats on port 3000. It visualizes the analytics data with charts.`guardian_runtime dashboard`\n\n: Tails the local JSONL event stream in real-time (`guardian_runtime logs`\n\n`tail -f ~/.guardian_runtime/logs/events.jsonl`\n\n). Perfect for debugging exactly why a specific prompt was blocked.: Generates a boilerplate`guardian_runtime init`\n\n`policy.yaml`\n\nfile in your current directory. Use this if you want to customize budgets, disable ML scanners, or enforce strict enterprise PII blocking.: Checks your`guardian_runtime validate`\n\n`policy.yaml`\n\nfor syntax errors before you restart the proxy.: Shows the health of the local installation, ML models, and storage directory.`guardian_runtime status`\n\n: Deletes your entire`guardian_runtime clean`\n\n`~/.guardian_runtime`\n\ndirectory. Use this if you want to permanently delete all local analytics, logs, and custom policies.\n\nGuardian Runtime is perfectly tuned out of the box with a $10 daily budget and strict secret scanning. If you need custom rules, run `guardian_runtime init`\n\nto create a `policy.yaml`\n\n:\n\n```\nversion: \"1.0\"\nagents:\n  default:\n    llm:\n      provider: openai\n      default_model: gpt-4o\n\n    input_guard:\n      scanner_enabled: true\n      jailbreak_detection: true\n      scanner_action: block \n      \n    cost:\n      daily_budget: 5.00        # Instantly block if daily spend exceeds $5.00\n      max_input_tokens: 20000   # Block massive context windows to save money\n      \n    optimizer:\n      enabled: true\n      terse_mode: true          # Slashes output tokens by forcing terse shorthand\n```\n\n**Where will I see the block?**\n\n**If using the Proxy:** You will see the block in the terminal running`guardian_runtime proxy`\n\n, AND inside the UI of the tool you are using (e.g., Claude Code or Aider).**If using the Python SDK:** It surfaces instantly in your standard Python server logs or terminal.\n\n**How is it blocked?**\n\n**Proxy Mode:** Guardian returns a graceful error with a clear message. This ensures CLI agents display a clean error message in their chat interface instead of crashing or freezing your session.**SDK Mode:** Guardian raises a`GuardianRuntimeBlockedError`\n\nexception that can be cleanly caught.\n\n**Example Block Message:**\n`BadRequestError: 🚨 [SECRET_DETECTED] AWS key AKIAIOS... found. Request blocked locally.`\n\nReleased under the **MIT License**.\n\nZero tracking, zero cloud dependencies. Your code is yours.", "url": "https://wpnews.pro/news/local-firewall-for-ai-agents-that-cuts-tokens-usage-and-cost-by-40-70", "canonical_source": "https://github.com/ashp15205/guardian-runtime", "published_at": "2026-06-11 17:28:27+00:00", "updated_at": "2026-06-11 17:34:25.618394+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-safety", "ai-infrastructure", "ai-products"], "entities": ["Guardian Runtime", "Claude Code", "Cursor", "Aider", "PyPI"], "alternates": {"html": "https://wpnews.pro/news/local-firewall-for-ai-agents-that-cuts-tokens-usage-and-cost-by-40-70", "markdown": "https://wpnews.pro/news/local-firewall-for-ai-agents-that-cuts-tokens-usage-and-cost-by-40-70.md", "text": "https://wpnews.pro/news/local-firewall-for-ai-agents-that-cuts-tokens-usage-and-cost-by-40-70.txt", "jsonld": "https://wpnews.pro/news/local-firewall-for-ai-agents-that-cuts-tokens-usage-and-cost-by-40-70.jsonld"}}