A Zero-Latency FinOps & Security Firewall for AI Applications. Intercept every prompt and response locally. Stop data leaks and runaway token costs.
🌐 Website & Docs: https://ashp15205.github.io/guardian-runtime/
📦 Available on PyPI: https://pypi.org/project/guardian-runtime/
🛑 The Core Problem: Why You Need Guardian🟢 The Solution: What is Guardian Runtime?🏗 Architecture🚀 Quickstart & Installation🎯 Comprehensive Use Cases (Where & How to Use)💻 Complete CLI Command Reference⚙️ Advanced Configuration (Policy YAML)📜 License
As AI coding agents (Claude Code, Cursor, Aider) become standard developer tools, they introduce two massive, hidden risks, and one regulatory headache:
Autonomous agents operate in loops. If an agent gets stuck retrying a bug fix or accidentally dumps a massive 1GB log file into its context window, you can wake up to a $100 API bill overnight. The Problem: You have zero visibility or control over session costs until the provider's bill arrives at the end of the month.
Coding agents require full local codebase access to be useful. However, if you accidentally leave an AWS_SECRET_KEY
or a database password in a .env
file, the agent will silently upload it to a third-party LLM provider (OpenAI, Anthropic). The Problem: Current observability tools (like Langfuse) only log the leak after the credentials have already reached the cloud.
Sending unauthorized PII (like SSNs or emails in a test database) to foreign LLM APIs violates GDPR and DPDP regulations.
Guardian Runtime is a local-first security middleware and FinOps firewall. It runs entirely on your local machine and intercepts LLM traffic before it leaves your infrastructure.
| The Problem | How Guardian Solves It |
|---|---|
| Cost Runaways | |
| Hard FinOps Budgets & Optimization: Tracks every token you spend locally. You can set a strict "$5.00 per day" limit. Advanced Terse Mode aggressively optimizes input context and provides output brevity enforcement via system prompt injection. In benchmarks across real developer prompts, it reduces output tokens by 40–70% while maintaining full technical accuracy. | |
| Data Exfiltration | |
| Zero-Latency Secret Scanners: Scans every prompt for API keys, AWS credentials, and secrets locally. If it detects a secret, it instantly drops the request before it reaches the internet. | |
| Compliance | |
| Local PII Blocking: Regex and ML scanners prevent PII from leaving your machine. |
Guardian intercepts traffic at the network layer or via SDK, passing it through a strict verification pipeline before it ever reaches the cloud.
Agent / Dev Guardian Runtime Cloud LLM
│ │ │
│ 1. Prompt + Context │ │
│ ──────────────────────────▶ │ │
│ │ │
│ │ [Security Firewall] │
│ │ ├─ Scan AWS Keys / Secrets │
│ │ └─ Block if Threat Detected ──┼─ (Drops Request)
│ │ │
│ │ [Token Optimizer] │
│ │ ├─ Compress Whitespace │
│ │ └─ Terse Mode (Output Trim) │
│ │ │
│ │ [FinOps Budget] │
│ │ ├─ Check Daily Spend Limit │
│ │ └─ Block if $5 Limit Hit ─────┼─ (Drops Request)
│ │ │
│ │ 2. Sanitized Prompt │
│ │ ────────────────────────────▶ │
│ │ │
│ │ 3. LLM Response │
│ │ ◀──────────────────────────── │
│ │ │
│ │ [Output Guard] │
│ │ Audit for Leaked PII/Secrets │
│ │ │
│ 4. Safe Response │ │
│ ◀────────────────────────── │ │
│ │ │
Guardian Runtime acts as an HTTP proxy or a native Python SDK, meaning it integrates effortlessly with almost any modern AI tool without modifying their internal code.
Visual IDEs: Cursor, Windsurf, VS Code (via Cline/RooCode)Terminal Agents: Claude Code, Aider, GitHub Copilot CLIFrameworks: LangChain, AutoGen, LlamaIndex, CrewAILLM Providers: OpenAI, Anthropic, Google Gemini (via OpenAI compatibility layer)Supported Models: Claude Fable 5, Claude Opus 4.8, GPT-4o, Gemini
pip install guardian_runtime
pip install "guardian_runtime[openai]"
pip install "guardian_runtime[anthropic]"
pip install "guardian_runtime[gemini]"
pip install "guardian_runtime[all]"
Done. No signup, no keys, zero configuration required. All monitoring data stays on your local machine in ~/.guardian_runtime/.
Guardian is designed to be universal. Here are the exact ways to deploy it based on your workflow.
Why use it here? CLI agents operate autonomously. They can accidentally read a .env
file containing your production AWS keys and send it to Anthropic/OpenAI as context. Guardian prevents this and ensures the agent doesn't blow your budget.
How to use:
- Start the proxy in a background terminal:
guardian_runtime proxy --port 8080
- Tell your agent to route traffic through the proxy using environment variables: In PowerShell:
$env:ANTHROPIC_BASE_URL="http://localhost:8080"
claude
In Mac/Linux/Git Bash:
export ANTHROPIC_BASE_URL=http://localhost:8080
claude
Why use it here? Modern GUI editors like Cursor have deep codebase access. While coding, you might highlight a file containing a secret and ask "explain this file". Guardian stops Cursor from sending that secret to the cloud.
How to use (Cursor Example):
-
Start the proxy in your terminal:
guardian_runtime proxy --port 8080 -
Open Cursor Settings (
Cmd/Ctrl + ,
) - Navigate to
Models > Override Base URL - Set the Base URL to:
http://localhost:8080
(Now all of Cursor's traffic is protected and tracked locally!)
Why use it here? If you are building a production chatbot or RAG pipeline, you must ensure your users cannot perform "jailbreak" prompt injections or trick the LLM into leaking internal system prompts.
How to use: Use Guardian as a drop-in replacement for the OpenAI/Anthropic SDK.
import os
from guardian_runtime import GuardianRuntime, GuardianRuntimeBlockedError
os.environ["OPENAI_API_KEY"] = "sk-proj-..."
gr = GuardianRuntime() # Zero-config initialization
try:
response = gr.complete(
messages=[{"role": "user", "content": "My AWS Key is AKIAIOSFODNN7EXAMPLE"}],
raise_on_block=True
)
print(response.content)
except GuardianRuntimeBlockedError as e:
print(f"Blocked Locally: {e.response.violations[0].detail}")
Why use it here? Frameworks that spawn multiple communicating agents can rapidly consume tokens. Guardian acts as a central cost-tracking hub for all agent nodes.
How to use:
Point your framework's base_url
to the local proxy.
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
model="gpt-4o",
base_url="http://localhost:8080", # Traffic routes through Guardian
api_key="sk-proj-..."
)
response = llm.invoke("Hello, Guardian!")
Why use it here? If you use the standard ChatGPT or Claude Web UI, up large PDFs eats up your context window quickly because PDFs contain massive amounts of hidden formatting bloat.
How to use: Use the built-in CLI to strip out formatting bloat and compress documents into pure Markdown before manually up them.
guardian_runtime convert <path/to/input.pdf> --out <path/to/output.md>
You can now upload cleaned_report.md to ChatGPT, saving huge amounts of context space and preventing hallucination.
Guardian ships with a powerful suite of offline CLI tools. All data is stored purely locally in ~/.guardian_runtime/
. Below is a detailed dive into every command, its flags, and exactly how and why to use it.
Starts the local HTTP interception server. This is the core engine for protecting tools that you cannot edit the source code for (like Cursor or Claude Code).
Flags & Options:
--port, -p <int>
: Port to listen on (Default:8080
).--host <str>
: Host to bind to. Use0.0.0.0
to expose on your local network (Default:127.0.0.1
).--policy <path>
: Path to a custompolicy.yaml
file. If omitted, uses the default Zero-Config policy ($10 budget).--reload
: Enables auto-reload if the policy file changes (useful for dev mode).
Example Usage:
$ guardian_runtime proxy --port 8080
⛨ GuardianRuntime Runtime Proxy
─────────────────────────────────────────
Listening on : http://127.0.0.1:8080
Policy : Default (Zero-Config)
Dashboard : guardian_runtime dashboard (run in another terminal)
Agent setup:
Claude Code → ANTHROPIC_BASE_URL=http://localhost:8080 claude
Aider → OPENAI_BASE_URL=http://localhost:8080 aider
Cursor → Settings → API Base → http://localhost:8080
Converts massive PDF, DOCX, and XLSX files into highly compressed, token-optimized Markdown.
Why use this? If you upload a raw PDF to a Web UI (like ChatGPT) or parse it in an agent, you waste thousands of tokens on hidden formatting bloat. This command strips the bloat before it hits the LLM context window.
Arguments & Flags:
<path>
: The absolute or relative path to the document you want to compress.--out, -o <path>
: Output file path for the converted Markdown. If omitted, prints a preview to the terminal.
Example Usage:
$ guardian_runtime convert <path/to/input_file> --out <path/to/output_file.md>
⛨ GuardianRuntime Document Converter
Processing: input_file...
✓ Conversion Complete!
• Original File: input_file
• Token Count: 14,205
• Saved to: output_file.md
Performs a local security scan on a specific text string using the ML InputGuard and Regex scanners.
Why use this? Use this to verify exactly what the firewall will catch before you send a massive codebase to an agent, or if you want to test how sensitive the PII/Secret detection is.
Example Usage:
$ guardian_runtime scan "My AWS key is AKIAIOSFODNN7EXAMPLE"
🛑 Scan failed! Threats detected:
- [HIGH] secret_detected: AWS Access Key ID found.
Prints a beautiful terminal summary of today's API costs, token usage, and intercepted threats broken down by tool.
Flags:
--all
: Shows all-time historical analytics instead of just today.
Example Usage:
$ guardian_runtime analytics
⛨ GuardianRuntime Session Analytics (Today)
──────────────────────────────────────────────
Claude Code
Cost: $2.3100
Requests: 54
Blocked: 3 (3 secret_detected)
Tokens: 82,000
: Prints the global help menu listing all available commands and flags.guardian_runtime --help
: Launches a beautiful React-based local Web UI tracking costs and threats on port 3000. It visualizes the analytics data with charts.guardian_runtime dashboard
: Tails the local JSONL event stream in real-time (guardian_runtime logs
tail -f ~/.guardian_runtime/logs/events.jsonl
). Perfect for debugging exactly why a specific prompt was blocked.: Generates a boilerplateguardian_runtime init
policy.yaml
file in your current directory. Use this if you want to customize budgets, disable ML scanners, or enforce strict enterprise PII blocking.: Checks yourguardian_runtime validate
policy.yaml
for syntax errors before you restart the proxy.: Shows the health of the local installation, ML models, and storage directory.guardian_runtime status
: Deletes your entireguardian_runtime clean
~/.guardian_runtime
directory. Use this if you want to permanently delete all local analytics, logs, and custom policies.
Guardian Runtime is perfectly tuned out of the box with a $10 daily budget and strict secret scanning. If you need custom rules, run guardian_runtime init
to create a policy.yaml
:
version: "1.0"
agents:
default:
llm:
provider: openai
default_model: gpt-4o
input_guard:
scanner_enabled: true
jailbreak_detection: true
scanner_action: block
cost:
daily_budget: 5.00 # Instantly block if daily spend exceeds $5.00
max_input_tokens: 20000 # Block massive context windows to save money
optimizer:
enabled: true
terse_mode: true # Slashes output tokens by forcing terse shorthand
Where will I see the block?
If using the Proxy: You will see the block in the terminal runningguardian_runtime proxy
, AND inside the UI of the tool you are using (e.g., Claude Code or Aider).If using the Python SDK: It surfaces instantly in your standard Python server logs or terminal.
How is it blocked?
Proxy Mode: Guardian returns a graceful error with a clear message. This ensures CLI agents display a clean error message in their chat interface instead of crashing or freezing your session.SDK Mode: Guardian raises aGuardianRuntimeBlockedError
exception that can be cleanly caught.
Example Block Message:
BadRequestError: 🚨 [SECRET_DETECTED] AWS key AKIAIOS... found. Request blocked locally.
Released under the MIT License.
Zero tracking, zero cloud dependencies. Your code is yours.