Zero-trust P2P gateway for LLM agents
v2.5.0 — Cross-Node Prompt Routing
Install • Quick Start • Why Tkngate? • Features • Comparison • Docs
Every developer building with LangChain, AutoGen, or CrewAI has experienced the same nightmare: a runaway while
loop that burns $50 in OpenAI credits in 10 minutes. You wake up to a surprise bill with no way to explain which agent did it, when, or why.
Tkngate solves this permanently.
It is a single Go binary that sits as a reverse proxy between your agents and your LLM providers (OpenAI, Anthropic, Groq, etc.). It enforces hard USD circuit breakers, inspects every prompt with a ZK-verified AI-WAF, caches redundant reasoning loops for free, and gives you a real-time dashboard — all without sending a single byte of your data to a third party.
Your AI Agent → tkngate (localhost:7477) → OpenAI / Anthropic / Groq / DeepSeek
"LiteLLM routes. Portkey observes.Tkngate protects."
| Tool | The Gap |
|---|---|
| LiteLLM | |
| Routes between models. No hard budget limits. Requires Python. | |
| Portkey | |
| Cloud-only. Your prompts go through their servers. | |
| Helicone | |
| Observability only. Can't stop a runaway agent. | |
| OpenRouter | |
| You use their shared keys, not yours. | |
| Kong AI Gateway | |
| Enterprise-only. Requires existing Kong infrastructure. |
None of them offer cryptographic prompt security. None of them have a P2P key mesh. All of them either require a cloud account or send your data somewhere.
| Feature | Tkngate | LiteLLM | Portkey | Helicone | |---|---|---|---|---| | Single binary (no Python, no Docker) | ✅ | ❌ | ❌ | ❌ | | 100% self-hosted, no data leaves your machine | ✅ | ✅ | ❌ | ❌ | ZK-SNARK AI-WAF attestations | ✅ | ❌ | ❌ | ❌ | Global P2P Mesh Network (libp2p) | ✅ | ❌ | ❌ | ❌ | P2P encrypted key mesh (DRR) | ✅ | ❌ | ❌ | ❌ | | Hard USD per-session circuit breakers | ✅ | ❌ | || | Shadow Mode A/B traffic splitting | ✅ | ❌ | ❌ | ❌ | | Multi-tenant RBAC + Organizations | ✅ | ✅ | ✅ | ❌ | | Semantic cache (save up to 80%) | ✅ | ❌ | ❌ | ❌ | | Embedded real-time dashboard | ✅ | ❌ | ✅ (cloud) | ✅ (cloud) | | Prometheus metrics export | ✅ | ✅ |
Every competitor routes prompts in plain text. Tkngate is the only LLM proxy that uses ZK-SNARKs (Groth16 BN254) to mathematically prove a prompt is safe without ever reading the prompt content. Enable it with one config flag:
mesh:
strict_zkp_mode: true
go build -o tkngate && ./tkngate serve
That's it. No pip install
. No Docker Compose. No cloud account. No data leaving your machine. SQLite is the only storage engine by default.
Tkngate lets you donate your spare API quota into a cryptographically secured pool. Keys are encrypted with AES-256-GCM and never exposed in plaintext, even to other nodes in the mesh. A Deficit Round-Robin (DRR) algorithm ensures free-riders are throttled automatically.
Most tools let you monitor spend. Tkngate lets you stop it. The moment a session or global budget is breached, the connection is terminated — not after the next billing cycle, right now, at the byte level.
Fork a percentage of live traffic to a second provider and compare outputs and costs — with zero changes to your agent code. Like Canary Deployments for LLMs.
Tkngate nodes automatically discover each other worldwide via Kademlia DHT and locally via mDNS. Reputation updates and fraud alerts are broadcast in real-time using GossipSub with cryptographic signatures. An Anti-Sybil Decay loop continuously penalises idle nodes to prevent adversaries from building dormant proxy armies.
p2p:
enabled: true
listen_port: 7479
enable_relay: true
enable_mdns: true
max_peers: 100
The built-in React dashboard (http://localhost:7478
) shows real-time WAF intercepts, ZKP attestation counts, mesh capacity, budget zones, and per-key spend — all served from the binary itself. The Mesh tab now features a full Peer Reputation Leaderboard with trust score bars, tier badges (🏆 PREMIUM / ✓ TRUSTED / ● NEW), a Fairness Engine health indicator, and network-wide violation tracking.
Run ./tkngate
with no arguments to launch a beautiful interactive menu. Browse organizations, check budgets, donate API keys, view connected P2P peers, test upstream providers, or launch the demo — all without memorising CLI flags.
git clone https://github.com/tkngate/tkngate.git
cd tkngate
go build -o tkngate
cp tkngate.example.yaml tkngate.yaml
./tkngate generate-master-key
./tkngate serve
The proxy starts on localhost:7477
. The telemetry dashboard opens at localhost:7478
.
Point your existing agent at tkngate — no code rewrites needed:
from openai import OpenAI
client = OpenAI(
api_key="your-tkngate-virtual-key",
base_url="http://localhost:7477/v1"
)
js
// Node.js (OpenAI SDK)
const openai = new OpenAI({
apiKey: "your-tkngate-virtual-key",
baseURL: "http://localhost:7477/v1"
});
./tkngate
./tkngate peers list
If you want to see the dashboard in action without pointing real agent traffic at it, you can use the built-in demo traffic generators. These scripts simulate live traffic, WAF intercepts, and reputation slashing without consuming any real API credits.
You can launch the demo in three ways:
Interactive Menu(Recommended): Run./tkngate
and selectdemo (Simulate Live Traffic)
. This starts the proxy, the dashboard, and the demo traffic generator simultaneously.CLI Flag: Run./tkngate serve --demo
. This performs the exact same action as the interactive menu.Standalone: Run./tkngate demo
orpython demo_traffic.py
if you just want to generate background traffic to an isolated database.
Note: The demo scripts safely create and write to a separate budget_demo.db file, ensuring your real configuration and budget limits are never altered.
Asymmetric End-to-End Encryption (E2EE)— Zero-trust prompt encryption using Ed25519-to-X25519 Diffie-Hellman key exchange** Latency-Aware Peer Selection**— dynamically route LLM prompts to the fastest available peer in the global mesh** Cross-Node Prompt Routing**— offload excess traffic to trusted mesh peers when your local rate limits are exhausted** Deficit Round Robin (DRR)— fair, weighted routing across all nodes in the mesh Enterprise Budget Guard**— hard USD limits per-session and globally, with GREEN/AMBER/RED zone alerts** Semantic Caching**— up to 80% savings on repeated or similar prompts (in-memory or Redis)** Dynamic AI-WAF**— intercepts prompt injections, PII leaks, and jailbreaks before they hit your provider** ZK-SNARK Attestations**— cryptographic proof of prompt safety using Groth16 BN254 circuits** Shadow Mode**— silently mirror a percentage of traffic to a second model for evaluation** Universal Fallback**— if OpenAI returns 5xx, auto-retry on Anthropic or DeepSeek transparently** Context Compressor**— strip whitespace and comments from code-heavy prompts (up to 40% token reduction)** Multi-Tenant RBAC**— Virtual Keys with per-org budgets and provider restrictions** Prometheus Export**— native metrics at/metrics
for Grafana, Datadog, or New RelicEmbedded Dashboard— real-time React UI served from the binary itself, no Node.js required** Peer Reputation Leaderboard**— live dashboard table ranking mesh nodes by trust score with tier badges and violation counts** Stake-and-Slash Reputation**— mesh nodes that route malicious prompts get their trust score slashed automatically** Interactive CLI**— beautiful TUI menu for managing everything without memorising flags
Tkngate natively exports Prometheus metrics at http://localhost:7478/metrics
. Hook it into any observability stack:
scrape_configs:
- job_name: 'tkngate'
static_configs:
- targets: ['localhost:7478']
Key metrics:
tkngate_budget_spent_usd_total
— total enterprise API costtkngate_virtual_key_spend_usd_total
— spend per virtual keytkngate_active_connections
— in-flight requeststkngate_cache_hits_total
— semantic cache hitstkngate_waf_intercepts_total
— requests blocked by the AI-WAF
As of v2.0.0
, Tkngate features a full Stake-and-Slash Reputation Engine for peer-to-peer enterprise pooling. Instead of a single bottleneck API key, teams donate multiple keys into a shared pool governed by game theory:
Deficit Round Robin (DRR): Free-riders who consume without donating are automatically throttled.Stake-and-Slash Reputation: Nodes route clean prompts → trust increases. Nodes route flagged prompts → trust is slashed. Low-trust nodes lose access to premium high-TPM keys.ZK-SNARK Verification: Instrict_zkp_mode
, every key access requires a valid zero-knowledge proof, making it impossible to abuse the mesh.Global libp2p Networking (v2.4.0): Nodes discover each other worldwide viaKademlia DHT, locally via** mDNS**, and broadcast reputation updates in real-time via** GossipSubwith Ed25519-signed messages. An Anti-Sybil Decay**loop continuously penalises idle nodes.
| Topic | Link |
|---|---|
| CLI Reference & Architecture | |
docs/budgeting.mddocs/drr-mesh.mddocs/virtual-keys.mddocs/rate-limiting.mddocs/cli-reference.md#wafApache 2.0