How I built a 3ms self-hosted AI drift proxy in Python (and wiped deleted prompts from SQLite WAL files)

A 15-year-old developer built Wallras, a single-container self-hosted transparent LLM proxy that adds roughly 3–5ms of overhead and detects silent model drift, model swaps and confident hallucinations while keeping prompt data on the user's own servers. The proxy combines three statistical signals with 150-iteration permutation shuffling to hold false alarms to about 1.7% on noisy baselines, scores internal certainty from token logprobs, and scrubs plaintext from SQLite WAL files and unallocated pages after GDPR deletions. It also ships a zero-dependency standard-library Python MCP server (wallras-mcp) exposing drift, trace, blind-spot and prompt-diff tools to coding agents such as Claude Code and Cursor.

If you run LLMs in production—whether OpenAI, Anthropic, Gemini, or self-hosted Ollama/vLLM—you have probably been burned by silent model drift . An upstream provider tweaks quantization or safety filters overnight, refusal rates jump, or an update subtly breaks edge-case prompts without any announcement. Most observability platforms solve this in two frustrating ways: Over the past few months I’m 15, building on a basic setup with no GPU cluster or funding , I built Wallras https://wallras.com —a single-container, self-hosted transparent proxy base url swap, ~3–5ms overhead that catches silent drift, model swaps, and confident hallucinations while keeping 100% of your data on your own servers. Here is a look under the hood at four engineering problems I had to solve while building it. A static rule like "alert if average response length shifts by 15%" breaks immediately on real traffic because some prompts naturally produce longer answers than others. Instead, Wallras combines three statistical signals: |z| 3 for multilingual refusal-rate spikes To avoid crying wolf, it runs 150-iteration permutation shuffling on baseline vs. recent outputs to measure that specific model's own natural random variance. It only raises a drift flag when a shift exceeds both a practical floor and the model's 99th-percentile random noise ceiling ~1.7% false-alarm rate on noisy baselines . You can read the exact formulas and thresholds on the measurement math page https://wallras.com . LLMs frequently sound 95% confident in plain English even when they are guessing a date, API parameter, or legal citation. When token logprobs are available from OpenAI, OpenRouter, Ollama, or vLLM , Wallras computes Shannon entropy across the top alternative tokens alongside the geometric mean and minimum token probability: internal certainty = 0.5 geometric mean prob + 0.5 min token prob Blending the average token probability with the single weakest token ensures that one guessed fact buried inside an otherwise fluent paragraph still triggers a mismatch flag when the model's text sounds overly certain. One of the trickiest security bugs I found during testing was with SQLite deletion requests . When a user triggers a GDPR deletion—or turns on field-level AES-256-GCM encryption for existing records—running a standard SQL DELETE or UPDATE removes the row logically, but leaves the plaintext sitting inside SQLite's write-ahead log .wal and unallocated database file pages until reused. To fix this properly: .wal file after deletion to verify zero plaintext remains. All 366 automated/attack tests, Semgrep/Bandit/Trivy scans, and live A+ Mozilla Observatory / SSL Labs checks are documented on the Wallras security page https://wallras.com . Developers increasingly debug inside Claude Code or Cursor rather than clicking through separate dashboards. Wallras ships a zero-dependency standard-library Python MCP server wallras-mcp and an llms.txt https://wallras.com spec: wallras drift , wallras traces , wallras blind spots , wallras prompt diff so your coding agent can diagnose production issues safely. --enable-writes allow Cursor or Claude to promote a prompt update to production Payments aren't even turned on yet while I validate proof-of-concept—every plan up to Scale is 100% free to try with zero telemetry or prompt data leaving your machine. Since I don't have a multi-GPU rig or heavy API budget to stress-test high concurrency, I would love feedback from anyone running local models or production LLM apps: Feel free to roast the math, proxy overhead, or SQLite/Postgres architecture in the comments