{"slug": "show-hn-gavio-open-source-interceptor-pipeline-for-production-llm-applications", "title": "Show HN: Gavio: open-source interceptor pipeline for production LLM applications", "summary": "Gavio, an open-source interceptor pipeline for production LLM applications, has been released. The tool sits between applications and LLM providers, offering PII protection, audit trails, reliability, and cost control through composable interceptors. It supports Python, Java, and JavaScript with identical behavior across languages.", "body_md": "**The open standard AI gateway for production systems.**\n\nPII protection Β· audit trails Β· reliability Β· cost control β as composable\ninterceptors. **Same API in Python, Java, and JavaScript.**\n\nπ **Docs:** [manojmallick.github.io/gavio](https://manojmallick.github.io/gavio/)\n\nGavio sits **between your application and any LLM provider**. Every request\npasses through a pre/post **interceptor chain** β PII redaction, retries, cost\ntracking, audit logging β before and after the provider call:\n\n```\nRequest β [ PII Guard Β· Secret Scanner Β· β¦ ] β Provider β [ β¦ Β· PII Restore Β· Audit ] β Response\n```\n\nEvery team re-implements the same production concerns around LLM calls: redact\nPII before it leaves the building, retry on 429s, fall back to a second\nprovider, log an audit trail, track spend. Gavio ships them once, as swappable\ninterceptors, with **identical behaviour across three languages** β enforced by\n[shared test vectors](/manojmallick/gavio/blob/main/test-vectors).\n\n**Provider-agnostic**β OpenAI, Anthropic, Gemini, Azure, Ollama, Mock. Switching is a config change.** Zero mandatory dependencies**in every core (stdlib HTTP everywhere β no vendor SDKs).** Dev mode**β the whole stack runs in-process with a mock provider. No API key, no network.** Audit by default**β every call logged as metadata + SHA-256 content hashes (never raw text).** Inspector**β opt-in dev-time visualizer: live traces, per-interceptor waterfall, PII redaction diffs, and pipeline lints at`http://127.0.0.1:7411`\n\n(`inspect(true)`\n\nor`GAVIO_INSPECT=1`\n\n).**Inspector agentic & production mode**β multi-agent call graphs and session views, trace replay & edit-resend (full mode only), RED stats, hash-chain verification, PII-sanitized export of any trace as a test case, and a read-only dashboard over a persisted audit store:`gavio inspect --store audit.jsonl`\n\n.\n\nStatus:v0.9.0 (Embedding call guard β F-SEC-10). Semver stability holds since v0.2.0; pre-1.0, some APIs may still change. See the[CHANGELOG].\n\nGavio is a thin core (`Gateway`\n\n+ `InterceptorChain`\n\n+ the request/response\nmodel) that everything else plugs into. A request flows through a **pre**\npipeline, hits a **provider adapter**, then flows back through a **post**\npipeline in reverse order:\n\n```\n βββββββββββββββββββββββββ Gateway.complete(request) βββββββββββββββββββββββββ\n β β\n request β PRE ββΆ PiiGuard ββΆ SecretScanner ββΆ PromptInjectionGuard ββΆ RateLimiter β\n ββββββββΆ β CostControl ββΆ CostRouter ββΆ SemanticCache βββ β\n β β (cache miss) β\n β ββββββββββββΌβββββββββββ β\n β β Provider Adapter β β\n β β OpenAI Β· Anthropic Β· β β\n β β Gemini Β· Azure Β· β β\n β β Ollama Β· Mock β β\n β ββββββββββββ¬βββββββββββ β\n β Guardrails ββ RiskScorer ββ PiiRestore βββββββ β\n ββββββββ β POST ββ Metrics ββ AuditInterceptor (hash-chained record) β response\n β β\n ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ\n```\n\n**Interceptors** implement`before()`\n\n/`after()`\n\n/`onError()`\n\n. Order is explicit β PII redaction runs before audit; audit runs last so it records what every other interceptor did. See[docs/architecture.md](/manojmallick/gavio/blob/main/docs/architecture.md).**Executor policies**(cache, retry, circuit breaker, load balancer, fallback) wrap the provider call itself β a cache hit or an open circuit short-circuits the provider entirely.**The audit record is metadata-only.** Prompts and responses are stored as SHA-256 hashes, never raw text; PII entity*types and counts*are logged, never values. Records are hash-chained (`F-OBS-02`\n\n) so any tampering is detectable.\n\n**Core data model** β identical fields across all three SDKs, defined once in\n[ spec/](/manojmallick/gavio/blob/main/spec) as JSON Schema and enforced by\n\n[shared test vectors](/manojmallick/gavio/blob/main/test-vectors):\n\n`GavioRequest` |\n`GavioResponse` |\n`AuditRecord` |\n|---|---|---|\n`trace_id` (UUID v7) |\n`trace_id` |\n`trace_id` Β· `parent_trace_id` |\n`agent_id` Β· `parent_trace_id` |\n`content` (PII restored) |\n`prompt_hash` Β· `response_hash` |\n`messages` Β· `model` Β· `provider` |\n`usage` Β· `cost_usd` Β· `latency_ms` |\n`pii_entity_types` Β· `risk_score` |\n`options` Β· `lineage` Β· `metadata` |\n`cache_hit` Β· `cache_type` |\n`previous_hash` Β· `lineage` Β· `schema_version` |\n\n| Python | JavaScript / TypeScript | Java |\n|---|---|---|\n|\n\n``` python\nfrom gavio import Gateway\nfrom gavio.interceptors.pii import PiiGuard\n\ngw = (Gateway.builder()\n .dev_mode(True)\n .use(PiiGuard())\n .build())\n\nr = await gw.complete(messages=[\n {\"role\": \"user\",\n \"content\": \"mail jan@example.com\"}])\nprint(r.content) # PII restored\nprint(r.audit.pii_entity_types)\n```\n\n |\n\n``` js\nimport { Gateway } from 'gavio'\nimport { piiGuard } from 'gavio/interceptors/pii'\n\nconst gw = new Gateway({ devMode: true })\n .use(piiGuard())\n\nconst r = await gw.complete({ messages: [\n { role: 'user',\n content: 'mail jan@example.com' }] })\nconsole.log(r.content) // PII restored\nconsole.log(r.audit.piiEntityTypes)\n```\n\n |\n\n```\nGateway gw = Gateway.builder()\n .devMode(true)\n .use(new PiiGuard())\n .build();\n\nvar r = gw.complete(GavioRequest.builder()\n .message(\"user\", \"mail jan@example.com\")\n .build()).join();\nSystem.out.println(r.content());\nSystem.out.println(r.audit().piiEntityTypes());\n```\n\n |\n\nAll three print the reply with the email **restored**, and an audit record\nshowing `EMAIL`\n\nwas detected and redacted before the (mock) provider ever saw it.\n\n| Language | Command | Docs |\n|---|---|---|\nPython 3.10+ |\n`pip install gavio` |\n|\n\n**JavaScript**(Node 18+)`npm install gavio`\n\n[packages/gavio-js](/manojmallick/gavio/blob/main/packages/gavio-js/README.md)Β·[docs/packages/javascript.md](/manojmallick/gavio/blob/main/docs/packages/javascript.md)**Java** 17+ (Maven)`io.github.manojmallick:gavio-core:0.9.0`\n\n[packages/gavio-java](/manojmallick/gavio/blob/main/packages/gavio-java/README.md)Β·[docs/packages/java.md](/manojmallick/gavio/blob/main/docs/packages/java.md)Gavio is a monorepo. Each SDK is independently versioned-in-lockstep and published to its native registry.\n\nThe **reference implementation**. Async-first (`await gw.complete(...)`\n\n), sync\nwrapper (`complete_sync`\n\n), full type hints + `py.typed`\n\n. Zero mandatory deps;\n`gavio[redis]`\n\nadds a distributed cache backend, other optional extras\n(`gavio[presidio]`\n\n, β¦) land in later versions.\n\n```\npip install gavio\n```\n\nβ ** Full Python guide** Β·\n\n[package README](/manojmallick/gavio/blob/main/packages/gavio-py/README.md)\n\nWritten in TypeScript, ships full type definitions, **dual ESM + CJS build**\nwith per-subpath `exports`\n\nfor tree-shaking. Native `fetch`\n\n, `node:crypto`\n\n.\nNode 18+, Deno, Bun.\n\n```\nnpm install gavio\n```\n\nβ ** Full JavaScript guide** Β·\n\n[package README](/manojmallick/gavio/blob/main/packages/gavio-js/README.md)\n\nMulti-artifact Maven project: `gavio-core`\n\nplus one artifact per interceptor\nfamily (`gavio-interceptor-pii`\n\n, `-audit`\n\n, `-reliability`\n\n, `-cache`\n\n,\n`-governance`\n\n, `-guardrails`\n\n, `-metrics`\n\n, `-quality`\n\n), one per provider\n(`gavio-provider-openai`\n\n, `-anthropic`\n\n, `-gemini`\n\n, `-azure`\n\n, `-ollama`\n\n), and\n`gavio-testing`\n\n. Immutable records + builders, `CompletableFuture`\n\nasync,\nJava 17+.\n\n```\n\n io.github.manojmallick\n gavio-core\n 0.9.0\n\n```\n\nβ ** Full Java guide** Β·\n\n[package README](/manojmallick/gavio/blob/main/packages/gavio-java/README.md)\n\nEvery feature below lands in **all three SDKs in lockstep**, at the same version,\ngated by the same [shared test vectors](/manojmallick/gavio/blob/main/test-vectors).\n\n| Feature | ID | Since |\n|---|---|---|\n| PII Guard β Email, IBANΒ·mod-97, BSNΒ·11-proef, CreditCardΒ·Luhn, Phone, IP, SSN | `F-SEC-01` |\n0.1.0 |\nSecret scanner β API keys, AWS `AKIA` , GitHub tokens, JWT, PEM, DB URLs |\n`F-SEC-04` |\n0.1.0 |\n| Prompt-injection defense β pattern corpus + optional semantic similarity | `F-SEC-05` |\n0.2.0 |\nEmbedding call guard β `gw.embed(texts)` runs the same PII pipeline before embedding APIs |\n`F-SEC-10` |\n0.9.0 |\n\n| Feature | ID | Since |\n|---|---|---|\n| Retry (exp backoff + jitter), Fallback chain, Timeout | `F-REL-01/02/07` |\n0.1.0 |\n| Circuit breaker, Load balancer (weighted round-robin) | `F-REL-03/04` |\n0.2.0 |\n| Streaming reliability β buffer response before post-interceptors run | `F-REL-06` |\n0.3.0 |\n\n| Feature | ID | Since |\n|---|---|---|\nPer-request `cost_usd` tracking (all providers) |\n`F-GOV-01` |\n0.1.0 |\n| Budget caps (soft/hard), rate limiting, model RBAC | `F-GOV-02/03/04` |\n0.2.0 |\nCost-optimiser routing β reroute simple prompts to a cheaper model |\n`F-GOV-06` |\n0.5.0 |\n\n| Feature | ID | Since |\n|---|---|---|\n| Semantic + exact cache (cosine + SHA-256), in-memory backends | `F-CACHE-01/02/03` |\n0.2.0 |\n| Redis distributed backend (shared hits across processes, zero-dep RESP2) | `F-CACHE-04` |\n0.4.0 |\n\n| Feature | ID | Since |\n|---|---|---|\nAudit interceptor + `AuditRecord` (SHA-256 hashes), stdout sink |\n`F-OBS-01/05` |\n0.1.0 |\n| Hash-chain (tamper-evident) audit, multi-agent DAG trace | `F-OBS-02/03` |\n0.2.0 |\n| Prompt lineage (template + variables + RAG sources) | `F-OBS-04` |\n0.3.0 |\n| Prometheus metrics (zero-dep text exposition) | `F-OBS-08` |\n0.3.0 |\n| Guardrails β JSON-schema + regex allow/deny | `F-QUA-01/02` |\n0.2.0 |\n| Composite risk scoring (PII + guardrail + injection signals) | `F-QUA-06` |\n0.3.0 |\nJSONL audit sink (`jsonl://` ) β the store the production dashboard reads |\n`F-DX-08` |\n0.7.0 |\n\n| Feature | ID | Since |\n|---|---|---|\n| Dev-time visualizer β live traces (SSE), waterfalls, PII diffs, pipeline lints, embedded UI | `F-DX-09/10` |\n0.6.0 |\nAgent call graphs + session views (`/api/dag` , `/api/sessions` ) |\n`F-OBS-10` |\n0.7.0 |\n| Trace replay & edit-resend (full capture mode only) | `F-DX-11` |\n0.7.0 |\nRead-only production dashboard β RED stats, hash-chain verifier, `gavio inspect --store` |\n`F-DX-08` |\n0.7.0 |\nExport any trace as a PII-sanitized `GavioTestKit` test / test vector |\n`F-DX-12` |\n0.7.0 |\n| Overhead benchmarks with CI-enforced budget (<1% metadata / <5% full p50) | `F-DX-09` |\n0.8.0 |\n\n| Feature | ID | Since |\n|---|---|---|\nDev mode, dry-run mode, `GavioTestKit` |\n`F-DX-01/02/03` |\n0.1.0 |\n| OpenAI drop-in shim, config loader | `F-DX-04/05` |\n0.2.0 |\nProviders β OpenAI Β· Anthropic Β· Gemini Β· Azure OpenAI Β· Ollama Β· Mock (all stdlib HTTP, no vendor SDKs) |\nβ | 0.1β0.2 |\n\nConformance-tested across all three SDKs on every push and PR\n([ ci.yml](/manojmallick/gavio/blob/main/.github/workflows/ci.yml) runs Python 3.10β3.12, Node 18/20/22,\nJava 17/21). Per-release test totals are in the\n\n[CHANGELOG](/manojmallick/gavio/blob/main/CHANGELOG.md); see the\n\n[interceptors guide](/manojmallick/gavio/blob/main/docs/interceptors.md)for every built-in interceptor.\n\nNot yet shipped(tracked on the roadmap): image PII ([#29],`F-SEC-09`\n\n), drift detection ([#31],`F-GOV-07`\n\n), right-to-erasure ([#32],`F-QUA-09`\n\n), license detection ([#33],`F-QUA-10`\n\n).\n\n**P1 Β· Interface-first**β every feature is a public interface you can swap or extend.** P2 Β· Interceptor chain**β pre/post hooks, explicit composition, no hidden magic.** P3 Β· Provider-agnostic**β no provider-specific code leaks into your app.** P4 Β· Zero infra in dev**β`dev_mode`\n\nruns everything in-process.**P5 Β· Audit by default**β opt-out, not opt-in.** P6 Β· Embeddable library**β runs in-process, no sidecar or proxy required.** P7 Β· Dry-run first**β log what*would*happen without blocking.**P8 Β· Typed everywhere**β TS generics, Python hints, Java generics.\n\n| Doc | What |\n|---|---|\n|\n\n[docs/architecture.md](/manojmallick/gavio/blob/main/docs/architecture.md)[docs/interceptors.md](/manojmallick/gavio/blob/main/docs/interceptors.md)[docs/inspector.md](/manojmallick/gavio/blob/main/docs/inspector.md)[docs/otel-mapping.md](/manojmallick/gavio/blob/main/docs/otel-mapping.md)[Grafana dashboard](/manojmallick/gavio/blob/main/docs/grafana/gavio-dashboard.json)[docs/packages/](/manojmallick/gavio/blob/main/docs/packages)[examples/](/manojmallick/gavio/blob/main/examples)[spec/](/manojmallick/gavio/blob/main/spec)[test-vectors/](/manojmallick/gavio/blob/main/test-vectors)[RELEASING.md](/manojmallick/gavio/blob/main/RELEASING.md)[CONTRIBUTING.md](/manojmallick/gavio/blob/main/CONTRIBUTING.md)\n\n```\ngavio/\nβββ spec/ canonical data model (JSON Schema)\nβββ test-vectors/ shared cases every SDK must pass\nβββ packages/\nβ βββ gavio-py/ Python SDK (PyPI: gavio)\nβ βββ gavio-js/ JS/TS SDK (npm: gavio)\nβ βββ gavio-java/ Java SDK (Maven: io.github.manojmallick:gavio-*)\nβββ docs/ documentation\nβββ .github/workflows/ ci.yml (test all 3) Β· release.yml (publish all 3)\n```\n\n[MIT](/manojmallick/gavio/blob/main/LICENSE) Β© 2026 Manoj Mallick\n\nMIT Β© 2026 [Manoj Mallick](https://github.com/manojmallick) Β· Made in Amsterdam π³π±", "url": "https://wpnews.pro/news/show-hn-gavio-open-source-interceptor-pipeline-for-production-llm-applications", "canonical_source": "https://github.com/manojmallick/gavio", "published_at": "2026-07-04 00:04:06+00:00", "updated_at": "2026-07-04 00:19:54.496034+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-infrastructure", "ai-safety", "developer-tools"], "entities": ["Gavio", "OpenAI", "Anthropic", "Gemini", "Azure", "Ollama", "Manoj Mallick"], "alternates": {"html": "https://wpnews.pro/news/show-hn-gavio-open-source-interceptor-pipeline-for-production-llm-applications", "markdown": "https://wpnews.pro/news/show-hn-gavio-open-source-interceptor-pipeline-for-production-llm-applications.md", "text": "https://wpnews.pro/news/show-hn-gavio-open-source-interceptor-pipeline-for-production-llm-applications.txt", "jsonld": "https://wpnews.pro/news/show-hn-gavio-open-source-interceptor-pipeline-for-production-llm-applications.jsonld"}}