Eng

Portkey's AI gateway adds a waste score and request logging to every call, with optional controls for caching, model routing, and security scans. The gateway logs and prices every request, including failures, and computes a 0–100 waste score with named flags. New projects default to 60 requests per minute, 8,000 max prompt tokens, 1,024 max output tokens, and a $5 monthly spend cap.

ControlWhat it doesDefault Request logging & pricingEvery request — including failures — is logged and priced against the versioned rate table.Always on Waste scoreA 0–100 score with named flags on every completed request: repeated prompt, large context, premium model on a small task, low output vs input, latency, error waste.Always on Provider credential encryptionAES-256-GCM with a master key held outside the database. Only a masked hint is ever displayed.Always on Exact response cacheDeterministic requests only — temperature 0, no tools, no response format. 24-hour entries; a hit costs nothing and records the original call's cost as the saving.Off → Recommend → Auto Model routingSmall, simple, low-temperature GPT-4-family calls can move to gpt-4o-mini. Features or endpoints named legal, medical, financial or security are never downgraded.Off → Recommend → Safe auto Prompt-injection scan inbound Scores untrusted input; enforce refuses high-risk requests with 403 before a provider is called.Off → Detect → Enforce Secret-leak scan outbound Signature and entropy detection on the response; enforce replaces the value with a redaction marker.Off → Detect → Enforce Personal-data scan outbound Emails, cards, SSNs, phones, IPs, IBANs; enforce masks them before the response is returned.Off → Detect → Enforce Prompt storageRequest bodies are stored only when you enable it per project — for richer recommendations. Responses are never written to the request log; metadata and one-way hashes are always kept.Off by default Provider failoverOne retry, then your secondary provider — for transient failures only 5xx, 429, network . A 4xx is returned as-is.Configurable Rate, token & spend ceilingsRequests per minute, maximum prompt and output tokens, and a monthly spend cap enforced before the provider is called. New projects start at 60/min, 8,000, 1,024 and $5.Configurable Streaming responsesstream: true forwards SSE chunks in the OpenAI format across every provider. Cost, tokens and the waste score are computed on the assembled response when the stream closes.Supported except under outbound enforce