A developer published a step-by-step guide for self-hosting LiteLLM as a private LLM gateway on a Vultr high-performance cloud instance, using Docker Compose to run the LiteLLM proxy alongside a Postg…
A developer outlines a pattern for adding large language models to applications through a thin, provider-agnostic AI layer, using the open-source Vercel AI SDK in TypeScript to route requests between …
A developer benchmarked the LiteLLM Rust gateway beta against the LiteLLM Python v1 proxy, Portkey OSS, and Bifrost 1.6.4, finding the Rust path added roughly 0.7 ms at p99, used 21.8 MB peak memory, …
LLM gateway budget caps such as LiteLLM's max_budget setting fail to enforce spending because they are implemented as advisory checks rather than reservations, according to a technical analysis of the…
Autonomous research agents identify remote code execution vulnerabilities at a 50% rate, nearly double the 26% seen across the broader CVE ecosystem, according to a GTIG report published September 30,…
LiteLLM launched Lens on September 30th, a tracing and analysis product that uses AI agents to find recurring problems across agent runs, according to co-founder and CTO Ishaan Jaffer. Lens stores tra…
Two LiteLLM releases, 1.82.7 and 1.82.8, were published to PyPI with a credential stealer inside after attackers tracked as TeamPCP stole a PyPI publishing token via a compromised Trivy GitHub Action,…
A comparison of five enterprise AI gateways — Bifrost, LiteLLM, Kong, Cloudflare, and Vercel — evaluates each against five procurement requirements: identity, authorization, auditability, isolation, a…
AT&T now routes roughly 40% of its internal AI requests through open-weight models, cutting AI coding costs by 56% while absorbing only a 2% decline in output quality, the company said. The telecom gi…
Google Cloud AI Research, with UNC-Chapel Hill, Stanford and Washington University in St. Louis, released RRSI (Regularized Recursive Self-Improvement), an Apache 2.0 research framework that lets an L…
SigNoz has added an AI Observability section that reads OpenTelemetry GenAI attributes on spans to report LLM cost, token usage, latency, errors, time to first token (TTFT), and tool calls. The Overvi…
SigNoz's AI Observability feature requires spans to carry specific OpenTelemetry GenAI semantic convention attributes, including gen_ai.request.model, gen_ai.provider.name, gen_ai.usage.input_tokens, …
CISA added CVE-2026-59822, an authentication bypass in LiteLLM's MCP Streamable HTTP endpoint rated CVSS 8.8, to its Known Exploited Vulnerabilities catalog on September 2 — the first MCP-specific fla…
AstraBox, an Apache-2.0 open-source, self-hosted alternative to Claude Managed Agents, was released, letting developers run Claude Code, Codex, Hermes, DeepSeek Harness and Pi as cloud agents on their…
Naor Peled, lead maintainer of the open-source PR-Agent project, published a guide and companion CDK repository (pr-agent-lambda-cdk) for self-hosting an AI code review agent on AWS Lambda. The setup …
A developer outlined a set of techniques for cutting LLM API bills by targeting four sources of waste: repeated context, latency-tolerant work, easy questions sent to expensive models, and calls that …
A one-character bearer token was enough to bypass authentication in LiteLLM's MCP Streamable HTTP endpoint, tracked as CVE-2026-59822, because the OAuth2 header path returned an empty authentication o…
A developer who built RelayPlane published an honest comparison of three LLM routing and cost-control tools: RelayPlane, LiteLLM, and OpenRouter. The writeup positions RelayPlane as a free, MIT-licens…
VLLM is an open-source large language model serving engine built around PagedAttention, which manages the KV cache the way an operating system manages virtual memory, and continuous batching, which ke…
Internet measurement has identified 39,155 assets presenting the LiteLLM AI gateway fingerprint, with deployments concentrated in the United States (9,792), China (2,629), Japan (2,030), India (1,855)…