On June 12, the US government ordered Anthropic to disable Fable 5 globally within six hours, leaving developers without access and highlighting the risk of regulatory takedowns. The following day, GP…
A developer benchmarked four AI gateways—GoModel, LiteLLM, Portkey, and Bifrost—on runtime and deployment overhead. GoModel, a small open-source gateway written in Go, showed significantly lower memor…
A developer outlines five critical observability requirements for production AI agents: full decision path, tool invocation details, per-step cost attribution, session context across restarts, and fai…
A developer built a policy-as-code evaluation pipeline for airline and transport chatbots using LiteLLM and Microsoft ASSERT. The system tests LLM assistants against structured policy scenarios before…
Oracle Cloud announced native support for LiteLLM as a provider for OCI Generative AI Infrastructure, enabling developers to route requests to OCI-hosted models through LiteLLM's gateway with built-in…
AgentGateway outperformed LiteLLM in a fixed-throughput benchmark at 3,000 QPS, sustaining 2,998.94 QPS with sub-millisecond P99 latency while LiteLLM achieved only 2,465.89 QPS with 30.6 ms P99 laten…
A benchmark comparing agentgateway and LiteLLM found agentgateway handled over 11× more requests per second (36,933 QPS vs 3,198 QPS) with sub-2 ms P99 latency while consuming under 30 MB of RAM, vers…
LiteLLM launched the Lite-Harness SDK, enabling developers to swap between AI agent harnesses such as Claude Code and Codex without rewriting application code. The SDK provides a unified query interfa…
OpenClaw-memgpt, a new plugin for the OpenClaw agent framework, implements MemGPT's three-tier memory architecture—core, recall, and archival memory—to provide persistent, searchable memory across ses…
Developer Lore released Wayfinder, an open-source LLM router that determines whether to send a prompt to a local or cloud model by analyzing structural features like length, headings, and code, withou…
An engineer at LiteLLM reveals five cost traps in AI API gateways that can silently inflate bills, including retry logic that multiplies API calls across fallback chains and misconfigured fallbacks th…
The U.S. Department of Commerce's ban on Anthropic's Claude Fable 5 and Mythos 5 models over autonomous zero-day discovery concerns is counterproductive for cloud security, according to a cloud engine…
PeekAI, a new open-source tool for local-first observability and debugging of Python AI agents, launched on Hacker News. The tool stores traces in SQLite, requires no cloud accounts or API keys, and s…
A developer at a company handling thousands of LLM calls per hour cut their API bill by 30% using semantic caching. By embedding prompts and checking cosine similarity against cached responses, they a…
A team of 40 engineers using Claude Code coding agents saw a 340% increase in AI costs, reaching $20K/month in unexpected spend due to raw API keys without per-developer budgets or team caps. The team…
A platform lead at a 150-person company faces the multi-runtime agent problem, where different teams use different agent runtimes (Claude Managed Agents, Bedrock, Cursor, etc.) and need a unified way …
Buildkite ran a game day that fault-injected OpenAI with 429s and 500s to test whether Bifrost's fallback config would reroute requests for an LLM-backed build-failure summariser. After fixing a retry…
OpenCode, an open-source terminal-first coding agent, now supports self-hosted large language models via OpenAI-compatible APIs, enabling users to connect it to a vLLM server running NVIDIA's Nemotron…
Tirith, a new terminal security tool, intercepts commands and pastes to detect homograph attacks, pipe-to-shell patterns, ANSI injection, credential leaks, and 200+ other threats in under 1ms. It cove…
Alibaba's Qwen3-80B and Qwen3-4B models were successfully co-located on a single NVIDIA DGX Spark using vLLM containers behind a LiteLLM proxy, but the 80B model's inability to emit tool calls in auto…