A team at an unnamed company built a customer support agent on LangChain that hallucinated a wrong return policy in a multi-step process, logging success while being confidently wrong. This incident h…
A developer built an open-source PII firewall for LLMs after accidentally sending a customer's credit card number to OpenAI during a benchmark test. The tool, called LLM Governance Engine, intercepts …
A developer evaluated six LLM observability tools—Langfuse, Helicone, Arize Phoenix, LangSmith, Braintrust, and Laminar—for their ability to monitor the audio layer in voice agents, not just LLM calls…
A developer argues that the industry's focus on LLM gateway latency benchmarks is misguided for agent systems. Production agents require session persistence, cost attribution, model routing, fallback …
A developer built ccglass, a local loopback reverse proxy for coding agents, after finding that SaaS AI observability tools like Langfuse and Helicone do not work with native CLIs such as Claude Code …
A developer built TokenWatch, an open-source tool for tracking LLM costs, after Helicone entered maintenance mode following its acquisition by Mintlify in March. The lightweight alternative uses SQLit…
Runcap, a new open-source developer tool, estimates the cost of coding agent runs before they begin and enforces a hard spending ceiling that physically stops the run when the limit is reached. The to…
A company shipped an AI chat feature to production but found the model was a black box, unable to answer basic operational questions about why answers were good or bad. The team added an observability…
A developer built an open-source LLM observability tool designed to prevent vendor lock-in after two major platforms, Helicone and Langfuse, were acquired within months of each other. The tool targets…
A developer built toklock, an open-source proxy that queues API requests to Anthropic's Claude models instead of crashing agents with 429 rate-limit errors. The tool sits between agents and Anthropic'…
Development of SafeRun, an inline reliability infrastructure for AI agents, and argues that post-incident debugging through state-capturing "Replay" functionality is more critical than pre-flight vali…