Arize AI aligned the automated evaluators for its Alyx AI assistant in Arize AX by sampling production sessions into a labeling queue and building a Custom View that places the full session transcript…
Arize Phoenix contributor Ashwin Govind Ugale released tracelint, an open-source linter that runs deterministic structural checks on OpenInference traces and returns a CI-usable exit code, avoiding LL…
Arize AI shipped a hosted MCP server for its Arize AX observability platform, exposing 45 tools plus server_info across projects, traces, datasets, experiments, evaluators, monitors, and prompts, conn…
Anthropic's Lance Martin published "Automating eval design and hillclimbing with Claude" on September 28, adding two commands to the claude-api skill: /claude-api build-eval, which interviews the user…
A comparison of four AI agent observability and evaluation tools — Langfuse, Arize Phoenix, Promptfoo, and Iris — finds they solve the same problem in fundamentally different ways: Langfuse and Phoeni…
Arize shipped long-term memory for Alyx, its AI engineering agent inside Arize AX, as a single structured text file per user and space capped at 8,000 characters that the agent reads in full on every …
Arize AX has added long-term memory to Alyx, its AI engineering agent, enabling it to retain project goals, conventions, preferences and prior decisions across sessions, and the feature is enabled by …
NVIDIA published a tutorial showing developers how to trace Hermes Agent behavior with NVIDIA NeMo Relay, running two Hermes Agent examples and inspecting the resulting traces for model and tool calls…
Arize AI's Jev 1.13.0 classifier matched LLM judges on accuracy while its returned probability strongly predicted its own correctness across 36,190 judgments, according to an Arize benchmark spanning …
Arize published a tutorial showing how to connect TypeSafe's Jev to Arize AX through a remote evaluator that scores recorded agent spans after the agent responds, returning a yes/no label and a probab…
Daytona cofounder and CEO Ivan Burazin said the agents he uses fail more often because they cannot access the tools they need than because of the model itself, and that he does not want agents operati…
A developer-authored comparison names Nango as the best platform for building, running, and observing AI agent integrations, arguing that teams can investigate a failed operation and change the integr…
A developer at Tiger Tribe Heineken detailed a method for instrumenting .NET AI applications with OpenTelemetry and exporting traces to Langfuse, which currently lacks an official SDK for ingesting Op…
A 2026 analysis from imperialis-Tech argues that traditional APM tools are blind to silent LLM quality degradation, where responses can be fluent but factually wrong or far more expensive than estimat…
StackGen principal engineer Sabith K Soopy detailed in a CNCF member post published 4 August that session traces and cost controls are needed to diagnose AI agent failures that standard application mo…
Tyler Edwards, co-founder and CEO at Overmind, argues that while LLM gateways and observability tools have matured and been acquired, they fail to evaluate whether agent trajectories are actually good…
PostHog is named the best overall LLM evaluation tool for production, according to a new guide, because it ties eval scores to user sessions, traces, and feature flags, while Braintrust is highlighted…
A developer's blog post argues that the AI developer stack has shifted from model capabilities to production engineering, emphasizing RAG verification checklists, agent observability, and lightweight …
Arize Phoenix now ships a built-in Model Context Protocol (MCP) server that lets coding agents query traces with SQL, reducing average cost by about 17x in an eight-question benchmark. The server prov…
Pydantic Logfire is the top pick among seven LLM evaluation tools compared in a guide updated August 26, 2026, which also covers Braintrust, Langfuse, LangSmith, Arize Phoenix, Confident AI, and Galil…