# MLflow 3.9: AI Assistant and MemAlign for Agent Developers > Source: > Published: 2026-10-25 05:09:09+00:00 MLflow 3.9 landed on October 6 with three features that address the exact pain points teams building production AI agents keep running into: a dashboard that surfaces quality regressions before users notice them, a judge optimizer that closes the gap between automated scoring and human judgment, and an AI assistant that actually reads your codebase instead of answering generic questions. None of these are flashy. All of them are useful. ## MemAlign: LLM Judges That Finally Agree With Humans The standout feature in 3.9 is MemAlign, a new algorithm for calibrating [LLM-as-a-judge evaluation](https://mlflow.org/llm-as-a-judge). If you’re running automated quality gates on your agent — and you should be — there’s a good chance your judges disagree with your human reviewers more than you’d like. MemAlign is built to fix that. The mechanism is straightforward: MemAlign learns from your past evaluation corrections and dynamically retrieves relevant examples at runtime. Think of it as RAG for judge calibration. When your judge sees a new trace to evaluate, it pulls in the most relevant past corrections as examples, nudging the judge toward your team’s actual preferences. This works with fewer than 50 labeled examples — a threshold most teams can reasonably hit. The numbers are notable. MLflow reports that MemAlign reduces judge errors by 74–89% across the most commonly misaligned evaluation dimensions, with overall agreement with human evaluators improving by 30–50%. For teams that have been frustrated watching automated evals say “looks great” while users report otherwise, this is worth testing immediately. ## The Overview Dashboard: Stop Building Your Own Monitoring Every team that ships an agent eventually builds some version of a quality monitoring dashboard. MLflow 3.9 ships that for you. The new “Overview” tab inside GenAI experiments provides three pre-built views with no configuration required: - **Usage:** Request volume, latency percentiles, error rates, and token consumption over time - **Quality:** Quality scores from your scorers and judges at a glance, so regressions surface immediately - **Tool Calls:** Breakdown of which tools your agent is calling, success rates, and where it fails This is the kind of thing that previously took an afternoon to wire up yourself. The dashboard won’t replace deep trace analysis, but it’s the right tool for the daily “is anything obviously broken?” check that every production team needs. ## The MLflow Assistant: AI Help That Knows Your Data MLflow 3.9 adds an [AI assistant](https://mlflow.org/docs/latest/genai/getting-started/try-assistant/) that runs on Claude Code, OpenAI Codex, or any AI Gateway or Ollama endpoint. The critical distinction from generic AI chat: this assistant has access to your actual MLflow data — your traces, experiment results, evaluation scores, and local codebase. In practice, that means instead of asking “how do I add tracing to my agent?” and getting a generic walkthrough, you can ask “why is my agent’s quality score dropping this week?” — and the assistant can look at your actual traces, spot the pattern, and point to specific spans. It can also instrument your project directly: adding tracing, evaluation hooks, and versioning with no manual copy-paste. There’s no additional cost if you already subscribe to Claude Code or Codex — MLflow provides the integration layer under Apache 2.0. An assistant with context of your actual evaluation data is a meaningfully different tool than a chatbot that doesn’t know what your agent does. ## What Else Shipped in MLflow 3.9 MLflow 3.9 also adds Python-logging-style severity levels to spans — DEBUG through CRITICAL — automatically assigned based on span type. LLM calls and tool invocations surface at INFO; internal plumbing steps stay at DEBUG. A new minimum log level slider in the redesigned trace explorer lets you filter out the noise without touching code, which matters when a single agent request generates dozens of spans. Distributed tracing now maintains trace continuity across microservices and sub-agent calls, giving teams a unified view even when requests fan out across services. ## Where MLflow Sits in a Crowded Field The [agent observability space is crowded in 2026](https://latitude.so/blog/best-ai-agent-observability-tools-2026-comparison). LangSmith owns teams deep in the LangChain ecosystem. Langfuse and Arize Phoenix serve teams that want something lightweight and fully self-hosted. MLflow’s position is the broadest open-source feature set for end-to-end lifecycle management — tracing, evaluation, model registry, and a built-in AI assistant — all under a permissive Apache 2.0 license. MemAlign and the assistant are genuinely differentiating; neither LangSmith nor Langfuse ships an equivalent today. To upgrade: `pip install mlflow==3.9.0`. The [full release notes](https://mlflow.org/releases/3.9.0/) and assistant setup guide are on the MLflow site.