{"slug": "new-in-arize-ax-first-class-sessions-agent-as-a-judge-and-vision-evals", "title": "New in Arize AX: first-class sessions, Agent-as-a-Judge, and vision evals", "summary": "Arize AI shipped sessions as a first-class unit of work in Arize AX between August 6 and September 18, letting users annotate, queue, preview, and filter whole conversations rather than individual spans. The release also makes Agent-as-a-Judge available on every plan, adds Claude Fable 5.1, GPT-6 Astra, Gemini 3.7 Flash, and Mistral models on AWS Bedrock to Playground and evaluators, and enables LLM-as-a-judge templates to reference image columns for vision evals. Dataset population from traces now writes every matching record instead of applying a sampling budget and appends 2-7x faster on every account tier.", "body_md": "Here’s everything that landed in [Arize AX](https://arize.com/products/ax/) between August 6 and September 18. The main point: sessions are now a first-class unit of work. You can annotate a whole conversation, queue it for review, preview it in an [evaluator](https://arize.com/glossary/evaluations/), and ask [Alyx](https://arize.com/products/alyx/) to filter for it in plain language. Everything else is below, and [you can explore our full Changelog to see everything we’re shipping](https://arize.com/docs/ax/release-notes).\n\nOh, and since it’s Jev week: [you can read about our latest thoughts around if decision models can replace LLM judges on our blog](https://arize.com/blog/typesafe-jev-llm-judge/?utm_source=hs_email&utm_medium=email&_hsenc=p2ANqtz-86CIZRCWMERIs6O74KYIMCuhinRObsaEPaoaHGpU_2d1eNU5N1_w0ANCiQ3ppjMPXpOw4C) (and expect more soon).\n\nLet’s jump in.\n\n## Work with whole sessions, not just spans\n\nSessions now behave like a single object you can act on. Annotate one from its detail view and the label is stored as `session_annotation.<name>`, ready to filter on anywhere. Send it to an [annotation queue](https://arize.com/glossary/human-evaluation/) from the Sessions table or the API, and whoever picks it up reviews the whole conversation, not a stray span. Evaluator setup previews results at the session level too, and Alyx builds the filter for you from a plain-language request on the Sessions tab.\n\n## Ask Alyx what Signal found\n\nSkip the re-investigation. Alyx can now read the issues [Signal](https://arize.com/blog/from-signal-to-pr/) has recorded for your space. You can ask what it found across a project, or attach a single issue from the `@` menu for a deep dive. And when you ask Alyx to debug a trace Signal already reviewed, it links you to the matching issue instead of redoing the work.\n\n## Agent-as-a-Judge is on every plan\n\nAn [agentic judge](https://arize.com/blog/agent-as-a-judge-evaluate-agents-with-agents/) inspects [traces](https://arize.com/glossary/trace/), finds the relevant spans, and labels issues, adapting its criteria to the failures that show up in production. It used to be Enterprise-only; now it’s on every plan. Spin one up from the New Evaluator menu in the evaluator gallery.\n\n## Claude Fable 5.1, GPT-6, Gemini 3.7 Flash, and Mistral on Bedrock\n\nMore models to reach for in Playground and evaluators: `claude-fable-5-1` (Anthropic, Bedrock, Vertex AI), OpenAI’s `gpt-6-astra`, and `gemini-3.7-flash`. Mistral models hosted on AWS Bedrock, including Mistral Large 3 and Pixtral Large, are grouped under Mistral AI in the model picker.\n\n## List a project’s traces from the Python SDK\n\n`client.traces.list()` returns each trace with its full span list and roll-up metadata. Pass a project by name or ID, narrow with a time range, a filter expression, and a limit, and page with the cursor the server returns. Beta.\n\n## Images and audio render everywhere, and judges can see them\n\nUploaded media now resolves to viewable content in trace and span views, datasets, experiment comparisons, Playground, and annotation queues, across Google Cloud, AWS, and Azure storage. Better yet, [LLM-as-a-judge](https://arize.com/guides/llm-as-a-judge/) templates can reference an image column, so your vision evals score the actual picture instead of a text stand-in.\n\n## Populate datasets with every matching record, faster\n\nNo more silently dropped rows. Populating a [dataset](https://arize.com/glossary/evaluation-dataset/) from traces now writes every matching record instead of applying a sampling budget that could drop rows, and appends complete 2-7x faster. This applies on every account tier.\n\n## More this month\n\n- Remote evaluator runs emit a trace per evaluated row when you turn on `Enable Tracing` , so retries and latency are inspectable.\n- Edit a managed agent automation’s prompt in place; the schedule, integration, repository, and run history stay put.\n- Ask Alyx for a line chart split by column value, up to 30 values, with new values picked up automatically.\n- Route [OTLP/HTTP](https://arize.com/blog/arize-ax-opentelemetry-genai-semantic-conventions/) traces to a project with an`x-project-name` header, for tools that cannot set a resource attribute.\n- Create standalone experiments from the CLI with `ax experiments create --space` and review runs in a dedicated table.\n- Pin `metric.*` span attributes to the trace slideover header.\n- Select time ranges past your retention window; rows outside it show where accessible data ends.\n- Alyx streams reasoning for GPT models in thinking mode and can run on `gpt-5.6-terra` .\n- Detach tags from monitors, evaluators, and prompts over REST; cap REST connections with `connection_pool_maxsize` .\n- Fixes across Alyx, dashboards, evaluators, Playground, cost tracking, and the SDKs.\n\n## Heads up\n\nThe standalone Managed Agents page is retired. Enable an AI provider for managed agents from the Managed Agents checkbox in the Integrations table, or the inline Enable action in the provider picker.", "url": "https://wpnews.pro/news/new-in-arize-ax-first-class-sessions-agent-as-a-judge-and-vision-evals", "canonical_source": "https://arize.com/blog/new-in-arize-ax-september-2026-updates/", "published_at": "2026-09-22 14:00:21+00:00", "updated_at": "2026-09-22 15:26:37.286490+00:00", "lang": "en", "topics": ["ai-products", "ai-agents", "large-language-models", "ai-tools", "mlops"], "entities": ["Arize AI", "Arize AX", "Alyx", "Signal", "Anthropic", "OpenAI", "Google", "Mistral AI"], "alternates": {"html": "https://wpnews.pro/news/new-in-arize-ax-first-class-sessions-agent-as-a-judge-and-vision-evals", "markdown": "https://wpnews.pro/news/new-in-arize-ax-first-class-sessions-agent-as-a-judge-and-vision-evals.md", "text": "https://wpnews.pro/news/new-in-arize-ax-first-class-sessions-agent-as-a-judge-and-vision-evals.txt", "jsonld": "https://wpnews.pro/news/new-in-arize-ax-first-class-sessions-agent-as-a-judge-and-vision-evals.jsonld"}}