Together AI integration: Evaluate open models and optimize evaluation costs in Arize AX

Arize AX now supports Together AI as an AI provider for evaluations, letting teams run LLM-as-a-Judge evaluations, data labeling, and model comparisons with open models through Together AI's OpenAI-compatible API without configuring custom endpoints. The integration adds Together AI models to existing Arize AX evaluation workflows so teams can test whether smaller, less expensive models produce sufficiently reliable judgments compared with frontier models used for the same tasks. Together AI models are available in the Arize AX Playground and through custom LLM-as-a-Judge evaluators, configured under Settings → AI Providers.

As AI teams scale their evaluation workflows, the cost of running those evaluations can add up. LLM-as-a-Judge evaluations often rely on powerful frontier models, even when smaller, less expensive models may be sufficient for the task. The challenge is knowing where a lower-cost model can deliver reliable results without compromising evaluation quality. Arize AX now supports Together AI as an AI provider for evaluations https://arize.com/docs/ax/security-and-settings/integrations-playground/together-ai , making it easier to test open models https://arize.com/blog/llm-benchmarks-mmlu-codexglue-gsm8k/ , compare performance, and explore opportunities to reduce evaluation costs without configuring custom endpoints. Together AI provides access to a broad selection of open models through an OpenAI-compatible API. With this integration, teams can use supported Together AI models for LLM-as-a-Judge evaluations https://arize.com/guides/llm-as-a-judge/ , data labeling, and model comparisons in the Arize AX Playground. Build better agents with Arize Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today. Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability https://arize.com/phoenix?utm source=blog&utm medium=referral&utm campaign=ax-inline-cta&utm content=together-ai-integration-arize-ax-inline-cta-phoenix . Why evaluation costs matter Agent evaluations https://arize.com/ai-agents/agent-evaluation/ help teams measure and improve AI application quality. But running evaluations at scale introduces its own costs, particularly when every test requires an additional call to an expensive model. Consider a team using a frontier model to evaluate thousands of customer support responses for correctness. Each evaluation adds to the overall cost of operating the AI system. A smaller open model might be able to handle the same evaluation task at a lower cost. Before making the switch, however, the team needs to understand how closely its judgments align with the existing evaluator and whether it reliably identifies the same failures. With Together AI configured as an AI provider in Arize AX, teams can test open models against their existing evaluation criteria and compare results. If a lower-cost model delivers sufficiently reliable judgments, it may be a better fit for running evaluations at scale. For more on this approach, see our guide to LLM-as-a-Judge evaluations https://arize.com/guides/llm-as-a-judge/ . What you can do with Together AI in Arize AX The integration adds Together AI models to existing Arize AX evaluation workflows, giving teams more options to balance evaluation quality and cost. Run evaluations with open models. Use Together AI models to evaluate and label your data. Arize AX supports custom LLM-as-a-Judge evaluators https://arize.com/docs/ax/evaluate/create-evaluators that score application outputs against criteria you define. Compare model performance. Test different models and prompts in the Arize AX Playground https://arize.com/docs/ax/improve/prompt-playground to understand how they respond to the same inputs. Explore lower-cost evaluations. Test whether smaller, less expensive models can produce sufficiently reliable judgments for evaluation tasks currently handled by more expensive models. Skip custom endpoint setup. Connect Together AI directly to AX without building or maintaining a custom model integration. These workflows are useful for teams looking to reduce evaluation costs or explore open models as alternatives to proprietary options. Rather than selecting models based on benchmark scores or pricing alone, teams can evaluate performance against their own data and determine whether the potential savings justify a change. Get started with Together AI in Arize AX In Arize AX, navigate to Settings → AI Providers , select Together AI, and enter your Together AI API key. Once connected, you can use supported Together AI models to run evaluations, generate labels, and compare model performance in the Playground. For additional configuration details, see the Together AI provider integration documentation https://arize.com/docs/ax/security-and-settings/integrations-playground/together-ai . Find the right balance of evaluation quality and cost As evaluation workloads grow, choosing the right model can have a meaningful impact on AI spend. With Together AI in Arize AX, teams can test lower-cost open models, compare evaluation results, and make informed decisions about where to optimize their evaluation spend. And evaluation costs are only part of the picture. For teams looking to optimize AI spend in production, Arize AX also offers Cost Agent https://arize.com/blog/arize-ax-cost-agent/ , a managed agent that analyzes production traces to identify inefficient model choices, redundant calls, and other opportunities to reduce costs.