{"slug": "together-ai-integration-evaluate-open-models-and-optimize-evaluation-costs-in-ax", "title": "Together AI integration: Evaluate open models and optimize evaluation costs in Arize AX", "summary": "Arize AX now supports Together AI as an AI provider for evaluations, letting teams run LLM-as-a-Judge evaluations, data labeling, and model comparisons with open models through Together AI's OpenAI-compatible API without configuring custom endpoints. The integration adds Together AI models to existing Arize AX evaluation workflows so teams can test whether smaller, less expensive models produce sufficiently reliable judgments compared with frontier models used for the same tasks. Together AI models are available in the Arize AX Playground and through custom LLM-as-a-Judge evaluators, configured under Settings → AI Providers.", "body_md": "As AI teams scale their evaluation workflows, the cost of running those evaluations can add up. LLM-as-a-Judge evaluations often rely on powerful frontier models, even when smaller, less expensive models may be sufficient for the task.\n\nThe challenge is knowing where a lower-cost model can deliver reliable results without compromising evaluation quality.\n\n**Arize AX now supports** [**Together AI as an AI provider for evaluations**](https://arize.com/docs/ax/security-and-settings/integrations-playground/together-ai), making it easier to test [open models](https://arize.com/blog/llm-benchmarks-mmlu-codexglue-gsm8k/), compare performance, and explore opportunities to reduce evaluation costs without configuring custom endpoints.\n\nTogether AI provides access to a broad selection of open models through an OpenAI-compatible API. With this integration, teams can use supported Together AI models for [LLM-as-a-Judge evaluations](https://arize.com/guides/llm-as-a-judge/), data labeling, and model comparisons in the Arize AX Playground.\n\n### Build better agents with Arize\n\nTrace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.\n\n**Prefer open source?**\n[Try Arize Phoenix for self-hosted, open source agent observability](https://arize.com/phoenix?utm_source=blog&utm_medium=referral&utm_campaign=ax-inline-cta&utm_content=together-ai-integration-arize-ax-inline-cta-phoenix).\n      \n\n## **Why evaluation costs matter**\n\n[Agent evaluations](https://arize.com/ai-agents/agent-evaluation/) help teams measure and improve AI application quality. But running evaluations at scale introduces its own costs, particularly when every test requires an additional call to an expensive model.\n\nConsider a team using a frontier model to evaluate thousands of customer support responses for correctness. Each evaluation adds to the overall cost of operating the AI system.\n\nA smaller open model might be able to handle the same evaluation task at a lower cost. Before making the switch, however, the team needs to understand how closely its judgments align with the existing evaluator and whether it reliably identifies the same failures.\n\nWith Together AI configured as an AI provider in Arize AX, teams can test open models against their existing evaluation criteria and compare results. If a lower-cost model delivers sufficiently reliable judgments, it may be a better fit for running evaluations at scale.\n\nFor more on this approach, see our  [guide to LLM-as-a-Judge evaluations](https://arize.com/guides/llm-as-a-judge/).\n\n## **What you can do with Together AI in Arize AX**\n\nThe integration adds Together AI models to existing Arize AX evaluation workflows, giving teams more options to balance evaluation quality and cost.\n\n### **Run evaluations with open models.**\n\nUse Together AI models to evaluate and label your data. Arize AX supports  [custom LLM-as-a-Judge evaluators](https://arize.com/docs/ax/evaluate/create-evaluators) that score application outputs against criteria you define.\n\n### **Compare model performance.**\n\nTest different models and prompts in the  [Arize AX Playground](https://arize.com/docs/ax/improve/prompt-playground) to understand how they respond to the same inputs.\n\n### **Explore lower-cost evaluations.**\n\nTest whether smaller, less expensive models can produce sufficiently reliable judgments for evaluation tasks currently handled by more expensive models.\n\n### **Skip custom endpoint setup.**\n\nConnect Together AI directly to AX without building or maintaining a custom model integration.\n\nThese workflows are useful for teams looking to reduce evaluation costs or explore open models as alternatives to proprietary options.\n\nRather than selecting models based on benchmark scores or pricing alone, teams can evaluate performance against their own data and determine whether the potential savings justify a change.\n\n## **Get started with Together AI in Arize AX**\n\nIn Arize AX, navigate to **Settings** → **AI Providers**, select Together AI, and enter your Together AI API key.\n\nOnce connected, you can use supported Together AI models to run evaluations, generate labels, and compare model performance in the Playground.\n\nFor additional configuration details, see the  [Together AI provider integration documentation](https://arize.com/docs/ax/security-and-settings/integrations-playground/together-ai).\n\n## **Find the right balance of evaluation quality and cost**\n\nAs evaluation workloads grow, choosing the right model can have a meaningful impact on AI spend.\n\nWith Together AI in Arize AX, teams can test lower-cost open models, compare evaluation results, and make informed decisions about where to optimize their evaluation spend.\n\nAnd evaluation costs are only part of the picture. For teams looking to optimize AI spend in production, Arize AX also offers  [Cost Agent](https://arize.com/blog/arize-ax-cost-agent/), a managed agent that analyzes production traces to identify inefficient model choices, redundant calls, and other opportunities to reduce costs.", "url": "https://wpnews.pro/news/together-ai-integration-evaluate-open-models-and-optimize-evaluation-costs-in-ax", "canonical_source": "https://arize.com/blog/together-ai-integration-arize-ax/", "published_at": "2026-10-12 18:27:10+00:00", "updated_at": "2026-10-12 18:49:07.639680+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "artificial-intelligence", "developer-tools"], "entities": ["Arize AX", "Together AI", "Arize Phoenix", "Arize AX Playground"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/together-ai-integration-evaluate-open-models-and-optimize-evaluation-costs-in-ax", "markdown": "https://wpnews.pro/news/together-ai-integration-evaluate-open-models-and-optimize-evaluation-costs-in-ax.md", "text": "https://wpnews.pro/news/together-ai-integration-evaluate-open-models-and-optimize-evaluation-costs-in-ax.txt", "jsonld": "https://wpnews.pro/news/together-ai-integration-evaluate-open-models-and-optimize-evaluation-costs-in-ax.jsonld"}}