{"slug": "together-ai-raises-800m-open-source-inference-just-got-serious", "title": "Together AI Raises $800M: Open-Source Inference Just Got Serious", "summary": "Together AI raised $800 million in Series C funding at an $8.3 billion valuation, reporting $1.15 billion in annual bookings and an inference engine achieving 500 tokens per second on DeepSeek-V3.1. The company offers over 200 open-source models via an OpenAI-compatible API, with pricing up to 20x cheaper than closed models, making open-source inference a cost-effective alternative for production workloads.", "body_md": "Together AI closed an $800 million Series C last week at an $8.3 billion valuation. The valuation is not the story. The story is $1.15 billion in annual bookings, 200+ open models behind a single OpenAI-compatible API, and an inference engine that hits 500 tokens per second on DeepSeek-V3.1. Open-source inference has crossed a line that closed-model providers can’t easily uncross. Developers running production workloads on GPT-5.x or Claude should run the math on switching.\n\n## The Cost Case Is Now Concrete\n\nThe 2025 argument for open models was qualitative: “the quality gap is closing.” The 2026 argument is quantitative: the gap is effectively closed for most workloads, and the cost difference is not incremental — it’s structural.\n\nQwen 3 235B runs on Together AI at $0.09 per million input tokens and $0.10 per million output tokens. GPT-5.4 Pro costs $30 input and $180 output. DeepSeek V4 posts 83.7% on [SWE-bench Verified](https://www.swebench.com/) while running roughly 34x cheaper per output token than GPT-5.5. Together AI’s reported figure — companies switching to open models achieve 6x to 20x lower costs — lines up with the public pricing data.\n\nDo the math on your current monthly API bill. Divide it by six. That’s your floor after switching to open models on Together, on the conservative end of their estimate.\n\n## Switching Is Two Lines of Code\n\nThe migration barrier is smaller than most developers assume. Together AI exposes an [OpenAI-compatible API](https://docs.together.ai/docs/openai-api-compatibility). You change two things: the API key and the base URL. Your existing OpenAI SDK calls — chat completions, streaming, function calling — run unchanged.\n\n``` python\nimport os\nimport openai\n\nclient = openai.OpenAI(\n    api_key=os.environ.get(\"TOGETHER_API_KEY\"),\n    base_url=\"https://api.together.ai/v1\",\n)\n\nresponse = client.chat.completions.create(\n    model=\"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\n\nprint(response.choices[0].message.content)\n```\n\nThe one adjustment: Together model IDs are namespaced. Instead of `gpt-4o`\n\n, you use `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`\n\n. The [Together model catalog](https://docs.together.ai/docs/serverless/models) lists every available model with its exact ID and per-token pricing.\n\n## ATLAS: The Inference Engine That Learns From Your Traffic\n\nSpeed was the last reasonable objection to open inference at scale. Together AI’s proprietary ATLAS engine (AdapTive-LeArning Speculative System) addresses it with an approach that’s more interesting than standard speculative decoding.\n\nATLAS runs two speculators simultaneously. A heavyweight static speculator, trained on a broad corpus, provides a reliable performance floor. A lightweight adaptive speculator continuously updates from live production traffic — it gets faster the longer it runs on your specific workload. A confidence-aware controller picks between them at each decoding step, using longer speculative chains when confident and shortening them when it detects drift.\n\nThe reported numbers: 400% faster than a [vLLM](https://github.com/vllm-project/vllm) baseline, 500 tokens per second on DeepSeek-V3.1, 460 TPS on Kimi-K2. These are company-reported figures; independent benchmarks have not been published yet. Run your own production workloads before making infrastructure commitments, but the architecture is sound and the mechanism is real.\n\n## Which Model to Use\n\nWith 200+ models in the catalog, the choice isn’t obvious. A practical starting point:\n\n**Coding tasks and agents:** DeepSeek V4 (83.7% SWE-bench Verified, ~$0.14/$0.28 per million tokens) or GLM-5 (77.8% SWE-bench, strong on agentic tool use)**Reasoning and math:** Qwen 3 235B (77.2% GPQA Diamond, cheapest capable open model at $0.09/$0.10)**General chat and summarization:** Llama 4 Maverick 17B — fast, cheap, strong quality for most tasks**Multimodal:** Llama 4 Scout or MiniMax M3 for vision and text together\n\nThe real advantage of 200+ models behind one API key is A/B testing. Swap models in your application config, run both on production traffic, and let your eval metrics decide. No vendor negotiation, no separate onboarding — just change the model string.\n\n## What the $800M Actually Means\n\nThe model war is effectively over for most production use cases. Open models are good enough — in many benchmarks, better. The next competitive dimension is who can run those models cheapest, fastest, and most reliably at scale. Together AI is making the same bet AWS made on cloud hosting: the winner won’t necessarily have the best software, but will have the best economics.\n\nWith 500+ MW of committed compute and a 50x capacity expansion target, Together AI is building for a world where inference costs continue to fall. That’s structurally good for developers. More infrastructure capacity creates pricing pressure across the entire inference market — including the closed-model providers.\n\nGet an API key at [api.together.ai](https://api.together.ai) and read the [Series C announcement](https://www.together.ai/blog/announcing-our-series-c) for Together AI’s own framing on where the infrastructure is headed. The [ATLAS technical blog post](https://www.together.ai/blog/adaptive-learning-speculator-system-atlas) goes deeper on the speculative decoding architecture if you want the implementation details.", "url": "https://wpnews.pro/news/together-ai-raises-800m-open-source-inference-just-got-serious", "canonical_source": "https://byteiota.com/together-ai-raises-800m-open-source-inference-just-got-serious/", "published_at": "2026-07-13 14:13:58+00:00", "updated_at": "2026-07-13 14:18:38.178128+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-startups", "ai-products"], "entities": ["Together AI", "DeepSeek", "Qwen", "Llama", "OpenAI", "GLM", "MiniMax", "vLLM"], "alternates": {"html": "https://wpnews.pro/news/together-ai-raises-800m-open-source-inference-just-got-serious", "markdown": "https://wpnews.pro/news/together-ai-raises-800m-open-source-inference-just-got-serious.md", "text": "https://wpnews.pro/news/together-ai-raises-800m-open-source-inference-just-got-serious.txt", "jsonld": "https://wpnews.pro/news/together-ai-raises-800m-open-source-inference-just-got-serious.jsonld"}}