AIArticle AMD bought Taalas, Nvidia gutted Groq, Cerebras went public — how to place inference workloads after the shakeout.
Mariana Souza Eighteen months ago, the fast-inference market had three interesting independents, each pushing a different answer to the same bottleneck. Today it has one. Nvidia paid $20 billion in late 2025 to license Groq's technology and hire away founder Jonathan Ross and his top engineers. Cerebras went public in May at a $66 billion valuation. And on August 6, AMD announced it's acquiring Taalas, the Toronto startup that etches model weights directly into silicon.
That's not three data points. That's the end of the standalone inference-ASIC era — and the ideas won even as the companies stopped being companies.
One bet, three doses #
Strip away the branding and all three companies made the same wager: LLM decoding is memory-bandwidth-bound, HBM is the bottleneck, so put the memory on the die. SRAM next to compute means no weight-fetching round trips, which means tokens come out faster per user than any GPU can manage. Where they differed was how much generality they were willing to burn for it.
Cerebras burned the least. Its wafer-scale engine is a general-purpose accelerator that happens to be the size of a dinner plate — the WSE-3 carries 44GB of on-die SRAM at a claimed 21 PB/s of bandwidth, and it runs whatever model you compile for it. Artificial Analysis clocked Cerebras serving GPT-OSS 120B at over 2,200 tokens per second, nearly 3x the fastest GPU cloud.
Groq burned more. The LPU is deterministic dataflow hardware — the compiler schedules every operation cycle-by-cycle in advance, with a couple hundred megabytes of SRAM per chip and no dynamic caches to introduce jitter. Any model runs, but only after the compiler has statically mapped it across a rack of chips. That determinism is what made Groq's latency so repeatable, and it's plausibly what Nvidia actually paid for.
Taalas burned everything. Its HC1 chip stores Llama 3.1 8B's weights in mask ROM — the model is literally part of the photolithography. On a trailing-edge TSMC 6nm process, an 815mm² die serves nearly 17,000 tokens per second per user, roughly 48x contemporary GPU serving. The chip cannot run anything else. Fine-tunes work via LoRA adapters held in SRAM, context length is configurable, and swapping the base model means re-spinning silicon — though Taalas claims only two metal layers change, which cuts the cost of a re-spin dramatically.
There's a clean gradient here: each step down trades flexibility for tokens per second per user, and each step made the company a sharper acquisition target than a platform.
Why the buyers moved #
The AMD deal is the most technically revealing of the three. AMD isn't buying Taalas to sell single-model appliance boxes. The stated plan is disaggregation inside Helios racks: Instinct GPUs handle prefill, Taalas ASICs handle decode. That mirrors where the whole serving stack has been heading — prefill is compute-bound and parallel, decode is bandwidth-bound and serial, and frameworks from Nvidia's Dynamo to vLLM have spent two years splitting them across different pools of hardware. Taalas is just the logical endpoint of that split: if decode is a separate tier anyway, why not make the decode tier a piece of hardware that is the model? A planned HC2 targets 20B parameters per chip, with pipeline parallelism across ~50 chips reaching trillion-parameter scale.
The Groq outcome is stranger and more instructive. Nvidia's $20 billion structure — a licensing deal plus mass hiring rather than an acquisition, which conveniently avoided merger review and drew scrutiny for exactly that reason — took the people and the IP but left the company standing. What remains of Groq raised $650 million in June and another $350 million in August at a $3.5 billion valuation, roughly half its 2025 peak, to pivot into a neocloud selling inference on Nvidia hardware. The company that built its brand on beating GPUs now resells them. If you want a one-sentence summary of who won the inference wars, that's it.
Cerebras is the counterfactual: the one that stayed independent, and notably the one that kept the most generality. It IPO'd off the back of real diversification — the S-1's alarming 87% revenue concentration in G42 gave way to Meta, Mistral, AWS, Notion, and Perplexity as customers. But a $66 billion valuation prices in that wafer-scale stays ahead of both Rubin-class GPUs and the SRAM-heavy decode ASICs its own success legitimized. That's a harder position in 2027 than it was in 2024.
What this means for your serving stack #
If you're consuming these chips through OpenAI-compatible APIs — which is how nearly everyone touches Cerebras and Groq today — the consolidation changes your risk calculus more than your code. Three practical takeaways: Per-user token speed is now a hardware-tier decision, not a provider quirk. Agentic workloads changed the economics: a coding agent running ten sequential tool-call rounds cares about tokens per second per user, not aggregate throughput, because latency compounds multiplicatively across the chain. That's the workload all this silicon exists for. If your agent loop spends most of its wall-clock waiting on decode, benchmark it on Cerebras or Groq before you buy more GPU capacity — the speedup is often the difference between an interactive tool and a batch job.
Don't build against a hardwired model's availability. The HC1 shipped in February 2026 baked around Llama 3.1 8B — a model that was already eighteen months old at tape-out. That gap is structural: silicon takes a year; frontier models turn over in months. AMD's disaggregated design hedges this (GPUs absorb the model churn, ASICs get re-spun for the stable winners), but if you're offered hardwired inference capacity, price in that the model is frozen and your fine-tuning path is LoRA-only.
Treat the abstraction layer as load-bearing. Groq's customers woke up one morning to find the company's founders worked at Nvidia. Nothing broke, because the product surface was an OpenAI-compatible endpoint. Keep it that way: route through an abstraction you control, measure cost per million tokens and per-user throughput per workload, and assume any specialty provider can be acquired, gutted, or re-platformed inside a year — because in the last twelve months, two of three were.
The architecture argument is settled: SRAM-heavy, specialized decode hardware is real, fast, and now owned by the same two companies that sell you GPUs. The startups proved the point. The incumbents collected it.
Sources & further reading #
AMD acquires AI chip startup Taalas to boost inference performance by etching models into silicon— theregister.com -
[AMD buys Taalas, startup that hardwires AI models into its silicon](https://www.cnbc.com/2026/08/06/amd-buys-taalas-startup-that-hardwires-ai-models-into-its-silicon.html)— cnbc.com -
[Groq raises $350M to fuel its pivot from AI chips to neocloud](https://techcrunch.com/2026/08/17/groq-raises-350m-to-fuel-its-pivot-from-ai-chips-to-neocloud/)— techcrunch.com -
AI chipmaker Groq confirms $650M raise, re-staffs after Nvidia's $20B not-acqui-hire deal— techcrunch.com - Cerebras' wafer-scale AI bet delivers blockbuster IPO— theregister.com - Taalas Launches Hardcore Chip With 'Insane' AI Inference Performance— forbes.com
Mariana Souza· Senior Editor Mariana covers the fast-moving world of machine learning and generative AI, with a particular focus on how these technologies are reshaping development workflows. When she isn't stress-testing the latest foundation models, she's usually at a local hackathon.
Discussion 0 #
No comments yet
Be the first to weigh in.