AMD has acquired Taalas, the AI inference chip startup co-founded by former Tenstorrent CEO Ljubisa Bajic. Bajic, a former AMD executive, and the Taalas team will join AMD under Vamsi Boppana’s AI organization. Terms of the deal weren’t disclosed.
“We founded Taalas to rethink AI inference from the ground up by building the hardware around the model,” Bajic said in a statement. “Our Canada-based team has combined deep technical expertise with a willingness to challenge conventional approaches. Joining AMD will give us the scale, engineering resources, and global reach to accelerate our innovation.”
Founded in 2023 in Toronto, Canada, Taalas came out of stealth mode in February with a demo chip that could achieve more than 16,000 tokens per second per user on Llama3.1-8B. This is multiples of what competitors can achieve with current-gen chips, but there is a catch—Taalas’s first chip only runs Llama3.1-8B.
The startup has borrowed some ideas from the structured ASICs of the early 2000s; it essentially hardwires a model’s dataflow between compute elements and burns in the weights. Inference runs very fast, but almost all the programmability is removed, so only one model can be run.
View All Switching to a new model requires designing a new chip—in practice, usually closer to two masks, which represent the model weights and the dataflow, Bajic told EE Times earlier this year. Since the solution is entirely SRAM-based, bigger models would need more chips; DeepSeek-671B would need somewhere in the order of 30 separate tape-outs. Part of Taalas’s secret sauce is the tool flow it’s developed to enable quick design of these masks based on specific models; the company aims to tape out workload-specific chips in around two months.
AMD said it plans to integrate Taalas’s technology into system-level solutions with its Instinct GPUs.
At the end of last year, market leader Nvidia all but acquired AI chip startup Groq, which, like Taalas, could generate tokens faster than Nvidia GPUs but required dozens of chips to run large models. Nvidia plans to use Groq chips in systems alongside its GPUs, relying on Groq for the critical part of the decode stage of inference only. This disaggregated inference setup frees Groq chips from having to hold the entire model and KV cache while maintaining token-generation speed, which was a bottleneck for GPUs.
AMD recently announced it’s working with Cerebras on a similar disaggregated solution, where AMD GPUs handle the prefill portion of the workload and the Cerebras wafer-scale engine accelerates the decode portion. A Taalas SRAM-based chip could potentially accelerate decode alongside AMD GPUs in place of Cerebras, but the entire system would be controlled and supplied by AMD in a way more analogous to Nvidia-Groq.
Other options for AMD would include using Taalas chips for the entire inference workload on small models, perhaps for growing applications such as physical AI, where it currently offers its FPGA-based accelerators and SoCs. Edge applications often run smaller models due to power constraints, are more cost-sensitive, and don’t need to update models as frequently, which could be a good fit for Taalas’s structured-ASIC-like approach. Competitor Intel targets network infrastructure and defense applications with the structured ASIC technology it acquired with eASIC in 2018.
The limit for a single Taalas chip would seem to be around 8 billion parameters, though that’s dependent on how aggressively the model has been quantized.
The acquisition is still subject to closing conditions and regulatory approvals.
Read also:
[Why Qualcomm Bought an OpenAI Software Stack](https://www.eetimes.com/why-qualcomm-bought-an-open-ai-software-stack/)
[Microchip Acquires Edge AI Chip Startup Hailo](https://www.eetimes.com/microchip-acquires-edge-ai-chip-startup-hailo/)
[Warning Shots Fired as AMD Announces New Data Center GPUs](https://www.eetimes.com/warning-shots-fired-as-amd-announces-new-data-center-gpus-at-advancing-ai-event/)