This week, AMD held what is now an annual Advancing AI event in Silicon Valley, and there is much talk about money, roadmaps, and product deep dives.
Like AMD chief executive officer Lisa Su, who has without a doubt not only saved the chip maker from some self-destructive tendencies but has broadened and deepened its lineup of hard and soft wares to make it an absolutely credible alternative to AI industry juggernaut Nvidia and former CPU rival Intel, we will start off our coverage of the event with an update on total addressable markets and the forces that drive them.
In the coming days, we will uncover the details of the “Altair” MI400 architecture, specifically for the MI455X variant that makes its debut in AMD’s “Helios” rackscale server designs later this year through the many OEM and ODM partners that have all jumped on the AMD bandwagon. We will also take a look at new details about the forthcoming “Venice” Epyc 9006 processors also expected later this year. And then we will take a look at the roadmaps that AMD gave some sneak peeks of, which will help them capture AMD’s fair share of the opportunities.
So, without further ado, let’s talk money.
In her opening keynote address, Su said that AI training workloads are increasing their compute needs by a factor of 5X every year since 2020, and that there is no signs of this abating. That drives a certain amount of compute for the AI model builders, large and small. But the big driver, according to Su, is the inevitable and much-anticipated shift from AI training at a relative handful of companies to AI inference with companies either renting GenAI models through APIs or buying AI systems to run their own inference with licensed or open source models.
That latter shift has been long anticipated, and the projections from several years ago, before GenAI hit, showed that inference would eventually drive maybe 3X or 4X the compute (and therefore revenues) as training. We are not quite there yet, but we are apparently on our way, according to Su, who proclaimed at last year in the AI accelerator market, inference and training were split about 50-50, but this year she expects for it to look more like 60 percent for inference and 40 percent for training. It was about 40-60 back in 2024:
The question we all want to know is when will it be 2:1, 3:1, or 4:1 for inference, or even 10:1. When that happens, that will mean that AI inference – and specially generative AI inference rather than traditional machine learning – really is a production workload for the enterprises as well as hyperscalers, cloud builders, and AI model builders. (By the way, if you tear apart some of the charts Su presented and reconstitute the data underlying those charts, you can derive a revenue ratio between AI training and inference, and even between GPU/XPU training and agentic on CPU plus
The curve for token consumption worldwide that Su showed off came from the State of the AI Economy report from Exponential View, and it shows the
monthlyincreases in the number of tokens that are consumed or generated by GenAI models from January 2024 through February of this year:
Exponential View estimated that an astounding 35 quadrillion tokens per month were chewed up or spat out in February, and given this exponential curve it should be well into 50 quadrillion tokens per month here as we finish July in eight days. All of the AI model builders and the hyperscalers and cloud builders that are serving them as well as themselves in the GenAI boom are hoping like hell that this curve keeps rising, therefore justifying the hundreds of billions of dollars in AI system capacity they are building out this year. The only way the curve can keep growing is to have more machines, but there is always the inverse of the Field of Dreams as a possibility: You build it, and they don’t come.
So far, demand for AI processing is exceeding supply, and that is why pricing for all key components –CPUs, GPUs, DRAM memory, flash, switch ASICs, optical components – are all rising along a similar curve.
Su’s job as the top executive at AMD is to be optimistic about the future in terms of such exponentials and about the affect that HPC in general and GenAI in particular will have on the world, and the total addressable markets that AMD is chasing and divulging at the Advancing AI event reflect this.
“When I think about where AI is today, the biggest change that we see is we are no longer talking about what might be possible,” Su said at the end if several hours of presentations by the top brass at AMD. “We are actually seeing how AI can have real and significant impact across every industry and every part of our personal lives. I have spent my entire career in tech believing that high performance computing can make the world an incredibly better place, and I have never ever believed that more than I do today. At AMD, what we are focused on is building the technology, the roadmaps, and the partnerships. And there has never been a more exciting moment than today for our 30,000 plus engineers. This is really the next phase of AI. This is where we give AI the opportunity, with all the tools and all the capabilities, to really bring meaningful real-world impact. And I can tell you, I could not be more excited to build all of that together with you, our ecosystem.”
Like Su, I believe that AI will have a huge impact – has already had a huge impact, in fact. I even agree on the size of the vectors of that impact. But I almost certainly differ in the direction of those vectors and the non-uniform distribution of AI forces across the cultures and economies of the world. Secondary and tertiary impacts will matter, and perhaps more than primary ones that are more obvious.
Outlining those and predicting impacts on cultures is not the job of The Next Platform, but let me assure you, we think about this day and night as many of you do. As we did for traditional ModSim HPC workloads or early machine learning as they emerged in the market. Our job here is to examine new technologies and to ascertain or predict their effects on the IT market. No more, no less.
Here is the revised datacenter AI accelerator TAM that Su divulged yesterday:
It goes out two more years than the June 2025 forecast, and the expectation is that AI training and AI inference together will drive more than a 45 percent compound annual growth rate between 2025 and 2030 to hit more than $1.4 trillion dollars. These are AMD’s own internal projections, and Su thankfully gave us the ratio of the TAM for AI training split from AI inference. We only have the endpoints in terms of data, but we can count pixels to derive the data for 2026 through 2029 in between those endpoints.
Here is how the datacenter CPU TAM will play out according to AMD, which is important because a lot of agentic AI workloads – sandboxes where models are creating code or performing other kinds of tasks, running a lot of Python code as they are hooked to GPU or XPU inference engines – are going to be running on CPUs, not just GPUs or XPUs. Take a gander:
Once again, Su thankfully broke the expected datacenter CPU revenue streams across all vendors around the world down into three buckets: The general purpose CPUs that drive back office applications and databases as well as Web infrastructure and various kinds of data analytics; the CPUs that are used as host processors for AI cluster nodes; and these new agentic AI clusters running those sandboxes full of churning Python code.
This is a very handy breakdown, and thank you, Lisa.
These agentic AI server clusters, which have slightly different performance and capacity requirements from the other two types of machines, are going to drive a revitalization in the server CPU segment of IT spending, and growth and revenue levels that we have never, ever seen before. Provided these golden TAM expectations actually pan out.
Su gave us one more chart we can work with, which is the total addressable market for compute that AMD is chasing across the datacenter, client devices, and embedded devices:
The total compute TAM across these categories, according to AMD, is going to grow at around a 40 percent compound annual growth rate between 2025 and 2030 to reach more than $2 trillion, and as you can see, the datacenter is going to be an even larger share of that donut than it was last year.
OK, with all of that, I can reconstitute the data Su presented, fill in some gaps, and then do some analysis. Which is what passes for fun for my brain. . . . And remember: This is the TAM for the revenue from compute engine chips, not for complete systems or devices with their DRAM, flash, interconnect, screens, keyboard, and chasses added in.
Here is a pretty little table that brings it all together and fills in those gaps, which are shown in bold red italics as usual:
The thing about compound annual growth rates that are a little misleading is that they just calculate the straight line average growth between two endpoints. You can’t really see what happens in between. But given the actual datacenter AI accelerator and datacenter CPU spending for all six years allows us to better fill in the broader data because that broader data is affected by these two categories greatly and must be larger than these two categories added together.
I think that the worldwide compute TAM is going to grow faster than that 40 percent CAGR in 2026 and 2027, and slow down towards it in 2028, and then slow radically in 2029 and 2030. Datacenter growth is going to be even higher in the next two years, and will also peter out.
This pattern, with a few wiggles, will also be expressed in the datacenter AI training accelerator TAM, as you can see from the actual AMD data, but AI accelerators aimed at inference are going to be the highest growth area in the datacenter.
Some things immediately jump out to me in this table. First, the total amount of datacenter compute is just enormous, at $6.45 trillion across the six years from 2025 through 2030. There was a time when client device compute was much larger than datacenter compute, but forget that. Datacenter is 8.3X larger in aggregate across those years than embedded and client, and will be 10X in 2030 alone. The gap is getting bigger, even with AI PCs and such.
The other thing to note is how even with a resurgence in CPU spending driven by GenAI workloads, spending on AI accelerators is going to far exceed that on CPUs. The ratio of physical devices might be 1:1 in the long run, but the money spent on datacenter AI GPUs and XPUs is going to exceed spending on datacenter CPUs, according to AMD’s analysis. The ratio of datacenter GPUs and XPUs to datacenter CPUs was 7.9X in 2025, and is expected to be 6.8X in 2026, but it settles down to around 6.4X for the next couple of years and averages 6.5X over the six years in the TAM figures supplied by AMD.
On other interesting thing: The agentic AI CPU TAM will be almost as large as the general purpose CPU TAM in 2026, and was zero last year. By 2030, that agentic AI CPU segment will be 5.2X larger than that general purpose CPU segment in the datacenter. Crazy, isn’t it?
There may be nothing that can properly be called agentic AI CPUs – they just have a different mix of features, but no special features – but people are sure going to buy a lot of CPUs for these agentic sandboxes if AMD is right.
Now here is where it gets interesting. If you want to try to extract AI training from AI inference across GPUs, XPUs, and CPUs, Su’s data lets you do that. If you take the AI training GPUs and XPUs and add it to the share of AI hosts CPUs that are in training systems (assuming the ratio for CPUs is the same as for GPUs), you get a cumulative spending of $1.42 trillion for the six years for the training side of datacenter compute. And if you take the inference GPUs and XPUs, add in their host CPU processors, and then add in all of the CPUs deployed in those agentic AI sandbox clusters, you get $3.82 trillion. So the ratio of AI inference compute across those six years compared to AI training compute is 2.7X. It was 1:1 in 2025, is expected to be 1.6X in 2026, and will grow to 3.2X by 2030.
That 3:1 ratio for inference to training is exactly what many expected for the earlier AI machine learning era. Now, GPUs are a lot more expensive than CPUs, and XPUs are somewhere in the middle, thanks to that HBM stacked memory and the interposers it requires to hook to compute engines, so the ratio of GPU/XPU devices to CPU devices in AI systems could end up being close to 1:1 averaged across all AI workloads in all datacenters in the world. Time will tell on that.
One last thing: AMD has not updated its own revenue expectations, and we have to believe with another $600 billion in expected TAM in 2030 and dragging upwards over the years between 2025 and 2030, AMD’s revenue forecast for 2026 may follow in kind, and maybe even be given a CAGR out to 2030 considering the visibility into customer spending that AMD certainly has as demand is much larger than supply for everything it sells into the datacenter.