# AMD on Agentic CPUs, and Why There's No One Size Fits All > Source: > Published: 2026-08-12 13:03:11+00:00 *Madhu Rangarajan spent 26 years in servers, at Dell, then Intel, then Ampere. Now he is Corporate VP at AMD for Compute and Enterprise AI Products, which covers the EPYC CPUs and every GPU that isn’t rack-scale. I sat down with him for the latest Bit by Bit conversation.* *His main argument: there is no one-size-fits-all for agentic infrastructure. The workloads under that label are too different from each other to land on one machine.* The industry has spent this year announcing agentic racks. Madhu thinks the label hides how different those machines actually are. Some of what gets sold as one is *“just dense liquid-cooled racks that take advantage of what was provisioned in the data center for GPUs already”. *But a gateway, a RAG database, a storage node, and a sandbox server each need different memory, storage, and CPU. What is AMD’s agentic Venice part is tuned for? With 256 cores at roughly 400 watts, his framing for it is **“maximizes threads per megawatt”**. When your power constraint is a fixed amount of MWs, threads is the number that decides how much agent capacity you get. The takeaways: **Agents turn tokens into CPU work.** Tool calls, sandboxes, verification, then a loop back around when the answer isn’t good enough. The GPU produces the tokens. The CPU runs the code those tokens describe.**CPU to GPU has reached about one to one**, and Madhu expects it to keep shifting toward CPUs. Every gain in GPU token output creates more tool calls and more sandbox runs, and those land on CPUs.**AMD now cuts the server market three ways.** Host nodes feeding GPUs, general purpose, and a new tier for agent orchestration and sandboxes.**Concurrency beats per-thread speed, for now.** Coding agents are coarse-grained and want every thread they can get.*“What’s happening today with agents may not be what’s happening in two years.”***Operators deploy as few SKUs as they can.** Every extra SKU is another load-balancing layer and more stranded capacity. One trick is to buy the 256-core part and run 128 cores for the turbo headroom if needed.**To date, agents mostly don’t run beside the GPUs.** They run on laptops, on-prem boxes, or general-purpose scale-out, decoupled from the nodes serving the LLM.**Tokenomics is a routing problem.** Not every task needs the PhD-grade model when an intern-grade one will do. AMD’s own IT is testing whether an MI350P on-prem can cut token spend by around 40%.**Openness wins the orchestration layer eventually.** Bespoke now, standardized later, with Kubernetes as the template. *Full transcript below (lightly edited for clarity).* **All right, hello everyone. We have a special guest today, Madhu Rangarajan from AMD. Welcome, Madhu. Tell us about your role at AMD.** **MR:** Hi. So, I’ve been at AMD for about three years now, and I run the compute and enterprise AI product team. So, that’s all of the EPYC CPUs and all of the GPUs that are not rack scale. So, think the PCIe card, think GPUs that are eight-way and so on. And I’ve been in the server industry for about 26 years now. Prior to this, I was at Ampere, prior to that at Intel, prior to that at Dell, predominantly in engineering and architecture roles, but more of a product role for the last few years. **Okay, awesome, interesting. So, perfect. You’re the perfect guest today because I want to talk CPUs for agentic AI, and then I also want to hit on enterprise AI, especially like sub rack scale. So, we’ll try to get to those both. So, let’s get into the fun stuff. So, let’s talk about CPUs. So, like maybe to lay the groundwork for listeners, at advancing AI recently, Lisa’s kind of called out these three different workloads in to explain sort of how the server CPU market is shaking out. Can you walk us through those and maybe tell us like why AMD sees the world this particular way?** **MR:** Yeah. So, stepping back a little bit, right? The transition from the chatbot era to the agentic era, basically drives a lot more CPUs because instead of being a chatbot just answering questions that you ask it, you now have an entire workflow where based on the token outputs from the GPUs, the real work is being done, whether it’s coding agents or any other sort of agents. And what that’s driving is a much more complex end-to-end workflow that involves a lot of CPU. And this is where I think we have been decomposing this workflow all the way from there’s a gateway. There’s a orchestration step. There is rag in order to inject enterprise data into your results. There’s the reasoning itself, which has been the interesting part for the last few years and all the attention has been there, but now making the reasoning do some useful work is driving CPU demand again. And after the reasoning, you’re going to have tool use because, well, at the end of the day, you want to do real work, and that means you have to run some tools and the tools tend to run on the CPUs. Then you have to do some verification to see, hey, did the agent do what I asked it to correctly? And then you loop around until the verification model says, yeah, that’s satisfactory. And then a user gets a response saying, okay, the agent did its thing. And at that point, the user might go, well, I don’t like the answer. Can you go do it again and this time fix this? So, you now have a series of steps that are CPU bound and some that are GPU bound running in a loop, and that’s what’s driving this crazy CPU demand we are seeing in the market right now. And the workload itself is not one size fits all. A lot of this workload is very familiar. All of the things that we know and love from the past like databases and CRM and ERP and all of the stuff that agents calling those workloads instead of humans calling those workloads. And in addition to that, you have some new emerging tiers like agentic orchestration and agentic tools running, where you now have agents creating code on the fly and executing those at extreme concurrency. So that’s creating a new tier of servers. So based on all of that analysis, we kind of broke it up into three large buckets, right? There is the host node itself, which is the CPU that connects up to the GPU, where you want a lot of single threaded performance, where you want a lot of memory bandwidth and IO bandwidth. And the goal is to make sure you maximize GPU throughput and the GPU is never left waiting for the CPU. Right? So that’s one tier of servers that we’ve been focused on. You’ve seen us have the Turin high frequency part serving that and we’re going to continue that with Venice and Verona. The second bucket of CPUs is the tried and true general purpose because agentic drives demand for databases and all and agents may run EDA tools and agents are going to do ERP. So all of the CPUs across enterprises and cloud that have been doing those workloads, you’re going to need a lot more of them now that agents are trying to make use of those CPUs. And the third bucket is this agentic orchestration and tools, where now you have swarms of agents trying to do a lot of work. So, you have to now think about it at a fleet scale level, where you want to balance how many agents you can run concurrently and how quickly any given agent is able to complete its task. And that’s what data center operators would be optimizing all day long and that’s that third tier that we introduced a new lower power Venice for in advancing AI recently. **Nice. Okay, this is helpful history. And I think like what I’m hearing you say if we kind of like go back in time was obviously like back in the cloud era, there’s these general purpose CPUs. You guys had different SKUs there to fit the different workload needs there, whether it’s a database or an API server, whatever. Then of course, fast forward to the early chatbot days 2023, 2022. All of a sudden there’s the need for the head node to feed the GPUs. And then of course with reasoning models and now agentic AI, it’s suddenly like, oh, we need this third bucket, the place where the agent’s going to live, do work, and a lot of that is CPUs and maybe ping-ponging back and forth with the GPU, but a lot is actually happening on those CPUs.** **MR:** Yeah, for example, right, the you ask it to you feed in a big piece of code and ask the agent to optimize it and make it more performant. You’re then going to eventually feed it into the GPU and the GPU is going to output a bunch of tokens, which is basically the modified code. And then you want to run that code and test it and see if it needs further modification. Or you could get more sophisticated and say, you know what, output five versions of this code for me, and then I’m going to run all five versions in a fan out manner and figure out which one works the best and then come back with the response. And all of that’s going to drive more CPU usage once the tokens are output. **Sure. Yes, that makes sense. So, okay, then if I’m thinking about like a one-word kind of priority for these different sockets, if you will, like what’s the priority for the host node versus the agent server versus general purpose?** **MR:** So, the general purpose, of course, it depends because general purpose is such a broad category, but in general, when you’re running general purpose at scale, you are trying to hit some kind of latency SLA while maximizing capacity, right? So, you don’t want to over index on just single threaded performance, but you don’t want to over index on throughput either because if you over index on throughput, then each individual user may be unhappy because their responses are slower. If you over index on single threaded performance, a single user might be happy, but then you end up with not being able to serve enough users, right? So there is always this fine balance that hyperscalers have been doing for years to get the exact right SKU in their data center, meeting the balancing latency and throughput. Right? So that’s kind of how I think about the general purpose. Of course, enterprises have different constraints, right? For them, they need to run their workload, but they may have constraints such as, I’m in a data center with only 10 kilowatts of rack power, or I don’t have any more space in my IT closet. So there are other constraints that make enterprises pick different SKUs, or there’s software licensing involved. I have a core-based licensing model, so therefore I may pick a lower core count SKU with a very high frequency. So, I think there isn’t a single answer in general purpose, but that’s also the reason why we have a lot of SKUs there. Now going to the host node, And you can see us advancing PCIe generations in the new Helios. You might have seen us have an actual coherent interconnect between the CPU and the GPU. So those are the areas we’re innovating there. And on the agentic, it’s going to be a mix and I think anyone that says we know exactly what agents are going to do now, will probably have a surprise because that’s going to change. But what we’re largely seeing today with the coding agent is they’re very coarse grain. They want extreme concurrency. They want as many threads as possible. They aren’t necessarily valuing per thread performance as much today because they want to run as many tools as possible and then they’re waiting for an LLM call, which happens in latencies that are pretty large. So, concurrency seems to be the primary metric today for most of the deployments we’re seeing, but at the same time, there could be more fine-grained agents. There could be some kind of agents and orchestration tiers where you need that more performance. But that’s the reason we have a portfolio, right? We have parts that have high single threaded performance and parts that have extreme concurrency because what’s happening today with agents may not be what’s happening in two years. Today, for example, agentic AI has a lot of Python code running. They create lots of Python code on the fly. That drives a certain kind of compute need. Two years from now, the agents could decide they’re going to run something else. They could come up with their own language that’s more efficient, right? And that’s going to need a different type of compute. So I think the best way to make sure you’re the best agentic CPU is to make sure you have a portfolio of CPUs that serve a fairly broad set of your usages effectively. **Sure. Yes, that makes sense. Oh, that’s helpful. So it feels like general purpose, it depends, but some sort of balance between throughput and latency. The host node, definitely it’s kind of what I hear you saying is speed. So things like clock speed, IPC, memory bandwidth, IO bandwidth and so on. And then, if what I felt like you were saying right now is for the agent server, it’s kind of it’s like about density and concurrency, but maybe, I felt like you were saying have an open mind because maybe they’ll maybe there’ll be some bifurcation where some of the workloads may need performance, some may So, do you foresee like a world where there’s like two different classes of parts for agentic AI?** **MR:** Yeah, I would never say never. I think data center operators try to keep as few SKUs as possible deployed at the data center level. And they usually try to pick the SKU that balances across multiple vectors the best. But if there is demand for any given workload that gets large enough to demand a certain set of CPU characteristics, they absolutely will demand unique will deploy unique infrastructure for that. **Sure. And I guess, but I so tell me a little bit more like why try to minimize the number of SKUs? Because I’m thinking out loud like, I guess you would have to orchestrate the different agentic workloads to the different SKUs. So that already is complex.** **MR:** Correct. So because think about it this way, right? You get a let’s say you get a thousand jobs that come in. If all of the servers you have are the exact same type of server, you just have to figure out which server is free and load balance your workload there. If you now have some servers with more memory and some servers with less memory and maybe your different storage configs, now you have the added layer of, well, this workload needs more storage, so it’s going to go here. This workload needs more memory, so it needs to go here. And then you add the third layer of, well, now different CPU SKUs optimized for different workloads. That’s another layer of how you load balance and orchestrate it. And the more you fragment things, the more you risk having stranded capacity. So if you’re operating at scale, you end up figuring out what’s the baseline demand for the different types of workloads and you can afford to kind of deploy different infrastructure for those different workloads. But until you can figure out what that baseline is, it’s a hard choice to make because you may make a bet on the wrong side. **Mhm. Mhm. I see. Yeah, that’s very interesting. Yeah, having stranded.** **MR:** And maybe just to add on to that, sorry. Yeah, go ahead. One more example I can think of is, for example, you could buy a 256 core Venice. And if there’s a workload that needs more memory bandwidth and a little higher single threaded performance, well, you could just keep 128 cores out of the 256 utilized and you can get that dynamic range to get that performance. So that’s one way to mitigate that is to say, well, I’ll deploy the higher core count CPUs, but if there are cases where I need to get a little more performance, a little more turbo frequency, that’s one way to say, you know what? I’ll modulate the utilization on that server to get more performance per core. So, there are different ways to handle this. Build a super set SKU and modulate the utilization, use two different SKUs. So, there are multiple ways I envision data center operators could is to could handle this situation. **Yeah, okay, that’s really interesting. I like the idea of sort of dynamically taking a particular SKU like that has lots of cores and kind of making it flex and behave differently for a different workload. Interesting. Okay, so now tell me, in the life cycle of hyperscalers and kind of the people on the bleeding edge who are needing agentic racks because they are orchestrating lots of agents, like where are we as far as like where are agents actually running? So I assume, and correct me if I’m wrong, that early on it was like, okay, we’re just getting into this agentic world. There’s no such thing as an agentic rack yet. Are they were they running those on the head node or were they actually not so concerned with like the IO latency and just sending them out to a CPU rack somewhere? And then how is that kind of changing? Like where are we today?** **MR:** Yeah, so I think there are a few different models. I think of how I use agents every day for myself. And at work, I use Claude Code a lot. And in that case, my agents are running right on my laptop. And the API calls are being made somewhere in the cloud to run the large frontier models to get tokens, right? So in that model, I got about four or five different agents running on my laptop. All of the API calls to the LLMs are happening in the cloud. So it’s that’s one model. The other model could be, well, the agents on my laptop, I can’t share them. So I want to share them with a bunch of my colleagues because I want that agent to be useful to all of them. So one option is, well, I could run it on a on-prem server in behind the firewall, well access controlled and everything that my colleagues can access. So now you start having those shared agents. Or you can run them on any virtual machine in the cloud anywhere, right? So that’s the other way to run it. So I think most of the these types of agents tend to run a little decoupled from the LLM inference call itself. On the other hand, I could have like a Strix Halo where I have one box where I’m doing the agents and I’m doing the LLM and I’m doing everything else. So I think multiple models exist. But for at scale deployments today, I’m predominantly seeing these agents run on largely general purpose servers in a scale out manner, quite decoupled from the LLM, the GPU nodes that are running the LLMs themselves because when you’re running, The GPU could be serving another LLM inference call and that’s a much better use of the GPU resources. So, at least my view is I’d be hard pressed to see why you would run it that way at scale unless there’s a very specific use case where it’s extremely tightly coupled. **Mm, okay. Yes. So, and I think you make a good point, which is there’s a lot of agentic use case today that’s happening that’s running on our laptop CPUs. So it’s not always even running in a VM on a server CPU. And so when it is, what I feel like I hear you saying is today it’s probably running on racks that are you mentioned scale out, but these would not be in the GPU data hall on the east-west network. They would be in a traditional data center somewhere.** **MR:** Yes. **Yeah. Gotcha. Okay. Yeah, that’s helpful. So then, maybe just extending this train of thought, when we hear about like, oh, so and so’s going to deploy a gigawatt of AI compute, of course a person can kind of like calculate how many head node CPUs that is. And we’ve talked about going from four to one ratio in the head node world to now it’s more closer to one. But how should people think about like, is there kind of like a rule of thumb of like, oh, a gigawatt’s getting deployed, so how many agentic CPUs that is? And I’m sure it’s it obviously is the answer is it depends. But should we think like people are buying agentic CPUs greenfield when they’re deploying a gigawatt?** **MR:** So, I think when they’re deploying a gigawatt, they have to think about the predominant use cases and the ratios, right? We’ve heard multiple ratios in the industry, 20 to 30% on compute nodes, maybe 70% on GPU nodes. We’ve heard ratios that are more even 50/50. So we’ve seen numbers that range all over the place. We’ve done some of our own internal testing with coding agents and how long they spend on the CPU nodes and how long they are actually spending on the LLM GPU nodes. So we got some coarse grain numbers from those as well. Now, some use cases stress the CPUs a lot more, some use cases stress the GPUs a lot more, so there’s going to be a range here. But generally we are seeing this one is to one becoming much more common based on those power ratios I just spoke about. But what we expect to happen over time is that ratio shifts even more in favor of the CPU because if you just plot what has been happening to let’s use SPECint_rate as a metric for CPU performance and plot what’s going on at SPECint_rate say over the last five years. And maybe even project forward what it looks like for the next five years. Project GPU tops and how many tokens GPUs put out over the last five years and project forward for the next five years, you’ll see that one line looks like this, the GPU line kind of goes significantly higher. And the more tokens that are put out by the GPU, the more CPU resource you need resources you need to do useful work with those tokens. So we actually think one is to one will climb to even more on the CPU side as we move forward in time driven by the sheer capability of the GPUs and all of the software innovations happening on quantization and so on. **Mm, that’s fascinating. So now, at the same time, you guys are always introducing like more and more cores per CPU, which maybe can kind of offset that a little bit. Like, but is there room to run with the number of cores that goes on a CPU or is like 256, 512, is that kind of going to be like the max?** **MR:** I have not seen that slowdown. We’ve always figured out how to put more cores and more importantly feed those cores, right? Because it’s not important it’s not enough to just have a lot of cores. You need enough memory bandwidth for those, you need enough fabric bandwidth, you need enough IO and networking. And you can see how quickly memory ecosystems been evolving, DDR5 from 6400 to 8,000 to MRDIMM 12800 and even faster speeds in the future. You can see the PCIe specs evolving fast with Gen 6 now and faster speeds coming soon. So we have managed to figure out how to squeeze in more cores and still feed the cores generation over generation and I envision that continues. **Okay, fascinating. Okay, so advancing AI, you guys had a lot of like, Venice SKUs that you launched and you were kind of tying it into this these SKUs could be good for host node versus agentic versus general purpose. Like, high level in case anyone didn’t watch it, like, what should we have in our head when we think AMD head node, do we think Venice HF and what do we think for each socket?** **MR:** Yeah. So, maybe I’ll talk about both Turin and Venice. Turin’s out now. Venice is going to be available in volume really soon. If I think head node host CPU, for Turin we had the 64 core 5 GHz high frequency part. And in Venice, we’ll have a 96 core 5 GHz high frequency part. For general purpose, big range of SKUs, right? Going all the way up to 192 cores, 500 watts in Turin, going up to 256 cores, 600 watts in Venice. And for these agentic tools and sandboxes, we have a power optimized 256 core, which is going to be closer to 400 watts instead of 600 watts. Optimized for sandbox concurrency. And again, the rationale behind that part is you’re going to have a lot of sandboxes provisioned on a server, but sandboxes also tend to have very spurty, bursty, branchy code running on it. So they’re not all going to be running like a throughput workload on a server at any given time. So, this 256 cores at around 400 watts maximizes how many threads you can provision in any given data center. And each thread still has the dynamic range to run as fast as possible when a given sandbox is active. So that’s kind of the rationale. It maximizes threads per megawatt. **Ah. Yeah. Maximizes threads per megawatt. That’s pretty that’s really interesting. Yeah, do you think, I mean, do you think there’s a point where people are talking about threads per megawatt as opposed to like CPUs per megawatt?** **MR:** I think they’re starting to get there sandboxes per megawatt, cores per megawatt because that is the unit of deployment and as you very well know, there’s a lot of power constraint in data centers. So if I have a 10 megawatt facility, how can I maximize how much sandbox capacity I can get there, right? So that is a critical metric and cores is a pretty good proxy for that as long as your cores perform well. **Yeah. All right, super interesting. Okay, so now let’s talk kind of competitive positioning. I for anyone who’s been following this closely knows there’s been a lot of companies in the space saying they’ve got an agentic CPU rack now. And I maybe to remind listeners, Nvidia has Vera, which was originally a head node and now they also have a standalone CPU rack. That was 88 cores and 176 threads. Arm launched their AGI CPU. I think it was up to 136 cores and 136 threads, so no SMT or multi-threading. And interestingly, which set us up nicely for this conversation, on Arm’s most recent earnings call, there was a sell side analyst who specifically used the AMD framing and he said head node,** **MR:** Yeah. So, the one SKU fits all, I think, anyone in this industry long enough to know that the customer environments are diverse, the workloads are diverse, the constraints are diverse, that I don’t necessarily buy into that view of the world. Sometimes you’re different driven to different SKUs based on workload, and sometimes you’re driven to different SKUs based on constraints. Sometimes you’re driven to SKUs based on, hey, I already plumbed this data center for liquid cooling because I have a bunch of racks and GPUs. And now I’d like to slide in a compute rack that also takes advantage of the liquid cooling, right? Some of what are being called agentic racks, in my view are just dense liquid cooled racks that take advantage of what was provisioned in the data center for GPUs already. I don’t think there’s anything that makes them agentic specifically because again, as I mentioned, agentic is a diverse set of steps. Many of those steps are like gateways running nginx, which are things that we know and love for many years now. And some of them are like these agentic tools and orchestration. So you’re going to have an entire data center that’s running a end-to-end agentic workflow, and you’re going to have multiple parts to save those to serve those workloads, right? Your rag databases are going to need something different from what your storage nodes are going to need, or are going to be something different from what your gateway needs and what the tools need, right? So, I think you will continue to see those types of diverse deployments, and even if you consolidate down to say two or three SKUs for your entire data center, you’ll still have multiple configurations in the rack when it comes to memory provisioning and how much storage you put in it and so on. **Yeah, that makes a lot of sense.** **MR:** Yeah, there’s no such thing as an agentic rack. **Yeah, well, no, and I think that’s very fair based on what we talked about, where you’ve outlined there are different workloads and they have different characteristics and therefore they need something different from the compute that they’re deployed on. And I actually like the way at the end saying there’s no such thing as an agentic rack, stripping away the marketing and just saying like, a dense liquid cooled rack, a dense air cooled rack, a maybe less dense but high performance liquid cooled rack that’s really what we’re talking about at the end of the day.** **MR:** Yes. Exactly. 100% agree with that. That’s how I think about the world. **Yeah, no, that is helpful. So thank you for sharing that. Okay, maybe one last question on the CPU front and agentic CPU front. A lot of hyperscalers have designed their own arm-based CPUs, Google Axion and Microsoft Cobalt and so forth. Traditionally those have been for general purpose and actually even a lot of them are for first-party workloads, but some are for third-party workloads as CSPs. Do you see those fitting into serving some of there’s obviously general purpose. Do you see them fitting into the agentic workloads at all?** **MR:** Yeah, I mean, they are general purpose CPUs and they can run a broad set of workloads, right? I mean, within the constraints of the Arm ecosystem, but if they wanted to run some Python code on a single thread, they could obviously do that really well. But our job is to make sure we run it even better than that and we give the concurrency and the PCO and the for what benefits that make it hard for them to choose their own CPU versus what we have to offer. So, that’s how I think about it is a competitive environment. We have other merchant silicon vendors that are really smart that we compete against, and then we have all of the hyperscalers who are building their own silicon with really smart people there, and we just have to do better than them and be generationally ahead, and that keeps justifying why our deployments continue to grow. **Nice. Okay, that’s a good answer. All right, let’s move into the enterprise AI stuff. So, can you walk us through listeners in case they forgot. Obviously, there’s a lot of talk about Helios lately, which is rack scale, but talk to us about the sub rack scale options.** **MR:** Yeah, and maybe we should talk, we can start talking about tokenomics and then we can talk about some other motivations like regulatory reasons and some other things like that. But if you think tokenomics, I think every corporation is starting to use AI far more extensively, and that has put a lot of pressure on token budgets, right? Every everyone I know is using a lot more tokens and the bills are growing dramatically. Now, the question that we have to ask is not every AI related task needs to go to a PhD scientist equivalent AI. Some of them could go to an intern equivalent AI, right? So, I think that’s really where tokenomics comes in is route the highest value requests to the highest cost AI resource, and things that can be done with smaller models in lower cost hardware, route them to that and create a more balanced environment in terms of token spend. I think that’s something that every single IT department across enterprises is looking at. Our own IT department is looking at that. We are doing experiments with the MI350P PCIe card, which can go into a broad set of on-prem enterprise servers, air cooled, slotted into a PCIe slot and get it going. And we are looking to see if we can serve save like 40% on token costs by load balancing the right tokens to that on-prem resource, and then the right set of token requests to the really large frontier models in the cloud. So, I think that’s what’s motivating customers to think about it is minimizing cloud token cost spend while still not compromising on the output of the AI. So, where you need the frontier models, you don’t want to run it on a smaller model because the quality of the output will be wrong and you’ll just run it five times to get the right answer. So, you don’t want to do that. You have to know what to balance where, and I think that’s where a lot of work is going on to have intelligent model routers and so on. **Yeah, okay, interesting. I mean, I can relate to this even from my own personal experience, which is I’ve got my $200 Claude plan and when I need Fable and I’m doing frontier model for coding, but then the stuff that I’m coding, I’m ultimately running and maybe have some lighter LLMs in it where I’m like, oh, dude, I’m totally not burning Fable tokens on some of this stuff. And in fact, and this is probably is actually very related to our conversation here. I have an AMD Ryzen AI Max, I think it’s 395 plus, it’s like the framework version, here in my And so even I have been thinking about like, how do I reduce my spend and orchestrate my tokens to running sort of from my own token generator, so free tokens if you will, versus paying for frontier tokens. And so you’re saying, a lot of enterprises are thinking the same way and MI350P is enough for their medium-sized models and maybe even the eight-way server nodes are enough to run even like 1 trillion plus parameter models.** **MR:** Yes. I think that’s what we’re seeing. You can do some medium to large reasoning on prem with those servers and then use the frontier models for the highest value stuff. And in some cases, they don’t even use those smaller GPUs on prem. They rent the smaller GPUs in the cloud. So that’s another model too. **Oh, there you go.** **MR:** Yeah. So I think both models are valid. You I think the whole world is was going hybrid even before this happened. And now it’s a much more complicated hybrid. **Sure, totally. So what about what are you seeing with customers like, are there concerns that to be able to run on premise or even on GPUs that you rent, you have to use an open source model, generally a little bit behind the frontier, versus the frontier models. Like is the open weight model that might be kind of six months behind a hindrance or not really?** **MR:** I think that’s where the model routers come in. If it’s a simple summarization task, I don’t think it’s necessarily as much of a hindrance. But if you’re asking it to do write the most complex piece of code, then it could be because it takes a few months for those models to catch up. So I think this is where how you route the models becomes important, right? Simpler use cases or use cases those models are good at, you route to those devices and the rest goes to the frontier models and that’s going to be a constant it’s a constantly shifting thing. **Yes. Indeed. Indeed. So and then what about desk side inference, whether it’s a laptop or a small kind of mini PC that are kind of gaining steam. Do you see those also just plugging into it sits behind the orchestration and or how are you thinking about it?** **MR:** Oh, I think you’re going to see all kinds of models, right? I’m setting up one of those at home to run some personal agents and I’m not necessarily and right now I’m still at a point where I don’t let the agents go unrestrained. So I would like to control the access policies and what they can access and so on. So I think there’s going to be a whole bunch of people doing things like that. I already spoke about running a bunch of agents on my laptop. I think that model will continue as well. I can envision a day when you even have a couple of smaller agents running on your phone. Right? So I think there is going to be agents deployed across multiple places and maybe even talking to each other and doing the right thing in the right place. I think this has always been the panacea of edge computing for a while is, hey, let’s have a far edge and a near edge and a regional data center and a central data center and the problem was always orchestrating it. But when AI can figure out how to do that, maybe that actually becomes more reality. **Sure, totally. Do you it feels like there’s probably opportunity for startups or companies to create this orchestration. I mean, is everyone doing their own bespoke orchestration or are there companies out there that can tackle this for customers?** **MR:** I think we are seeing some open source orchestration stacks pop up, but if I think about a lot of the biggest customers out there, it’s still a little bespoke. I think just like any rapidly evolving industry, it’ll start with things that are bespoke and people are modifying and then eventually become a little more standardized and open source. I think Kubernetes is a great example of that and I think we will see the same evolution here. I think eventually openness wins. **Sure, totally. And what about security? This just popped in my mind. We see the Anthropic’s got models that are very advanced and they’re very good at cyber security, either defense or offense. If I’m buying my own token generator, running stuff on prem, do I need to get more sophisticated from a security perspective in like now my on premise models need to be able to prevent someone’s other models from attacking them or something or is this just like a networking firewall issue?** **MR:** My view on AI and security is especially the most advanced AI models are extremely useful tools to make your entire system more secure, right? So I think you have to leverage regardless of what how you are splitting your workload between on prem GPUs and advanced frontier models. I think you’d use the most advanced models to find all of the security attack surfaces and close them. So I think of this as an useful tool to secure the system, no matter how you compose the system. **I like it. Yeah, that makes sense. AI is yet another tool to help you level up and do even better from a security perspective. Yes. Okay. So this has been great, Madhu. Let’s see, last question. I so I feel like what I’m hearing the theme for this whole conversation is AMD has a portfolio of compute for your different workloads, for your different size customers. We talked about CPUs and now we’ve talked about essentially sub rack scale GPUs. Anything else is that like the correct maybe vision and the way we should think about AMD going forward and oh, and is there anything that we’re under appreciating with that sort of portfolio and you’ve mentioned open before the open and the portfolio value.** **MR:** Yeah, and I think I should also add, right? There’s the CPUs, there’s the sub rack scale GPUs, there’s the Helios to run the most advanced frontier models. We have our Pensando DPUs to connect all of these together. Then we have the laptops and the we have the smaller AI boxes. So we got a portfolio that spans all of those and I think that’s important in this era because people are deploying this so many different ways. **Yes. So the TAM will continue to grow from the desktop to the edge, to the far edge, to the cloud and AMD has offerings for all of it.** **MR:** Yes. And we will continue on the server CPU side to make sure we have a portfolio because anyone that says they know exactly what agents are going to do say three years from now, well, I would love them to give me some stock advice as well. But I think having a portfolio that serves a broad set of workloads is what is going to win the win at the end of the day. **Yes, I agree with you. Yes, like you said earlier, maybe it runs Python today, maybe the agents decide to run rust or go tomorrow. Who knows? But** **MR:** Or they come up with their own new language too. **Right? Yes, there’s probably something even more efficient out there. There you go. Awesome. Well, this was a great conversation. We covered a lot of ground. Thank you for your time and we’ll have to stay in touch.** **MR:** Yeah, absolutely. Thank you so much.