Google has quietly dropped two results in the span of a week that should worry Nvidia a lot more than another splashy chip launch would have.
SemiAnalysis, the semiconductor research outfit whose InferenceX benchmark is popular for comparing AI chips on real-world economics, published the first-ever TPUv7 FP8 results on its Agentic Inference Benchmark (AgentX) — and TPUv7, Google’s latest custom AI chip, came out ahead of Nvidia’s flagship GB300 NVL72 rack system on performance-per-dollar across most of the curve. Not by a rounding error either. The first submission showed TPUv7 delivering up to 1.67x better performance per total cost of ownership (TCO) than GB300 NVL72 running at FP4 precision. A follow-up submission a few days later put the gap at 1.42x, while showing TPUv7 beating AMD’s MI355X by roughly 2.5x and Nvidia’s older GB200 NVL72 by up to 3.16x at medium-to-high interactivity.
For context, GB300 NVL72 is Nvidia’s current top-of-the-line inference system, and FP4 is the lower-precision mode Nvidia has leaned on heavily to post its best efficiency numbers. Google beating it required just four TPUv7 chips running its own TorchTPU vLLM stack, against Nvidia’s 72-GPU rack.
What’s Actually Being Measured #
The benchmark in question isn’t a synthetic throughput test — it’s built specifically around agentic workloads, which behave very differently from the simple chatbot queries most AI benchmarks were designed around. SemiAnalysis’s AgentX framework captures four things that define how modern AI agents actually run in production: sessions with dozens or hundreds of back-and-forth turns rather than a handful of exchanges, context that balloons quickly because of system prompts, tool definitions and accumulated history, heavy reuse of previous context as a conversation progresses linearly, and bursty spikes in memory usage when a session spins up multiple short-lived sub-agents.
This matters because it’s the workload every major AI lab is now building for. As coding assistants, research agents and enterprise automation tools get increasingly agentic — chaining together tool calls, spinning up sub-agents and holding onto long conversation histories — the cost of serving them is diverging sharply from the cost of serving a simple Q&A chatbot. A chip that wins on raw FLOPs but struggles with high-prefix-reuse, bursty, long-context workloads could end up being meaningfully more expensive to run in practice than one that’s architected around exactly that pattern. Google appears to be betting that TPUv7, paired with its own serving stack, is built for precisely this shift — and the numbers it just submitted back that up on a benchmark it doesn’t control.
The test was run against Alibaba’s Qwen3.5 397B model, a genuinely external, non-Google workload, which blunts the usual criticism that Google’s chip claims only hold up on its own models or its own infrastructure. SemiAnalysis even flagged the results as an “Official Preview,” noting the numbers could shift as validation continues — a normal caveat for a brand-new chip’s first public submission, but one worth keeping in mind before treating 1.67x as gospel.
The Asterisks Nvidia Will Point To #
Google’s advantage isn’t unconditional, and the data makes that reasonably clear. At low interactivity — where fewer tokens per second per user are demanded, typically batch-heavy, less latency-sensitive workloads — GB300 NVL72 FP4 still wins, and GB200 NVL72 FP4 held onto roughly a 1.2x edge in the same regime. Nvidia’s architecture also pairs its wins with aggregated versus disaggregated serving nuances: GB200’s comparisons in the second SemiAnalysis post were run using disaggregated serving, a more complex and resource-intensive setup, while TPUv7 posted its numbers using simpler aggregated serving — meaning Google may still have room to extract more performance once it adopts the same techniques Nvidia is already using.
So this isn’t “TPUv7 beats Nvidia everywhere.” It’s TPUv7 beating Nvidia’s best shipping rack-scale system on cost-efficiency across most of the pareto frontier for a specific, increasingly important class of workloads, while Nvidia retains an edge in low-interactivity scenarios for now.
Why Google Externalizing This Matters More Than The Numbers #
The more consequential story here isn’t the specific multiplier — benchmark numbers shift with every new submission and software optimization — it’s that Google is putting TPU performance in front of external, independent scrutiny at all. For most of the TPU’s decade-long existence, it was an internal-only tool, powering Search, YouTube recommendations and Google’s own AI products, with Nvidia GPUs remaining the default choice for virtually every other AI lab. That calculus has visibly started to shift: Google has been striking large external TPU commitments, and Anthropic in particular has leaned into using TPU capacity at scale.
Posting a genuine, third-party-validated agentic benchmark against Nvidia’s current flagship, on an external model, is Google explicitly inviting direct comparison on a battlefield Nvidia has dominated almost unchallenged. It’s a signal of confidence in both the TPUv7 silicon and the maturity of Google’s vLLM-based serving software — historically the weaker link relative to Nvidia’s deeply entrenched CUDA ecosystem.
For enterprises and AI labs sizing up inference infrastructure for agentic products — the kind of multi-turn, tool-using, sub-agent-spinning systems that Gemini 4 Argon-class models and their rivals are increasingly built around — this gives them a credible, apples-to-apples data point for the first time. It also hands every AI lab currently negotiating GPU pricing with Nvidia fresh leverage, regardless of whether they ever actually deploy a single TPU. Nvidia doesn’t need Google to win outright for this to cost it margin; it just needs TPUv7 to be a believable enough alternative that customers can use it as a bargaining chip. Whether this meaningfully dents Nvidia’s near-monopoly in AI accelerators will come down to things a single benchmark can’t settle — supply, software portability, enterprise support, and whether Google is willing to sell TPU capacity as aggressively and broadly as Nvidia sells GPUs. But for the first time, the performance-per-dollar argument against Nvidia’s best hardware is backed by an independent number, not just a Google slide.