OpenAI unveiled hard benchmark numbers for its Jalapeño inference chip at Hot Chips 2026 last week: 1.7x to 3.6x lower end-to-end latency and 1.5x to 1.9x better work per watt than Nvidia’s GB300 — at half the power draw. The chip, co-developed with Broadcom and built on TSMC 3nm, went from initial RTL to tapeout in nine months with significant AI assistance. It is not for sale. But eventually, every API call developers make to OpenAI will run on one.
What the Benchmarks Actually Say #
OpenAI used SemiAnalysis’s public InferenceX benchmark across three models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Against Nvidia GB200 and GB300 reference systems, Jalapeño delivered:
- 1.5x to 1.9x more AI work per watt at peak throughput
- 1.7x to 3.6x lower end-to-end latency
- 2.1x to 4.1x better performance on highly interactive and agent workloads
The power comparison is the most striking number: Jalapeño runs at 700W; the Nvidia GB300 system it was benchmarked against runs at 1,400W. OpenAI is claiming better performance at half the power budget.
One important caveat: Jalapeño uses HBM4 memory; the Nvidia GB200 and GB300 use HBM3e. Memory bandwidth is a primary determinant of inference performance, so this is not a clean hardware-generation comparison. The more honest rival is Nvidia’s Rubin platform, which also ships with HBM4 — and those numbers have not been compared yet. These are also OpenAI’s own benchmarks. Independent validation has not appeared.
None of that makes the numbers fake. It makes them provisional. A 700W chip outperforming a 1,400W chip on LLM inference is notable regardless of the memory footnote.
Why Agent Workloads Are the Real Story #
The 2.1x to 4.1x figure on interactive workloads is the number most relevant to developers actively building today. Latency in a single API call feels like milliseconds, but latency in an agent chain compounds. A ten-step agent that shaves 3x off each call does not complete 3x faster — it completes faster still, because orchestration overhead also shrinks with the hardware. Jalapeño was benchmarked on multi-turn traffic with KV cache reuse and context lengths above one million tokens. That is the workload profile OpenAI expects to grow fastest.
The chip’s architecture supports this directly. Its 64 memory/core slices each pair with dedicated HBM4 to guarantee predictable latency. The NUMA-style design means it does not need to shuffle KV cache across peers the way some specialized ASICs do — a meaningful advantage as context windows grow.
What Developers Actually Get Right Now #
Nothing. Not directly.
OpenAI has been explicit: Jalapeño will not be sold externally or offered as a cloud instance. Internal demand is already so high that OpenAI cannot route capacity outward. The deployment schedule is small-volume rollout in late 2026, with Microsoft taking roughly 40% of initial production, and broader scale arriving in 2027. OpenAI has not committed to passing infrastructure savings to API pricing.
What developers might eventually see:
- Stable API pricing while competitors raise rates
- Higher rate limits at existing price tiers
- Faster response times on certain models as Jalapeño capacity grows
- Lower-cost long-context and agent workloads as inference economics improve
The honest timeline for any of this showing up in developer-facing pricing or rate limits is 2027 at the earliest. Do not reprice your product or renegotiate your OpenAI contracts based on Jalapeño benchmarks today.
OpenAI Designed This Chip With AI #
More than half of Jalapeño’s RTL was written in XLS, Google’s open-source hardware description language, with OpenAI’s own AI models searching for power, performance, and area improvements. Results against human baselines: 56% improvement on a BF16 multiplier, 21% on an FP4 dot-product block, 10% reduction in matrix unit area. Major changes were still landing on the day of RTL freeze.
The design went from initial RTL to silicon tapeout in nine months. The timeline at Hot Chips 2026 is documented: RTL started February 2025, taped out November 2025, first silicon arrived May 2026, Codex was running on it the same month. The loop is now closed: OpenAI’s models designed the silicon that will serve future OpenAI models.
What to Watch #
Nvidia Rubin comparison: When Rubin benchmarks on HBM4 land, the Jalapeño numbers will be tested properly. If the gap holds on equal memory generations, the efficiency story is real.Independent benchmarks: OpenAI used SemiAnalysis’s InferenceX methodology but ran it themselves. Third-party replication on equivalent hardware is the next credibility gate.API pricing signals:The Registernotes Jalapeño gives OpenAI the infrastructure margin to sustain its aggressive pricing trajectory — or extend it. Watch for rate limit or price announcements in Q1 2027.
Jalapeño is a serious inference chip with serious benchmark numbers and a serious asterisk. For now, developers keep building on the same API with the same pricing. The hardware underneath is getting better. When that shows up in the tools you actually use is still OpenAI’s call.