OpenAI just published benchmark results for Jalapeño — its first custom AI inference chip — and the numbers are credible and significant. Compared to leading NVIDIA hardware (GB200 and GB300), Jalapeño delivers better throughput, better latency, and better power efficiency simultaneously. Not a tradeoff between them. All three at once.
"Jalapeño delivers both higher throughput and lower latency with one architecture, where existing hardware systems often have to make a tradeoff between the two."
That's the claim, and the benchmarks back it up.
Testing ran against the InferenceX public benchmark from SemiAnalysis across three models — GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. None of these are OpenAI-exclusive:
The chip is rated at 700W but ran at or below 550W on tested workloads. Compare that to the GB300 at 1,400W. Half the power, faster output.
Jalapeño isn't just a chip. It's the first output of OpenAI's strategy to co-design models, chips, memory, networking, and software together — rather than rent GPUs and adapt.
The architecture optimises specifically for language model inference phases: prefill (compute-heavy) and decode (memory-bandwidth-heavy) handled on the same balanced chip, minimising data movement between phases. For agents — where many inference calls chain together — this compounds. Every 3x latency improvement on a single call becomes more significant when you stack 10 calls in sequence.
Nine months from initial design to tapeout. OpenAI used earlier AI models to help design and verify circuits, then used Codex with GPT-Astra to bring three open-weight models to production performance in two months.
The headline stat: for selected attention and MoE blocks, AI-generated kernels ran 1.5–1.8x faster than human-expert implementations. That loop — design hardware so AI can program it, use AI to program it better — is the part of this announcement worth watching long-term.
Jalapeño deploys this year. Gen 2 is already in deep development. Gen 3 is taking shape.
Source: OpenAI Jalapeño announcement ✏️ Drafted with KewBot (AI), edited and approved by Drew.