OpenAI Jalapeno Chip: What It Means for Developers

OpenAI unveiled Jalapeno, its first custom inference chip co-developed with Broadcom, at the Hot Chips conference, reporting independent benchmarks showing it outperforms Nvidia's GB200 and GB300 systems by up to 1.9x in throughput per watt and cuts latency by up to 3.6x. The chip, built on TSMC's 3nm N3P process with 216 GiB of HBM4 memory and 13.4 PFLOPS of MXFP4 compute, is not for sale and will begin deploying in OpenAI's infrastructure at the end of 2026, with API calls expected to run on it by 2027. OpenAI aims to reduce inference costs by roughly 50% compared to Nvidia GPU infrastructure.

OpenAI stood at the Hot Chips conference at Stanford last Tuesday and published something the chip industry didn’t expect: independent benchmarks showing its first custom inference chip beats Nvidia’s flagship GB200 and GB300 systems on throughput per watt by up to 1.9x and cuts latency by up to 3.6x. The chip is called Jalapeno. It went from initial design to tapeout in nine months — reportedly an industry record for this class of hardware. It is not for sale. And your API calls probably won’t run on it until sometime in 2027. What Jalapeno Actually Is Jalapeno is a purpose-built inference ASIC — not a general-purpose GPU that happens to run LLMs, but a chip designed from scratch to run transformer inference and nothing else. OpenAI co-developed it with Broadcom https://openai.com/index/openai-broadcom-jalapeno-inference-chip/ . It’s manufactured on TSMC’s 3nm N3P process, the same node Nvidia uses for Vera Rubin. The package pairs one reticle-sized compute die with six HBM4 memory stacks from Samsung, delivering 216 GiB of memory at 15.4 TB/s of bandwidth. Compute sits at 13.4 PFLOPS of MXFP4 matrix operations — the low-precision floating-point format that inference workloads tolerate well and training does not. The whole chip draws 700W under rated conditions and stays below 550W in measured workloads. Nvidia’s GB200 is rated at 1,200W. The GB300 hits 1,400W. Purpose-built silicon has obvious efficiency advantages over general-purpose GPUs designed to handle every conceivable workload. One genuinely interesting detail: OpenAI used its own models to accelerate chip design — cutting the RTL-to-tapeout timeline to nine months, which Broadcom and OpenAI describe as the fastest cycle for a high-performance ASIC of this scale. The Benchmark Numbers OpenAI tested Jalapeno against GB200 and GB300 using InferenceX, a public benchmark from SemiAnalysis https://techcrunch.com/2026/08/25/openais-jalapeno-chip-is-built-for-fast-inference-at-scale-benchmarks-show/ . Three models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. SemiAnalysis engineers independently verified the results — which matters, because our chip is faster announcements without external verification are standard marketing. This one had a third party in the room. Results across the three models: Peak throughput per kilowatt: 1.5x to 1.9x better than Nvidia End-to-end latency: 1.7x to 3.6x lower Interactive workloads the chatty, low-latency kind agents generate : 2.1x to 4.1x faster The honest caveat: wall time — how long it takes to process a specific request start to finish — is about 1.7x slower than Nvidia’s systems. Jalapeno wins on efficiency per watt and on interactive throughput, not raw speed. For inference at scale, efficiency per watt is what drives cost. For real-time agent responses, interactive throughput is what drives user experience. Wall time matters less for these use cases than for training runs. What This Actually Changes for You Practically speaking, right now: nothing. Jalapeno begins deploying in OpenAI’s infrastructure at the end of 2026 in very small volumes, per Richard Ho, OpenAI’s head of hardware. There is no SDK. There is no capacity reservation. There is no way to request that your API call runs on Jalapeno. OpenAI says it has so much internal demand that it cannot imagine selling the chip externally. What you might eventually see is lower API pricing. The official benchmark release https://openai.com/index/jalapeno-first-results/ targets roughly 50% lower inference cost compared to Nvidia GPU infrastructure. OpenAI already cut GPT-5.6-luna pricing by 80% in July. If Jalapeno ramps meaningfully in 2027, another round of cuts becomes financially easier to absorb. The latency improvement is the more interesting near-term story. Agents are especially sensitive to latency because every tool call compounds. A 2 to 4x improvement in interactive inference latency doesn’t just make chat feel snappier — it meaningfully changes what’s practical to build with multi-step agents. If response times drop from 2 seconds to under 1 second per step, workflows with ten-step chains go from 20 seconds to under 10. That’s a product difference, not just a performance metric. The Part That Doesn’t Get Covered Jalapeno is exclusively internal. Which means if your organization is evaluating on-premise inference or comparing total cost of ownership against OpenAI’s API, you’re comparing your Nvidia GPU costs against OpenAI’s Jalapeno economics. That’s a structural disadvantage that grows over time as Jalapeno capacity scales. CNBC noted https://www.cnbc.com/2026/08/26/openai-jalapeno-ai-chip-nvidia.html that this is a direct threat to Nvidia’s inference margins — but it’s also a deepening of vendor lock-in for OpenAI API customers. The broader trend is unambiguous: every major AI lab is building custom silicon in 2026 — Google’s TPUs, Meta’s MTIA, AWS Inferentia, Microsoft Maia. This isn’t about replacing Nvidia, which posted a $96 billion quarter the same week OpenAI published its Jalapeno benchmarks. It’s about controlling the cost curve of inference as it becomes the dominant ongoing expense of running AI products. OpenAI is now in control of its own silicon roadmap. Generation two reportedly reaches tapeout within months. Generation three is already in design. Watch for when OpenAI starts routing meaningful traffic percentages to Jalapeno — that’s when the downstream pricing effects actually reach developers building on the platform https://thenewstack.io/openai-jalapeno-inference-chip/ .