[AINews] Hot Chips: OpenAI’s Jalapeño, Cerebras CS-5, Groq 3 LPX, Apple M6

At the 37th Hot Chips conference, OpenAI unveiled its custom inference chip Jalapeño, claiming it outperforms NVIDIA's GB200/GB300 systems with 1.5–1.9× more work per watt at peak throughput, 1.7–3.6× lower end-to-end latency, and 2.1–4.1× higher performance for interactive workloads, while staying at or below 550W despite a 700W rating. OpenAI said deployment into its own infrastructure begins by year-end, with Gen 2 in development and Gen 3 underway, and that GPT-Astra + Codex helped optimize kernels, running 1.5–1.8× faster than human-written code for selected blocks.

By far the biggest announcement at the 37th Hot Chips conference https://hotchips.org/about/ was OpenAI’s stunning progress on their own chip, less than a year after the Broadcom announcement https://www.latent.space/p/ainews-the-custom-asic-thesis?utm source=publication-search … and that it isn’t an ASIC; but a full on Blackwell-beating https://x.com/SemiAnalysis /status/2092253723640598761 alternative. The key metric now is shifting to performance per watt, and Jalapeno delivers: The full Hot Chips presentation is not yet out but various takes are below. For a fuller breakdown, watch along with the rest of OpenAI: AI News for 8/24/2026-8/25/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space . You can opt in/out of email frequencies AI Twitter Recap OpenAI’s Jalapeño Inference Chip and the Shift in the Inference Stack Jalapeño’s published numbers are the day’s biggest technical story : OpenAI released first benchmark details for its custom inference chip Jalapeño , claiming materially better efficiency and latency than NVIDIA GB200/GB300 systems on real model workloads. In OpenAI’s tests, Jalapeño delivered 1.5–1.9× more work per watt at peak throughput and 1.7–3.6× lower end-to-end latency , with 2.1–4.1× higher performance for highly interactive workloads; the chip is rated at 700W but reportedly stayed at or below 550W on the tested runs. OpenAI says deployment into its own infrastructure begins by year-end , with Gen 2 already deep in development and Gen 3 underway OpenAI announcement https://x.com/OpenAI/status/2092300846675505602 , deployment roadmap https://x.com/OpenAI/status/2092300851482108064 , Sam Altman https://x.com/sama/status/2092339694210040187 . Why engineers care : the claim is not just raw perf, but a more balanced inference architecture that reduces the usual throughput/latency tradeoff. Multiple technical reactions highlighted that some comparison points are especially notable because Jalapeño reportedly performed well even without tricks like aggressive prefill/decode disaggregation or speculative decoding in some setups, while beating systems that did use them gdb https://x.com/gdb/status/2092273740239552780 , kimmonismus summary https://x.com/kimmonismus/status/2092261453449327052 , eliebakouch analysis https://x.com/eliebakouch/status/2092287935664328816 , You Jiacheng https://x.com/YouJiacheng/status/2092280093766766949 . SemiAnalysis framed it as unusually strong for a first-generation ASIC and compared it directly against Blackwell and Rubin -class systems SemiAnalysis https://x.com/SemiAnalysis /status/2092253723640598761 , dylan522p https://x.com/dylan522p/status/2092258594628706778 . A second-order story is model-assisted systems optimization : OpenAI’s post also said GPT-Astra + Codex helped write and optimize low-level kernels, bringing three previously unplanned open-weight models to high performance on Jalapeño in about two months; for selected attention and MoE blocks, these implementations reportedly ran 1.5–1.8× faster than existing human-expert-written code kimmonismus https://x.com/kimmonismus/status/2092314583981539731 , eliebakouch https://x.com/eliebakouch/status/2092267891898917275 . That is a meaningful signal that compiler/kernel work is increasingly being folded into the model improvement loop, not just application-layer coding. Broader infra implication : several posts tie Jalapeño to a larger industry transition in which frontier labs may no longer be strictly downstream of NVIDIA for inference economics, even if packaging and foundry capacity remain a hard bottleneck Liam Fedus https://x.com/LiamFedus/status/2092279297113559373 , teortaxesTex reaction https://x.com/teortaxesTex/status/2092268381269323815 , LearnOpenCV caveat on TSMC/CoWoS capacity https://x.com/LearnOpenCV/status/2092302563987341406 . Agent Harnesses, Memory Systems, and Eval Engineering Becoming First-Class Harness quality is increasingly as important as model choice : several papers and launches converged on the same theme: agent performance depends heavily on the surrounding system. A new Microsoft-led paper on AutoSaddler treats the harness as code and patches prompts, tool configs, and control logic offline using failure traces, reporting gains of +9.0 on GAIA2 , +9.6 on SWE-Bench Pro , and +10.0 on Terminal-Bench 2.0 over base harnesses paper summary https://x.com/omarsar0/status/2092246879702769956 . In parallel, another paper quantified harness variance directly, finding that swapping harnesses could move scores far more than swapping models, with model-pair rankings flipping across scaffolds; the proposed fix is a structured Harness Card disclosure standard analysis https://x.com/omarsar0/status/2092412718573899970 , “There Is No Neutral Harness” https://x.com/dair ai/status/2092386565045747719 . Long-horizon software engineering remains very unsolved : SWE Refactor Bench measures whole-repository migration tasks like C→Rust , Maven→Gradle , and POSIX→WebAssembly across real projects including SQLite , zlib , and libsodium . Across 520 runs , only 28 survived all three stages, for a 5.4% survival rate, and 13/20 tasks were solved by nobody EinsiaAI https://x.com/EinsiaAI/status/2092258194097901654 . This is a useful corrective to strong bug-fix numbers on more local coding benchmarks. Memory systems are being redesigned as programmable state, not compressed chat history : one Alibaba paper summarized by DAIR backs agent sessions with an append-only event log plus a persistent Python kernel , binding tool outputs and derived state to typed variables instead of continually serializing them into prompts. Reported results include 94.8% on LongMemEval S , 73.1% on BEAM 10M +5.1 over the previous best published memory system , and 86.7% on LOCA 256K with Qwen3.8-Max summary https://x.com/omarsar0/status/2092274559898755485 . Related work on Knowledge Triage showed that naive context compaction destroys exact-rule retention; after five rounds of compaction, one setup preserved only 10% of safety rules, while type-aware retention policies preserved 2–4× more summary https://x.com/omarsar0/status/2092326207077634351 . Practical eval-engineering is moving from ad hoc to productized workflows : LangChain/partners shared a concrete loop for turning traces and human feedback into task specs , synthetic environments, and evals that can be used to measure and post-train agents over time Vtrivedy10 https://x.com/Vtrivedy10/status/2092267628869882164 , hwchase17 https://x.com/hwchase17/status/2092268188633546943 . LangSmith Engine also shipped 2× better performance on key internal benchmarks with better issue detection/clustering, SaaS and self-hosted support, Slack/Linear integrations, and cost-tiered analysis modes LangChain https://x.com/LangChain/status/2092311894786716159 . Local-First Agents, On-Device Inference, and the New Personal Compute Stack Perplexity’s Portable Computer is the clearest local-agent product launch of the day : Perplexity launched Portable Computer on NVIDIA DGX Spark , positioning it as a fully local version of Perplexity Computer where the orchestrator LLM , subagent LLM , and agent harness all run on local hardware with no cloud dependency Perplexity launch https://x.com/perplexity ai/status/2092268362386780270 , model details https://x.com/perplexity ai/status/2092268398319481039 , NVIDIA https://x.com/nvidia/status/2092269109086126575 , Arav Srinivas https://x.com/AravSrinivas/status/2092270041471598820 . The initial local stack uses a post-trained PPLX 27B with Qwen 3.8 27B also available; Nemotron 3.5 Lightning support is coming. The deeper trend is persistent, always-on local agents : Srinivas explicitly sketched a future of background processes that continuously ingest context from connectors, perform multi-hop reasoning in a perpetual loop, and run on your own hardware Arav Srinivas https://x.com/AravSrinivas/status/2092428727338865110 . Community reactions were split between excitement about privacy/control and skepticism that “local-first” should mean a $5k DGX Spark rather than commodity consumer devices theo critique https://x.com/theo/status/2092382967427653677 , theo follow-up https://x.com/theo/status/2092383482983157999 . Apple/macOS local AI tooling is also maturing : exo said Apple featured it on new M5 Ultra Mac Studio and M6/M5 Pro Mac Mini pages, emphasizing low-latency RDMA over Thunderbolt 5 to cluster Macs and run models like Kimi K3 and GLM-5.3 at API-like speeds, with 4× M5 Ultra scaling to about 4.8 TB/s aggregate memory bandwidth exo https://x.com/exolabs/status/2092320487019880735 . Related posts pointed to Apple’s faster PCIe storage and ANE-based vision pipelines as making small local clusters and mixed CPU/ANE/GPU inference more practical anemll https://x.com/anemll/status/2092268637935882285 , onirenaud https://x.com/onirenaud/status/2092275271449944512 . Tooling continues to fill in around local runtimes : Ollama v0.33 added one-toggle integration to let Claude Desktop use Ollama as a third-party gateway for cloud and local models Ollama https://x.com/ollama/status/2092453536634380763 ; OpenCode v2 was shown running inside a Cloudflare Durable Object , illustrating how small agent runtimes are becoming embeddable in edge environments fayazara https://x.com/fayazara/status/2092251058148130935 . Models, Retrieval, and Search Infrastructure Qwen 3.8 is showing up across the stack : enthusiasm around the Qwen3.8 release was visible in both deployment and evaluation posts, with Together adding fine-tuning and dedicated inference support for Qwen3.8-27B Together https://x.com/togethercompute/status/2092339003777573069 and Unsloth claiming full QLoRA fine-tuning of the 27B model on free 2× Tesla T4 Kaggle instances using optimized kernels danielhanchen https://x.com/danielhanchen/status/2092262487651713507 . On the application side, Qwen3.8-27B reached 1 among open models in the Image-to-WebDev Arena and 7 overall , while priced at $0.40 / $3 per million input/output tokens arena https://x.com/arena/status/2092301580091711491 . Search and retrieval infra got multiple substantive updates : Hugging Face published a detailed architecture writeup for the Papers with Code search engine: PostgreSQL + pgvector , Qwen 3 Embedding 0.6B , hybrid retrieval, embeddings generated on an NVIDIA L4 via Hugging Face Jobs, artifacts in buckets, and live serving via Inference Endpoints; the same stack powers “related papers” on paper pages Niels Rogge https://x.com/NielsRogge/status/2092217649199489238 . Keenable came out of stealth with a Web Search API and Web Query Language for AI, built by former Yandex Search leaders and backed by a $26M seed , explicitly targeting agent-scale web retrieval styskin https://x.com/styskin/status/2092265673041084505 . Retrieval model design remains active territory : there was renewed discussion around late interaction / multivector retrieval , with claims that scaling behavior is finally becoming visible in retrieval workloads and that model+DB co-design matters at least as much as storage format mixedbread perspective https://x.com/aaxsh18/status/2092297534379352501 , Silvio Martinico https://x.com/SilvioMartinico/status/2092232159377391898 . Robotics, Physical World Models, and Embodied Data Figure’s “Index” is a major robotics data announcement : Figure introduced Index , described as the largest and most diverse robot dataset in the world, with reported ingestion at 30 minutes of video uploads per second , 16M video uploads , $15M already paid out for data, and 264k downloads . The company also says it will spend $1B over the next 12 months on data and compute Brett Adcock https://x.com/adcock brett/status/2092303633559982106 , follow-up https://x.com/adcock brett/status/2092304599466303972 . That scale matters because many robotics labs still appear more bottlenecked on demonstration and perception data than on architecture novelty. Large-scale physics/world modeling continues to push context limits : Anima Anandkumar highlighted Accelerated Understanding , a startup building large AI models for physical simulation across modalities and 4D spacetime, claiming 1T parameters during pretraining , 1T context during training, and 5T context at inference without subsampling or patching Anima Anandkumar https://x.com/AnimaAnandkumar/status/2092236528898675014 . The details are sparse, but the post is notable as a statement of where some frontier non-language modeling work is heading: massive-context multimodal simulation rather than only text/video generation. Embodied policy generalization remains an active benchmark target : a separate robotics post introduced S1 , a manipulation model that can complete tasks from a single demonstration outside its training distribution anag004 https://x.com/anag004/status/2092310314406887612 . Google Research also shared AgentHands , an XR system that augments conversational agents with synchronized hand gestures for spatial guidance during physical tasks Google Research https://x.com/GoogleResearch/status/2092331108314845361 . Top tweets by engagement OpenAI chip launch : @sama on Jalapeño https://x.com/sama/status/2092339694210040187 , @OpenAI benchmark announcement https://x.com/OpenAI/status/2092300846675505602 drove the largest technical conversation by far. Local agent launch : @perplexity ai launching Portable Computer https://x.com/perplexity ai/status/2092268362386780270 was the biggest product release outside the chip story. Developer platform / agent-native web : @OpenAIDevs announcing the WebMCP Challenge https://x.com/OpenAIDevs/status/2092344873764704345 and WebMCP support in ChatGPT desktop https://x.com/OpenAIDevs/status/2092344959248761263 signal OpenAI pushing websites toward explicit agent interfaces. Open-source local task agents : @AndrewYNg on OpenWorker https://x.com/AndrewYNg/status/2092315079576555806 stood out for combining open harnesses, local models, and security-focused workflows. Benchmark realism for coding agents : @EinsiaAI on SWE Refactor Bench https://x.com/EinsiaAI/status/2092258194097901654 is one of the more useful benchmark releases in the set because it targets whole-repo migrations instead of local edits.