[AINews] Hot Chips: OpenAI’s Jalapeño, Cerebras CS-5, Groq 3 LPX, Apple M6

By far the biggest announcement at the 37th Hot Chips conference was OpenAI’s stunning progress on their own chip, less than a year after the Broadcom announcement… and that it isn’t an ASIC; but a full on Blackwell-beating alternative.

The key metric now is shifting to performance per watt, and Jalapeno delivers:

The full Hot Chips presentation is not yet out but various takes are below. For a fuller breakdown, watch along with the rest of OpenAI:

AI News for 8/24/2026-8/25/2026. We checked 12 subreddits,

[544 Twitters]and no further Discords.[AINews’ website]lets you search all past issues. As a reminder,[AINews is now a section of Latent Space]. You can[opt in/out]of email frequencies!

AI Twitter Recap

OpenAI’s Jalapeño Inference Chip and the Shift in the Inference Stack

Jalapeño’s published numbers are the day’s biggest technical story: OpenAI released first benchmark details for its custom inference chip** Jalapeño**, claiming materially better efficiency and latency than NVIDIA** GB200/GB300systems on real model workloads. In OpenAI’s tests, Jalapeño delivered 1.5–1.9× more work per wattat peak throughput and 1.7–3.6× lower end-to-end latency**, with** 2.1–4.1× higher performancefor highly interactive workloads; the chip is rated at 700Wbut reportedly stayed at or below 550Won the tested runs. OpenAI says deployment into its own infrastructure begins by year-end**, with** Gen 2already deep in development and Gen 3underway (OpenAI announcement,deployment roadmap,Sam Altman).Why engineers care: the claim is not just raw perf, but a more balanced inference architecture that reduces the usual throughput/latency tradeoff. Multiple technical reactions highlighted that some comparison points are especially notable because Jalapeño reportedly performed well even without tricks like aggressiveprefill/decode disaggregation** orspeculative decoding in some setups, while beating systems that did use them (gdb,kimmonismus summary,eliebakouch analysis,You Jiacheng). SemiAnalysis framed it as unusually strong for a first-generation ASIC and compared it directly againstBlackwell andRubin-class systems (SemiAnalysis,dylan522p).A second-order story is model-assisted systems optimization: OpenAI’s post also said** GPT-Astra + Codexhelped write and optimize low-level kernels, bringing three previously unplanned open-weight models to high performance on Jalapeño in about two months; for selected attention and MoE blocks, these implementations reportedly ran1.5–1.8× faster** than existing human-expert-written code (kimmonismus,eliebakouch). That is a meaningful signal that compiler/kernel work is increasingly being folded into the model improvement loop, not just application-layer coding.Broader infra implication: several posts tie Jalapeño to a larger industry transition in which frontier labs may no longer be strictly downstream of NVIDIA for inference economics, even if packaging and foundry capacity remain a hard bottleneck (Liam Fedus,teortaxesTex reaction,LearnOpenCV caveat on TSMC/CoWoS capacity).

Agent Harnesses, Memory Systems, and Eval Engineering Becoming First-Class

Harness quality is increasingly as important as model choice: several papers and launches converged on the same theme: agent performance depends heavily on the surrounding system. A new Microsoft-led paper onAutoSaddler treats the harness as code and patches prompts, tool configs, and control logic offline using failure traces, reporting gains of**+9.0 on GAIA2**,+9.6 on SWE-Bench Pro, and**+10.0 on Terminal-Bench 2.0** over base harnesses (paper summary). In parallel, another paper quantified harness variance directly, finding that swapping harnesses could move scores far more than swapping models, with model-pair rankings flipping across scaffolds; the proposed fix is a structuredHarness Card disclosure standard (analysis,“There Is No Neutral Harness”).Long-horizon software engineering remains very unsolved:** SWE Refactor Benchmeasures whole-repository migration tasks like C→Rust**,** Maven→Gradle**, and** POSIX→WebAssemblyacross real projects including SQLite**,** zlib**, and** libsodium**. Across** 520 runs**, only** 28survived all three stages, for a 5.4%survival rate, and 13/20tasks were solved by nobody (EinsiaAI). This is a useful corrective to strong bug-fix numbers on more local coding benchmarks.Memory systems are being redesigned as programmable state, not compressed chat history: one Alibaba paper summarized by DAIR backs agent sessions with an append-only event logplus a persistent Python kernel**, binding tool outputs and derived state to typed variables instead of continually serializing them into prompts. Reported results include94.8% on LongMemEval_S,** 73.1% on BEAM_10M**(+5.1 over the previous best published memory system), and** 86.7% on LOCA_256Kwith Qwen3.8-Max**(summary). Related work on** Knowledge Triageshowed that naive context compaction destroys exact-rule retention; after five rounds of compaction, one setup preserved only10%** of safety rules, while type-aware retention policies preserved2–4× more (summary).Practical eval-engineering is moving from ad hoc to productized workflows: LangChain/partners shared a concrete loop for turning traces and human feedback into** task specs**, synthetic environments, and evals that can be used to measure and post-train agents over time (Vtrivedy10,hwchase17). LangSmith Engine also shipped**>2×** better performance on key internal benchmarks with better issue detection/clustering, SaaS and self-hosted support, Slack/Linear integrations, and cost-tiered analysis modes (LangChain).

Local-First Agents, On-Device Inference, and the New Personal Compute Stack

Perplexity’s Portable Computer is the clearest local-agent product launch of the day: Perplexity launched** Portable Computeron NVIDIA DGX Spark**, positioning it as a fully local version of Perplexity Computer where the** orchestrator LLM**,** subagent LLM**, and** agent harnessall run on local hardware with no cloud dependency**(Perplexity launch,model details,NVIDIA,Arav Srinivas). The initial local stack uses a post-trainedPPLX 27B withQwen 3.8 27B also available;Nemotron 3.5 Lightning support is coming.The deeper trend is persistent, always-on local agents: Srinivas explicitly sketched a future of background processes that continuously ingest context from connectors, perform multi-hop reasoning in a perpetual loop, and run on your own hardware (Arav Srinivas). Community reactions were split between excitement about privacy/control and skepticism that “local-first” should mean a**$5k DGX Spark** rather than commodity consumer devices (theo critique,theo follow-up).Apple/macOS local AI tooling is also maturing: exo said Apple featured it on new** M5 Ultra Mac Studioand M6/M5 Pro Mac Minipages, emphasizing low-latency RDMA over Thunderbolt 5to cluster Macs and run models like Kimi K3and GLM-5.3at API-like speeds, with 4× M5 Ultrascaling to about 4.8 TB/s aggregate memory bandwidth**(exo). Related posts pointed to Apple’s faster PCIe storage and ANE-based vision pipelines as making small local clusters and mixed CPU/ANE/GPU inference more practical (anemll,onirenaud).Tooling continues to fill in around local runtimes:** Ollama v0.33added one-toggle integration to let Claude Desktopuse Ollama as a third-party gateway for cloud and local models (Ollama); OpenCode v2 was shown running inside aCloudflare Durable Object**, illustrating how small agent runtimes are becoming embeddable in edge environments (fayazara).

Models, Retrieval, and Search Infrastructure

Qwen 3.8 is showing up across the stack: enthusiasm around the** Qwen3.8release was visible in both deployment and evaluation posts, with Together adding fine-tuning and dedicated inference support forQwen3.8-27B**(Together) and Unsloth claiming full** QLoRAfine-tuning of the 27B model on free 2× Tesla T4Kaggle instances using optimized kernels (danielhanchen). On the application side,Qwen3.8-27B reached#1 among open models** in theImage-to-WebDev Arena and**#7 overall**, while priced at**$0.40 / $3 per million input/output tokens**(arena).** Search and retrieval infra got multiple substantive updates**: Hugging Face published a detailed architecture writeup for the** Papers with Codesearch engine: PostgreSQL + pgvector**,** Qwen 3 Embedding 0.6B**, hybrid retrieval, embeddings generated on an** NVIDIA L4via Hugging Face Jobs, artifacts in buckets, and live serving via Inference Endpoints; the same stack powers “related papers” on paper pages (Niels Rogge). Keenable came out of stealth with aWeb Search API** andWeb Query Language for AI, built by former Yandex Search leaders and backed by a**$26M seed**, explicitly targeting agent-scale web retrieval (styskin).** Retrieval model design remains active territory**: there was renewed discussion around** late interaction / multivector retrieval**, with claims that scaling behavior is finally becoming visible in retrieval workloads and that model+DB co-design matters at least as much as storage format (mixedbread perspective,Silvio Martinico).

Robotics, Physical World Models, and Embodied Data

Figure’s “Index” is a major robotics data announcement: Figure introduced** Index**, described as the largest and most diverse robot dataset in the world, with reported ingestion at** 30 minutes of video uploads per second**,** 16M video uploads**,$15M already paid out for data, and264k downloads. The company also says it will spend**$1B over the next 12 months** on data and compute (Brett Adcock,follow-up). That scale matters because many robotics labs still appear more bottlenecked on demonstration and perception data than on architecture novelty.Large-scale physics/world modeling continues to push context limits: Anima Anandkumar highlighted** Accelerated Understanding**, a startup building large AI models for physical simulation across modalities and 4D spacetime, claiming** 1T parameters during pretraining**,** 1T contextduring training, and>5T context** at inference without subsampling or patching (Anima Anandkumar). The details are sparse, but the post is notable as a statement of where some frontier non-language modeling work is heading: massive-context multimodal simulation rather than only text/video generation.Embodied policy generalization remains an active benchmark target: a separate robotics post introduced** S1**, a manipulation model that can complete tasks from a** single demonstrationoutside its training distribution (anag004). Google Research also sharedAgentHands**, an XR system that augments conversational agents with synchronized hand gestures for spatial guidance during physical tasks (Google Research).

Top tweets (by engagement) OpenAI chip launch:@sama on Jalapeño,@OpenAI benchmark announcementdrove the largest technical conversation by far.Local agent launch:@perplexity_ai launching Portable Computerwas the biggest product release outside the chip story.Developer platform / agent-native web:@OpenAIDevs announcing the WebMCP ChallengeandWebMCP support in ChatGPT desktopsignal OpenAI pushing websites toward explicit agent interfaces.Open-source local task agents:@AndrewYNg on OpenWorkerstood out for combining open harnesses, local models, and security-focused workflows.Benchmark realism for coding agents:@EinsiaAI on SWE Refactor Benchis one of the more useful benchmark releases in the set because it targets whole-repo migrations instead of local edits.