# [AINews] Hot Chips: OpenAI’s Jalapeño, Cerebras CS-5, Groq 3 LPX, Apple M6 > Source: > Published: 2026-08-27 01:31:22+00:00 By far the biggest announcement at the [37th Hot Chips conference](https://hotchips.org/about/) was OpenAI’s stunning progress on their own chip, less than a year after the [Broadcom announcement](https://www.latent.space/p/ainews-the-custom-asic-thesis?utm_source=publication-search)… and that it isn’t an ASIC; but a full on [Blackwell-beating](https://x.com/SemiAnalysis_/status/2092253723640598761) alternative. The key metric now is shifting to performance per watt, and Jalapeno delivers: The full Hot Chips presentation is not yet out but various takes are below. For a fuller breakdown, watch along with the rest of OpenAI: AI News for 8/24/2026-8/25/2026. We checked 12 subreddits, [544 Twitters]and no further Discords.[AINews’ website]lets you search all past issues. As a reminder,[AINews is now a section of Latent Space]. You can[opt in/out]of email frequencies! **AI Twitter Recap** **OpenAI’s Jalapeño Inference Chip and the Shift in the Inference Stack** **Jalapeño’s published numbers are the day’s biggest technical story**: OpenAI released first benchmark details for its custom inference chip** Jalapeño**, claiming materially better efficiency and latency than NVIDIA** GB200/GB300**systems on real model workloads. In OpenAI’s tests, Jalapeño delivered** 1.5–1.9× more work per watt**at peak throughput and** 1.7–3.6× lower end-to-end latency**, with** 2.1–4.1× higher performance**for highly interactive workloads; the chip is rated at** 700W**but reportedly stayed at or below** 550W**on the tested runs. OpenAI says deployment into its own infrastructure begins** by year-end**, with** Gen 2**already deep in development and** Gen 3**underway ([OpenAI announcement](https://x.com/OpenAI/status/2092300846675505602),[deployment roadmap](https://x.com/OpenAI/status/2092300851482108064),[Sam Altman](https://x.com/sama/status/2092339694210040187)).**Why engineers care**: the claim is not just raw perf, but a more balanced inference architecture that reduces the usual throughput/latency tradeoff. Multiple technical reactions highlighted that some comparison points are especially notable because Jalapeño reportedly performed well even without tricks like aggressive**prefill/decode disaggregation** or**speculative decoding** in some setups, while beating systems that did use them ([gdb](https://x.com/gdb/status/2092273740239552780),[kimmonismus summary](https://x.com/kimmonismus/status/2092261453449327052),[eliebakouch analysis](https://x.com/eliebakouch/status/2092287935664328816),[You Jiacheng](https://x.com/YouJiacheng/status/2092280093766766949)). SemiAnalysis framed it as unusually strong for a first-generation ASIC and compared it directly against**Blackwell** and**Rubin**-class systems ([SemiAnalysis](https://x.com/SemiAnalysis_/status/2092253723640598761),[dylan522p](https://x.com/dylan522p/status/2092258594628706778)).**A second-order story is model-assisted systems optimization**: OpenAI’s post also said** GPT-Astra + Codex**helped write and optimize low-level kernels, bringing three previously unplanned open-weight models to high performance on Jalapeño in about two months; for selected attention and MoE blocks, these implementations reportedly ran**1.5–1.8× faster** than existing human-expert-written code ([kimmonismus](https://x.com/kimmonismus/status/2092314583981539731),[eliebakouch](https://x.com/eliebakouch/status/2092267891898917275)). That is a meaningful signal that compiler/kernel work is increasingly being folded into the model improvement loop, not just application-layer coding.**Broader infra implication**: several posts tie Jalapeño to a larger industry transition in which frontier labs may no longer be strictly downstream of NVIDIA for inference economics, even if packaging and foundry capacity remain a hard bottleneck ([Liam Fedus](https://x.com/LiamFedus/status/2092279297113559373),[teortaxesTex reaction](https://x.com/teortaxesTex/status/2092268381269323815),[LearnOpenCV caveat on TSMC/CoWoS capacity](https://x.com/LearnOpenCV/status/2092302563987341406)). **Agent Harnesses, Memory Systems, and Eval Engineering Becoming First-Class** **Harness quality is increasingly as important as model choice**: several papers and launches converged on the same theme: agent performance depends heavily on the surrounding system. A new Microsoft-led paper on**AutoSaddler** treats the harness as code and patches prompts, tool configs, and control logic offline using failure traces, reporting gains of**+9.0 on GAIA2**,**+9.6 on SWE-Bench Pro**, and**+10.0 on Terminal-Bench 2.0** over base harnesses ([paper summary](https://x.com/omarsar0/status/2092246879702769956)). In parallel, another paper quantified harness variance directly, finding that swapping harnesses could move scores far more than swapping models, with model-pair rankings flipping across scaffolds; the proposed fix is a structured**Harness Card** disclosure standard ([analysis](https://x.com/omarsar0/status/2092412718573899970),[“There Is No Neutral Harness”](https://x.com/dair_ai/status/2092386565045747719)).**Long-horizon software engineering remains very unsolved**:** SWE Refactor Bench**measures whole-repository migration tasks like** C→Rust**,** Maven→Gradle**, and** POSIX→WebAssembly**across real projects including** SQLite**,** zlib**, and** libsodium**. Across** 520 runs**, only** 28**survived all three stages, for a** 5.4%**survival rate, and** 13/20**tasks were solved by nobody ([EinsiaAI](https://x.com/EinsiaAI/status/2092258194097901654)). This is a useful corrective to strong bug-fix numbers on more local coding benchmarks.**Memory systems are being redesigned as programmable state, not compressed chat history**: one Alibaba paper summarized by DAIR backs agent sessions with an** append-only event log**plus a** persistent Python kernel**, binding tool outputs and derived state to typed variables instead of continually serializing them into prompts. Reported results include**94.8% on LongMemEval_S**,** 73.1% on BEAM_10M**(+5.1 over the previous best published memory system), and** 86.7% on LOCA_256K**with** Qwen3.8-Max**([summary](https://x.com/omarsar0/status/2092274559898755485)). Related work on** Knowledge Triage**showed that naive context compaction destroys exact-rule retention; after five rounds of compaction, one setup preserved only**10%** of safety rules, while type-aware retention policies preserved**2–4×** more ([summary](https://x.com/omarsar0/status/2092326207077634351)).**Practical eval-engineering is moving from ad hoc to productized workflows**: LangChain/partners shared a concrete loop for turning traces and human feedback into** task specs**, synthetic environments, and evals that can be used to measure and post-train agents over time ([Vtrivedy10](https://x.com/Vtrivedy10/status/2092267628869882164),[hwchase17](https://x.com/hwchase17/status/2092268188633546943)). LangSmith Engine also shipped**>2×** better performance on key internal benchmarks with better issue detection/clustering, SaaS and self-hosted support, Slack/Linear integrations, and cost-tiered analysis modes ([LangChain](https://x.com/LangChain/status/2092311894786716159)). **Local-First Agents, On-Device Inference, and the New Personal Compute Stack** **Perplexity’s Portable Computer is the clearest local-agent product launch of the day**: Perplexity launched** Portable Computer**on** NVIDIA DGX Spark**, positioning it as a fully local version of Perplexity Computer where the** orchestrator LLM**,** subagent LLM**, and** agent harness**all run on local hardware with** no cloud dependency**([Perplexity launch](https://x.com/perplexity_ai/status/2092268362386780270),[model details](https://x.com/perplexity_ai/status/2092268398319481039),[NVIDIA](https://x.com/nvidia/status/2092269109086126575),[Arav Srinivas](https://x.com/AravSrinivas/status/2092270041471598820)). The initial local stack uses a post-trained**PPLX 27B** with**Qwen 3.8 27B** also available;**Nemotron 3.5 Lightning** support is coming.**The deeper trend is persistent, always-on local agents**: Srinivas explicitly sketched a future of background processes that continuously ingest context from connectors, perform multi-hop reasoning in a perpetual loop, and run on your own hardware ([Arav Srinivas](https://x.com/AravSrinivas/status/2092428727338865110)). Community reactions were split between excitement about privacy/control and skepticism that “local-first” should mean a**$5k DGX Spark** rather than commodity consumer devices ([theo critique](https://x.com/theo/status/2092382967427653677),[theo follow-up](https://x.com/theo/status/2092383482983157999)).**Apple/macOS local AI tooling is also maturing**: exo said Apple featured it on new** M5 Ultra Mac Studio**and** M6/M5 Pro Mac Mini**pages, emphasizing** low-latency RDMA over Thunderbolt 5**to cluster Macs and run models like** Kimi K3**and** GLM-5.3**at API-like speeds, with** 4× M5 Ultra**scaling to about** 4.8 TB/s aggregate memory bandwidth**([exo](https://x.com/exolabs/status/2092320487019880735)). Related posts pointed to Apple’s faster PCIe storage and ANE-based vision pipelines as making small local clusters and mixed CPU/ANE/GPU inference more practical ([anemll](https://x.com/anemll/status/2092268637935882285),[onirenaud](https://x.com/onirenaud/status/2092275271449944512)).**Tooling continues to fill in around local runtimes**:** Ollama v0.33**added one-toggle integration to let** Claude Desktop**use Ollama as a third-party gateway for cloud and local models ([Ollama](https://x.com/ollama/status/2092453536634380763)); OpenCode v2 was shown running inside a**Cloudflare Durable Object**, illustrating how small agent runtimes are becoming embeddable in edge environments ([fayazara](https://x.com/fayazara/status/2092251058148130935)). **Models, Retrieval, and Search Infrastructure** **Qwen 3.8 is showing up across the stack**: enthusiasm around the** Qwen3.8**release was visible in both deployment and evaluation posts, with Together adding fine-tuning and dedicated inference support for**Qwen3.8-27B**([Together](https://x.com/togethercompute/status/2092339003777573069)) and Unsloth claiming full** QLoRA**fine-tuning of the 27B model on free** 2× Tesla T4**Kaggle instances using optimized kernels ([danielhanchen](https://x.com/danielhanchen/status/2092262487651713507)). On the application side,**Qwen3.8-27B** reached**#1 among open models** in the**Image-to-WebDev Arena** and**#7 overall**, while priced at**$0.40 / $3 per million input/output tokens**([arena](https://x.com/arena/status/2092301580091711491)).** Search and retrieval infra got multiple substantive updates**: Hugging Face published a detailed architecture writeup for the** Papers with Code**search engine:** PostgreSQL + pgvector**,** Qwen 3 Embedding 0.6B**, hybrid retrieval, embeddings generated on an** NVIDIA L4**via Hugging Face Jobs, artifacts in buckets, and live serving via Inference Endpoints; the same stack powers “related papers” on paper pages ([Niels Rogge](https://x.com/NielsRogge/status/2092217649199489238)). Keenable came out of stealth with a**Web Search API** and**Web Query Language** for AI, built by former Yandex Search leaders and backed by a**$26M seed**, explicitly targeting agent-scale web retrieval ([styskin](https://x.com/styskin/status/2092265673041084505)).** Retrieval model design remains active territory**: there was renewed discussion around** late interaction / multivector retrieval**, with claims that scaling behavior is finally becoming visible in retrieval workloads and that model+DB co-design matters at least as much as storage format ([mixedbread perspective](https://x.com/aaxsh18/status/2092297534379352501),[Silvio Martinico](https://x.com/SilvioMartinico/status/2092232159377391898)). **Robotics, Physical World Models, and Embodied Data** **Figure’s “Index” is a major robotics data announcement**: Figure introduced** Index**, described as the largest and most diverse robot dataset in the world, with reported ingestion at** 30 minutes of video uploads per second**,** 16M video uploads**,**$15M** already paid out for data, and**264k downloads**. The company also says it will spend**$1B over the next 12 months** on data and compute ([Brett Adcock](https://x.com/adcock_brett/status/2092303633559982106),[follow-up](https://x.com/adcock_brett/status/2092304599466303972)). That scale matters because many robotics labs still appear more bottlenecked on demonstration and perception data than on architecture novelty.**Large-scale physics/world modeling continues to push context limits**: Anima Anandkumar highlighted** Accelerated Understanding**, a startup building large AI models for physical simulation across modalities and 4D spacetime, claiming** 1T parameters during pretraining**,** 1T context**during training, and**>5T context** at inference without subsampling or patching ([Anima Anandkumar](https://x.com/AnimaAnandkumar/status/2092236528898675014)). The details are sparse, but the post is notable as a statement of where some frontier non-language modeling work is heading: massive-context multimodal simulation rather than only text/video generation.**Embodied policy generalization remains an active benchmark target**: a separate robotics post introduced** S1**, a manipulation model that can complete tasks from a** single demonstration**outside its training distribution ([anag004](https://x.com/anag004/status/2092310314406887612)). Google Research also shared**AgentHands**, an XR system that augments conversational agents with synchronized hand gestures for spatial guidance during physical tasks ([Google Research](https://x.com/GoogleResearch/status/2092331108314845361)). **Top tweets (by engagement)** **OpenAI chip launch**:[@sama on Jalapeño](https://x.com/sama/status/2092339694210040187),[@OpenAI benchmark announcement](https://x.com/OpenAI/status/2092300846675505602)drove the largest technical conversation by far.**Local agent launch**:[@perplexity_ai launching Portable Computer](https://x.com/perplexity_ai/status/2092268362386780270)was the biggest product release outside the chip story.**Developer platform / agent-native web**:[@OpenAIDevs announcing the WebMCP Challenge](https://x.com/OpenAIDevs/status/2092344873764704345)and[WebMCP support in ChatGPT desktop](https://x.com/OpenAIDevs/status/2092344959248761263)signal OpenAI pushing websites toward explicit agent interfaces.**Open-source local task agents**:[@AndrewYNg on OpenWorker](https://x.com/AndrewYNg/status/2092315079576555806)stood out for combining open harnesses, local models, and security-focused workflows.**Benchmark realism for coding agents**:[@EinsiaAI on SWE Refactor Bench](https://x.com/EinsiaAI/status/2092258194097901654)is one of the more useful benchmark releases in the set because it targets whole-repo migrations instead of local edits.