A custom self-attention in-memory computing architecture built on charge-based gain cells reduces attention latency by up to two orders of magnitude and energy consumption by up to four orders of magn…
A new installment in the "How To Scale Your Model" series lays out a theory of sharded matrix multiplication for training large ML models across thousands of accelerators, using named-axis notation to…
Bend, a high-level programming language that compiles mathematically verified code to run natively and in parallel on both CPUs and GPUs without manual multi-threading, is pitched as a constraint laye…
Technology leaders are rethinking their AWS, Azure and Google Cloud commitments as AI workloads drive cloud bills beyond forecast, according to CIO.com. CIOs report that data movement across regions a…
A forum poster clarified that their original proposal was not to use an NVMe SSD as VRAM, but to design future consumer GPUs with an expandable, lower-cost memory tier managed directly by the GPU, sim…
A modern AI inference platform is built from five layers, and its capacity is bound by the lower of two independent ceilings: how many requests fit in memory and how many the latency target allows, ac…
Samsung has patented a method to store AI embedding tables on a dedicated memory semantic SSD (MS SSD) that hands vectors directly to a GPU and receives updated vectors back after each training step, …
SK hynix held its 2026 Future Forum on September 9, 2026, where the company charted its business and technology direction under the theme "Now Is the Golden Time to Leap Forward." The forum covered 3D…
Researchers propose Symphony, a hybrid programmable/specialized architecture that orchestrates data throughout the memory hierarchy to reduce unnecessary data movement and distances, achieving 31× imp…
A constraint-aware GPU allocator built by Dharma AI raised GPU utilization by up to 33 percentage points and priority-weighted output by up to 105% over a FIFO scheduler across seven benchmark scenari…
Engineers are integrating large language models into real-time data pipelines, facing challenges such as state synchronization and KV cache transfer bottlenecks. Optimizations using TensorRT-LLM and a…
A structural shift in storage economics, driven by AI's demand for flash memory, has widened the price gap between flash and HDD from 1:3 to 1:10 or 1:15, making all-flash architectures prohibitively …
A developer explains that neural networks process text by converting words into token IDs, which are then transformed through matrix multiplication and non-linear activation functions. The post detail…
Microsoft has decided to keep its capital expenditure unchanged, signaling a strategic pivot from building massive AI infrastructure to maximizing the return on existing hardware, according to an anal…
A technical analysis of backpropagation kernels reveals that performance walls in certain layers are caused by low occupancy, where the GPU fails to keep enough warps active on the Streaming Multiproc…
A Verge investigation reveals that the environmental cost of GPUs extends far beyond AI data centers, encompassing manufacturing, supply chains, and e-waste. The report highlights that producing a sin…
A developer on Hacker News has released a forward-only, autograd-free Physics-Informed Neural Network (PINN) that uses zero-copy memory interlock to reduce VRAM consumption to approximately 1/1000 of …
A new benchmark, Atrex-Bench, drawn from full-cluster production inference traces shows that even the best frontier coding agents reach only ~10% of hardware roofline on real GPU operators, with corre…
Google Cloud Vice President of Technology and Customer Engineering for APAC Moe Abdula said the company has not faced infrastructure supply constraints amid surging AI demand, citing strong supplier p…
Google's AICore transforms AI models from app-bundled assets into system services on Android, enabling heterogeneous parallelism across NPU, GPU, and DSP. The approach addresses the memory wall via ze…