Agentic AI workloads are shifting the compute ratio, making CPUs the critical coordination substrate rather than a support component for GPUs. This inversion is exposed by current Xeon supply tightnes…
Carmen Li, CEO of Silicon Data and Compute Exchange, said the compute market is shifting from on-demand to forward contracts as GPU price volatility rises, with fungibility challenges complicating tra…
A comprehensive list of 33 metrics for evaluating large language models (LLMs) has been compiled, covering performance indicators such as time to first token, average tokens per second, throughput, er…
A Hacker News user asks the community how they choose between GPU rental providers for machine learning workloads, questioning whether price, reliability, or setup environment is the deciding factor w…
Neurophos announced its Optical Processing Unit (OPU), a photonic AI chip achieving 0.47 ExaOPS at 300 TOPS/W, claiming 10,000x size reduction over traditional photonic tensor cores. The company says …
Anjney Midha, founder and CEO of AMP, said AI development is consuming unprecedented resources and that big companies dominate the landscape due to their ability to justify massive spending on AGI. He…
Researchers at an undisclosed institution reverse-engineered Apple's Metal 4.1 tensor compute path on the M4 Max GPU, revealing that the fp8 matmul2d operation is emulated rather than hardware-acceler…
Engineering teams face a convergence of latency budgets, quorum math, and cache TTLs as the same underlying constraint, with ML systems intensifying the squeeze by placing LLM-as-judge loops and predi…
A developer has built a carbon-aware training pipeline for PyTorch that schedules GPU workloads around real-time electricity carbon intensity, reducing CO2 emissions by delaying training until low-car…
Ollama, an open-source runtime for large language models, enables users to run models locally on Mac, Windows, or Linux with a single command, eliminating the need for cloud dependencies or complex en…
Researchers introduced Tangram, a serving system that enables non-uniform Key-Value cache compression for multi-turn large language model inference. The system uses deterministic budget allocation, he…
A developer has created a credit-based billing system for Next.js AI applications using Stripe and Supabase, solving the problem of flat-rate pricing that can lead to margin erosion from heavy users. …
Researchers published a proof-of-concept AI-driven worm that embeds an open-weight LLM on compromised GPUs to autonomously scan, exploit, and propagate across Linux, Windows, and IoT devices. The worm…
A developer built a continuous AI agent memory system called `akm improve` that runs autonomously on local hardware, processing 14,189 memories across 48 scheduled runs in 24 hours with zero failures.…
WebGL fingerprinting is a hardware-level anti-bot technique that uses GPU rendering differences to create unique device identifiers, making it difficult to spoof compared to software-level signals. Th…
Researchers have demonstrated that large language models can serve as selective surrogates for GPU kernel performance evaluation, accurately forecasting kernel runtime without requiring repeated compi…
NVIDIA Apex's FusedAdam optimizer and FusedLayerNorm normalization layers can accelerate Transformer training by up to 30% compared to standard PyTorch implementations, according to benchmark tests. T…
A developer has found that centralized resource scheduling, like Kubernetes-style limits, wastes 30-40% of compute capacity in AI agent swarms by failing to account for agents' real-time utility needs…
Speculative decoding optimizes LLM inference by using a cheap draft model to predict multiple tokens, which are then verified in a single forward pass of the target model, reducing memory-bandwidth bo…
Modern GPUs spend most of their time during AI inference waiting for data, as memory bandwidth cannot keep pace with compute throughput. This fundamental bottleneck has driven the AI hardware market, …