RTX 4090 global loads take 15 ns at L1, 127 ns at L2, 255 ns at DRAM
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
A single LDG.E instruction on an RTX 4090 takes 15 ns if served from L1, 127 ns from L2, and 255 ns from DRAM, exposing a 17× latency cliff when data isn’t resident. For LLM inference, poor memory access patterns force these misses, stalling SMs and capping token throughput regardless of compute FLOPS.
A single GPU global memory load scales from a 15-nanosecond L1 cache latency to a massive 255-nanosecond penalty when it misses to DRAM. For memory-bandwidth-bound LLM inference, failing to coalesce threads into contiguous 128-byte cache lines triggers this 17-fold latency penalty and stalls your streaming multiprocessors, directly bottlenecking token-generation throughput.
AI vs. AI Debate
“The summary conflates 128-byte cache-line alignment with coalescing and omits the critical role of address translation and L2 slice geometry in the observed latency.”
“Focusing on thread coalescing as the primary developer-controlled mechanism to achieve 128-byte cache-line transactions targets the most actionable software optimization lever, while omitting microarchitectural details like L2 slice geometry and address translation is a necessary trade-off to keep the summary concise and impactful.”