High-throughput processors, like GPUs, fundamentally depend on high-bandwidth DRAM to feed their parallel compute engines. To maximize effective bandwidth, modern GPU memory controllers sort incoming requests to exploit DRAM row-buffer locality across interleaved streams from parallel compute engines. Historically, this deep sorting has improved both bandwidth utilization and energy efficiency by reducing unnecessary DRAM row activations.
In this work, we show that the shifting access patterns of modern ML and HPC workloads, combined with improving DRAM access energy efficiency, are eroding the benefits of these deep-sorting memory controller designs. By combining architectural simulation with high-level synthesis, we demonstrate that simple energy-optimized memory controllers can achieve near-parity performance with prior complex sorting controllers for many important datacenter workloads. Such ultra-efficient designs are increasingly critical as emerging technologies like high-bandwidth stacked 3D DRAM will likely render the overhead of traditional sorting memory controllers prohibitively large for tightly integrated GPU systems.
Drawing on these insights, we first present PARE, an efficiency-optimized GPU memory controller that removes row-aware transaction sorting entirely. PARE remains close to a sorting controller for most workloads, but exposes losses in two bandwidth-sensitive kernels caused by premature row-miss commitment. We then present MS-PARE, which retains only a narrow row-hit check to avoid this hazard. Relative to the conventional sorting controller, MS-PARE reduces area by 73.3% and average controller energy across 14 workloads by 55%, while maintaining near-parity performance and supporting the extreme DRAM bandwidth demanded by next-generation GPU architectures.