The VRAM Wall: NVIDIA H200 vs. AMD MI325X for Massive LLMs
A technical comparison of the NVIDIA H200 and AMD Instinct MI325X finds that the AMD chip's 256GB of HBM3e memory and 6.0 TB/s bandwidth let it host larger shards of 400B-parameter and 671B MoE models…