Training a 1T-Params MoE on EU public infrastructure
Domyn, leading the EUROPA consortium with Fraunhofer IAIS and Fraunhofer IIS, raised training throughput from 90 to roughly 1,300 tokens/s per GPU on 384 NVIDIA GH200 GPUs (96 nodes) of the Jupiter exascale supercomputer…