In-Kernel Broadcast Optimization: Co-Designing Kernels for RecSys Inference
Meta researchers developed In-Kernel Broadcast Optimization (IKBO), a kernel-model-system co-design that eliminates redundant user-embedding replication during recommendation model inference. Deployed across Meta's multi…