18:14
2026-09-21
newsletter.semianalysis.com
ai-infrastructure
Computation and Data Movement for Inference
Mixture of Experts has changed the structure of AI inference serving by altering which tensors are active per token, what must remain close together, which transfers need strong local bandwidth, and h…