Delayed Tensor Parallelism for Faster Transformer Inference
Kog Team researchers introduced Delayed Tensor Parallelism (DTP), a Transformer architecture that hides communication overhead behind computation and weight streaming to accelerate batch-size-one infe…