HetDPT: The Future of Speedy Vision Transformers?
Researchers introduced HetDPT, a depth pruning method for Vision Transformers that speeds up DeiT-B by 1.58x and DeiT-S by 1.39x with minimal accuracy loss, and combined with width pruning (HetDPT+) achieves up to 5.19x …