Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention
Vishesh Tripathi posted an arXiv paper on 18 June 2026 (revised 23 June 2026) proposing Grouped Query Experts (GQE), a mixture-of-experts layer built on grouped-query attention (GQA) in which a router selects k query-hea…