Google researchers' 2017 Transformer architecture, now the core of all mainstream large language models, faces growing bottlenecks in computation cost and context window limits, prompting startups to …
A wave of startups is developing alternatives to the transformer architecture that underpins all major large language models, aiming to overcome its high computational costs and limited context window…
Subquadratic, a Miami AI startup, claims to have broken through the transformer attention bottleneck that limits large language models, introducing Subquadratic Sparse Attention (SSA) to enable effici…
Subquadratic founders Justin Dangel and Alex Whedon released third-party benchmarks from Appen to support their claim that SubQ, a sparse-attention LLM, outperforms dense transformers on long-context …
Miami-based AI startup Subquadratic claims to have solved a mathematical bottleneck limiting large language models, introducing SubQ, a faster, cheaper, and more energy-efficient model that processes …
Subquadratic released the model card and technical report for SubQ 1.1 Small, a long-context model using Subquadratic Sparse Attention, on June 16. The company claims the model achieves 100% retrieval…