SubQ: The First Fully Subquadratic LLM — Cost Comparison with Transformers [2026]

A new LLM architecture called SubQ achieves fully subquadratic attention, reducing computational cost from O(n²) to O(n log n). This breakthrough eliminates the scaling bottleneck that forces models to break with long inputs, potentially replacing RAG and chunking for large-context tasks.

There is a fundamental problem with all the large language models you use today — and it has to do with high school math. The heart of the Transformer is attention : each token needs to compare itself with every other token. A text of 1,000 words requires 1 million comparisons 1,000² . A text of 1 million tokens requires 1 trillion comparisons . The computational cost grows with the square of the context — O n² . It is the reason why models “break” with very long inputs, why we use RAG, chunking, and agents instead of simply giving the entire document to the model.