Zyphra AI Introduces Zyphra Hierarchical Attention (ZHA): A Path to True Subquadratic Attention

Zyphra AI introduced Zyphra Hierarchical Attention (ZHA), a tree-indexed sparse attention method that retrofits frozen LLMs using 4B tokens of distillation. Zyphra reports ZHA matches dense attention on reasoning and 256K retrieval benchmarks, positioning it as a path to true subquadratic attention.