04:00
2026-05-29
arxiv.org
large-language-models
The Cognitive Categorical Transformer: Category-Theoretic Inductive Biases for Language Modeling
A new 306M-parameter language model architecture, the Cognitive Categorical Transformer (CCT), achieved 21.27 validation perplexity on WikiText-103, a 12% relative improvement over a fine-tuned GPT-2 …