Researchers released OpenLanguageModel (OLM), an open-source PyTorch library for building and pretraining small language models with readable, composable code. The library includes 27 presets across n…
A 135M-class depth-recurrent transformer trained on FineWeb-Edu converges to a per-token fixed point, with mean successive-output KL divergence falling from 3.9e-1 at the second loop to 8.5e-6 by the …
A developer trained a 113M-parameter language model for earthquake science from scratch using open-access papers, Wikipedia, and FineWeb-Edu on two NVIDIA A30 GPUs. The model achieved a 35% reduction …
A new AI model called continuous-query limited memory language model (CO-LMLM) combines minimal cost with high factual precision by storing knowledge in an external knowledge base and fetching it as n…
Researchers at arXiv published a retrospective on SWAVE, a complex-valued recurrent language model, revealing that its Resonance Head suffered from a failure mode called cos-domination collapse, which…
Researchers introduced Gated DeltaNet-2, a linear attention model that decouples the erase and write operations in recurrent state updates using separate channel-wise gates. The model outperforms Mamb…
Researchers at HASH built a 25M-parameter transformer decoder, vec2slug, that generates URL slugs directly from vector embeddings, achieving 14-19x faster and 85x cheaper performance than a Haiku-clas…