A new study finds that backdoor triggers in large language models often fail to reach the model due to context pipeline preprocessing, invalidating many robustness evaluations. In about 1 million tria…
A developer built a free web tool that visualizes and edits the internal representations of open-source language models, inspired by Anthropic's research on the Jacobian Lens. The tool allows users to…
A new study analyzing seven pretrained models reveals that the positional encoding scheme in transformers determines the spectral properties of attention heads, with RoPE inducing rotational spectra a…
Researchers introduced Pollux, a new class of decoder-only LLMs that use native Leech-lattice quantization to achieve 0.76 bits per parameter, compressing a 1B-class model into 76 MB of SRAM. The mode…
Researchers at the University of Oxford and other institutions found that large language models (LLMs) develop false-belief task (FBT) performance late in pretraining, dependent on model size and trai…
A new measurement framework reveals that eight open-source transformer LLMs partition into two distinct clusters based on runtime probability distribution dynamics, with an order-of-magnitude gap in G…
Researchers introduced trajectory extrapolation error, a measure of how much a language model's hidden states deviate from a linear path during word processing, and found it independently predicts hum…
Researchers have developed a bifurcation theory that detects when neural networks acquire structured representations during training in real time, without relying on retrospective or label-dependent m…