A research team posted arXiv:2610.02462v1, a framework for predicting capability loss from LLM compression across pruning, quantization, and distillation, measured in mathematics, code generation, and…
A new arXiv paper (2609.35852v1) analyzes Transformer weights at a mesoscopic level using row and column scale fields — median-centred log-RMS profiles over a matrix's channels — which, with a global …
A new arXiv paper (2609.10830v1) reports that language models' apparent memory of training data is largely confounded, using exact duplication counts from the published pretraining corpora of OLMo-2 a…
A new arXiv paper (2609.05448v1) formulates structured post-training pruning of vision and language transformers as a damage-aware multi-armed bandit problem under a fixed candidate-evaluation budget,…
A developer detailed a 17-day project building an 'epistemic gate' to prevent LLM data poisoning during fine-tuning, culminating in the Beatriz architecture. The work, conducted on a 2006 Toshiba, evo…
A new technique called canonical-basis realignment rotates a Transformer's internal coordinate system into a lossless canonical basis, enabling every hidden axis to be independently measured and contr…
Most AI models marketed as 'open' are open-weight rather than open-source, and the deciding factor is the licence, according to the Open Source Initiative's OSAID 1.0 standard. Open-weight models like…
A new arXiv preprint (2608.02829) finds that converting a pretrained 1.4B-parameter Pythia model into a 410M-parameter sibling works best through initialization transfer, not weight projection. The au…
A new study from arXiv preprint 2607.24797v1 finds that decoder-only large language models (LLMs) such as GPT-2, OPT, and Pythia entangle reading and writing into a single code, unlike the human brain…
A new study finds that backdoor triggers in large language models often fail to reach the model due to context pipeline preprocessing, invalidating many robustness evaluations. In about 1 million tria…
A developer built a free web tool that visualizes and edits the internal representations of open-source language models, inspired by Anthropic's research on the Jacobian Lens. The tool allows users to…
A new study analyzing seven pretrained models reveals that the positional encoding scheme in transformers determines the spectral properties of attention heads, with RoPE inducing rotational spectra a…
Researchers introduced Pollux, a new class of decoder-only LLMs that use native Leech-lattice quantization to achieve 0.76 bits per parameter, compressing a 1B-class model into 76 MB of SRAM. The mode…
Researchers at the University of Oxford and other institutions found that large language models (LLMs) develop false-belief task (FBT) performance late in pretraining, dependent on model size and trai…
A new measurement framework reveals that eight open-source transformer LLMs partition into two distinct clusters based on runtime probability distribution dynamics, with an order-of-magnitude gap in G…
Researchers introduced trajectory extrapolation error, a measure of how much a language model's hidden states deviate from a linear path during word processing, and found it independently predicts hum…
Researchers have developed a bifurcation theory that detects when neural networks acquire structured representations during training in real time, without relying on retrospective or label-dependent m…