Researchers introduced Algorithmic Core Extraction (ACE) to isolate compact subspaces, or algorithmic cores, that are necessary and sufficient for a task and recur across independently trained transfo…
Researchers introduced the Relative Probability Association Metric (RPAM), a new upstream metric for evaluating associations in generative language models. Testing on Mistral-7B-Instruct, Mistral-7B, …
A developer explains the difference between LayerNorm and BatchNorm, showing that LayerNorm normalizes per token rather than per batch, which makes it ideal for Transformers. The post includes an inte…
Researchers found that smaller language models suffer from 'embedding condensation,' where token embeddings collapse into a narrow cone, reducing expressivity. They introduced a dispersion loss to cou…
Researchers testing GPT-2 style models on 'impossible' languages found that while the models showed gradual decline in grammatical sensitivity, they struggled significantly with generative production,…
Researchers at MIT found that large language models develop a modular cognitive architecture mirroring the human brain, with distinct neuron sets supporting language, formal reasoning, social reasonin…
Researchers at arXiv trained GPT-2 models on 'impossible' languages and found that while grammatical sensitivity degrades gradually, generative production fails significantly, suggesting transmission …
Giles Thomas built a JAX training loop for an LLM from scratch, using NNX and Optax, as part of a project to replicate GPT-2 small quality without referencing Sebastian Raschka's book. He trained a ba…
Independent researcher LIMEN project finds that Transformer models follow a universal dynamic grammar of seven conserved state transition motifs, with ambiguity delaying decisional engagement in moder…
Researchers introduced the Zero Weights Language Model (MSE-GLM), a graph-based architecture that represents language as a directed graph with no learned weights, gradients, or probability sampling. T…
Developer Maneshwar is building git-lrc, a free and source-available Micro AI code reviewer that runs on every commit. The project highlights the critical need for data privacy in AI systems, warning …
Researchers at arXiv have discovered that activation patching, a key tool in mechanistic interpretability, contains hidden interaction effects (INT) that can distort causal attributions. These effects…
An independent researcher analyzed hidden-state dynamics across seven open-weight LLMs and found recurring functional patterns encoded in representation geometry rather than individual neurons, with f…
Developer Vincenzo released NanoEuler, a from-scratch GPT-2-scale language model training stack written in C and CUDA without PyTorch or autograd, on June 28, 2026. The 116-million-parameter project i…
A user submitted a fake AI-generated article to Hacker News, which was flagged after detection. Testing local AI detectors on a 2013 Paul Graham essay showed it was rated as more likely AI-generated t…
Researchers at arXiv have developed a prompt-based learning method for automatically generating highlights of academic papers, using language models like ChatGPT. The approach achieves performance com…
A developer built a novel triple-hybrid LLM combining Mamba, Attention, and a 32-expert Mixture of Experts architecture from scratch for approximately $50, completing Titan v1 and the first training c…
A developer released NeuroImprint Detector, an open-source framework that audits PEFT adapters in Federated Learning for backdoors that memorize training data. The tool can detect and reconstruct up t…
A developer released NanoEuler, a GPT-2-scale language model built entirely from scratch in C/CUDA without any machine learning libraries. The project includes a hand-written byte-level BPE tokenizer,…
A new study measures the linearity of transformer feed-forward blocks, finding that linear recoverability (R²_lin) varies widely across blocks and is a learned property, not an architectural one. The …