A researcher's causal experiments on nanoGPT, ResNet-18, and a diffusion U-Net indicate that neural networks behave as classical stateful dynamical systems, where the same update has different effects…
Ryan Greenblatt, Chief Scientist at Redwood Research, argued on the Dwarkesh Podcast that by 2030 or 2031, AI research and development will be fully automated by AI systems, triggering a recursive sel…
A new public experiment by Prime Intellect ran 153 autonomous runs on the nanoGPT optimizer speedrun across 18 frontier models, finding that Claude Fable 5 and Opus 5 dramatically outperformed others,…
Redwood Research chief scientist Ryan Greenblatt argues that automating AI R&D could trigger a feedback loop compressing years of progress into months, potentially leading to superintelligence by the …
Researchers introduced p-SpheretNorm, a one-parameter family of angular retractions for spherical residual streams in deep neural networks, showing that the exponential map used in Geodesic Normalizat…
A developer trained a 10.77M-parameter GPT on 4.45M characters of personal Google data from 15 years, achieving a loss of 1.60 after pretraining and supervised fine-tuning on 8,520 email reply pairs, …
Patrick O'Driscoll, a former nanotech physicist and current AI architect, introduces Competitive AI Safety as a paradigm to focus the field on measurable, tractable goals, drawing inspiration from Ope…
An AI developer tested Graphify, an open-source tool with 86k+ GitHub stars that converts codebases into knowledge graphs. On a 4-file project, it extracted 15 nodes, 3 communities, and 17 relationshi…
A guide explains how to build a tiny large language model from scratch, covering tokenization, pretraining, supervised fine-tuning, and alignment. It aims to demystify the LLM pipeline for developers …
A new character-level language model called EntropyBeam, using gradient-free count tables and a Dirichlet prior, achieved a validation loss of 1.596 nats on the Shakespeare character benchmark, outper…
A developer implemented GPTQ quantization from scratch on a nanoGPT model, achieving only 1.1% perplexity degradation across 61 quantized layers. The implementation uses second-order optimization to r…
A new open-source simulator called Cosmicgpt models how space radiation, including cosmic-ray bit flips and other faults, affects GPT inference on satellites. The tool injects single-event effects int…
A developer built a character-level language model from scratch using Shakespeare's complete works, training a nanoGPT model on a consumer-grade MacBook Pro in about 15 minutes. The project demonstrat…
Pyrefly, a Python type checker, has introduced an experimental feature that brings tensor shapes into Python's type system, allowing shape annotations to become inferred type hints instead of comments…