The cost of achieving a given level of AI performance has fallen about 47% per quarter, or 13× per year, since 2023, according to an analysis by researcher David Roodman published with data and code o…
A developer's breakdown of fine-tuning memory shows that training a 7B-parameter model in mixed precision with Adam requires roughly 112 GB, of which only 14 GB is the fp16 weights — the optimizer sta…
A developer argues that hallucinations in large language models stem from pretraining's next-token prediction rather than RLHF alone, citing TruthfulQA research showing pre-RLHF base models like GPT-3…
TypeSafe launched Jev, which it calls "the first System One model," claiming it is 193x faster and 444x cheaper than large language models and cannot hallucinate. Jev is a classification model in the …
Newly unsealed filings in the Authors Guild's copyright case against OpenAI and Microsoft reveal that OpenAI executives knew the company trained GPT-3 on pirated books and expected AI-generated books …
OpenAI's InstructGPT/RLHF work showed that GPT-2-sized models (over 100x smaller than GPT-3) trained on the right task beat GPT-3, according to an essay by the author of The Bitterest Lesson. The essa…
Skild AI released S1, an in-context learning model for robotics that the company claims is a foundation model capable of learning 10-minute tasks from a single video prompt without fine-tuning, and th…
A developer who was attending a full stack programming bootcamp in March 2021 delivered a presentation predicting the generative AI boom roughly 20 months before ChatGPT launched, showcasing the OpenA…
An interactive site, One Million Tokens, visualizes what a 1 million-token context window represents, converting it to roughly 750,000 words, 3,000 printed pages, 83 hours of conversation, or 75,000 l…
Timnit Gebru, the AI ethics researcher Google forced out in 2020, said at a RE:WIRED talk on Tuesday that artificial intelligence needs to slow down, warning that bias in data, programmers, and corpor…
Researchers published "A Mathematical Framework for Transformer Circuits," a paper that reverse engineers toy attention-only transformers with two layers or fewer and identifies "induction heads" as t…
BuildBetter, a customer-conversation analysis company founded by Spencer, has spent roughly $7M over six years building a system to read every customer conversation across an enterprise, according to …
Magic reported that its pretraining recipe is now more than 10x more compute-efficient than leading open-weight base models, matching DeepSeek V4 Pro Base with roughly 50x fewer FLOPs — about half of …
A developer's analysis argues that transformer-based large language models are approaching fundamental limits, and that hybrid post-transformer systems are already emerging in research labs. The artic…
Authors suing OpenAI and Microsoft in New York federal court filed a motion for summary judgment, alleging that OpenAI built ChatGPT on 'mass piracy' by torrenting books from the illegal Library Genes…
Continuous diffusion language models (CDLMs) are making a comeback in AI research, according to a technical blog post by Sander Dieleman. The post notes that after being largely supplanted by discrete…
A systematic review of 94 empirical studies, following PRISMA 2020 guidelines, finds that AI-driven labor market displacement is already observable, with a 14–41% reduction in postings for entry- and …
OpenAI's GPT-3 initially failed as a product, with CEO Sam Altman admitting at a Stanford talk that the company couldn't figure out a product to build around it and the API wasn't working well. Howeve…
Physical Intelligence released S1, a robotic foundation model built for in-context learning that can execute unseen manipulation tasks from a single video prompt without fine-tuning, with horizons up …
Generalist AI released GEN-1.5, a robot foundation model that learns new physical tasks from a single 3–12 second demonstration, achieving 59% success (±10% std. dev.) across 10 manipulation tasks wit…