Microsoft researchers introduced LoRA (Low-Rank Adaptation) in 2021, a parameter-efficient fine-tuning technique that reduces trainable parameters by approximately 10,000 times and GPU memory requirem…
AI scaling laws, first published by OpenAI in January 2020, show that model performance improves predictably with scale, contradicting the narrative of unpredictable AI progress. The laws, which hold …
A Forbes analysis by contributor John Werner finds that AI's environmental costs vary widely, with data centers consuming significant water and energy, but smarter placement, scheduling, hardware, and…
In a blog post, an unnamed author argues that AI performance will not plateau, countering the common belief that progress is driven by brute-force scale with diminishing returns. The author contends t…
Jie Tang's team at Z.ai argues that LLM scaling should optimize for fixed inference budgets, favoring smaller, deeper architectures trained longer over wider models. Their ablation shows a 7B model tr…
Eron, in a post on X, explains that large language models like ChatGPT, Claude, and Gemini operate by repeatedly predicting the next word in a sequence, a simple task that, when scaled to billions of …
Generalist AI, a Cambridge, Massachusetts-based startup founded by former Google DeepMind and Boston Dynamics engineers, demonstrated robots that learn new tasks from short instructional videos withou…
Unigram, a new Rust crate released on crates.io, encodes bytes as words that each cost exactly one LLM token, making an N-byte value cost exactly N tokens under Claude and other models. The bijective …
OpenAI CEO Sam Altman admitted the past year was difficult and largely his fault, but predicted the next 12 months will be the company's best yet, according to a conversation he had. Altman revealed t…
Scaling machine learning models is more complex than simply increasing size, requiring careful engineering to avoid pitfalls, according to a technical essay by Beren Millidge. The author explains that…
A new LessWrong post by Evan R. Murphy proposes applying a red team vs. blue team framework to AI evaluations, arguing that current evaluation methodologies fail to account for models that can subvert…
Redwood Research chief scientist Ryan Greenblatt argues that automating AI R&D could trigger a feedback loop compressing years of progress into months, potentially leading to superintelligence by the …
Leopold Aschenbrenner's prediction of AGI by 2027 faces skepticism as scaling laws show diminishing returns, with the jump from GPT-3 to GPT-4 being a seismic shift but later iterations offering only …
Quantum computing, which uses qubits to explore multiple states simultaneously, is emerging as a business priority because it can solve problems intractable for classical computers, including AI train…
In July 2020, founder Sharif Shameem posted a video showing GPT-3 building a functioning React app from a plain English description, a moment widely cited as an early sign of the AI coding wave. Six y…
A developer recounts building a mail classifier with GPT-3 before structured output and tool-calling existed, using a custom pipeline that generated JSON schemas and repaired outputs, and argues that …
In a retrospective essay, AI forecaster and author of the 2021 science fiction story 'Bayeswatch' reflects on how the AI landscape has changed, noting that AI now writes all of his software at 10× to …
Prompt injection remains possible in large language model applications because models cannot distinguish between trusted instructions and untrusted user input in the token stream, according to a techn…
Ilya Sutskever, co-founder of Safe Superintelligence, argued at NeurIPS in December 2024 that the scaling era of AI pre-training is ending because high-quality human-generated data is finite, not beca…
OpenAI is offering academic researchers up to $1,000 in API credits per approved application through a structured program targeting AI safety, ethics, and societal impact research. The credits are val…