Reinforcement Learning from Human Feedback (RLHF) is the primary mechanism that teaches large language models (LLMs) to refuse inappropriate requests, with models like GPT-4 showing greater jailbreak …
LLM red teaming is an adversarial testing methodology that intentionally provokes large language models into producing incorrect, biased, harmful, or insecure outputs to identify vulnerabilities befor…
Open weights AI models, predominantly from China, threaten the dominance of frontier labs like OpenAI and Anthropic, which have grown revenues rapidly but may be disrupted before capitalizing on their…
Agentic AI offers writers a new form of leverage through Specification-Driven Authoring (SDA), a workflow that keeps the writer in control while structuring AI collaboration for planning, drafting, ve…
A developer tested whether repeatedly asking an AI to "think" before fixing the same Python bug improves debugging performance, finding that the benefit of reasoning prompts like "Let's think step-by-…
Increasing LLM context windows beyond 1M tokens does not eliminate hallucinations but changes how they manifest, according to a technical analysis of context-driven errors. The analysis identifies fou…
Elon Musk predicted in an August 2020 tweet that GPT-5 or GPT-6 could become indistinguishable from the smartest humans, extrapolating from the rapid improvement seen with GPT-3. Musk, who had left Op…
OpenAI's o3 model scored 87.5% on the ARC-AGI benchmark in December 2024, up from GPT-4o's 5%, by using 5.5 billion tokens of inference-time compute instead of scaling model parameters. The benchmark,…
GLM 5.2, a 744-billion-parameter model, runs inference on consumer-grade hardware by activating only 40 billion parameters per token using Mixture of Experts (MoE) architecture. The MoE routing mechan…
OpenAI CEO Sam Altman told the board in an October 2022 email that the company should soon release a GPT-3-class language model that runs locally on consumer hardware, arguing the move would discourag…
University of Illinois Urbana-Champaign researchers have released an open-source AI teaching assistant on HuggingFace that runs 11 separate models in parallel for text/image retrieval, generation, mod…
A new database from Startups.RIP tracking 105 Y Combinator founders shows that OpenAI and Anthropic are the top destinations for founders whose startups were acquired or shut down, with Sam Altman sti…
GitHub launched a technical preview of GitHub Copilot, an AI pair programmer that suggests whole lines or entire functions as developers type. Developed with OpenAI and powered by OpenAI Codex, the to…
Ray's new Resource Isolation feature, built using Linux kernel control groups (cgroup-v2), delivers up to 1.5x speedup and eliminates node deaths and job failures on previously unstable workloads such…
Researchers have found that large language models (LLMs) can generate diverse software code, potentially reducing common-mode failures, but reliability improvements depend on programming language and …
In mid-2021, while the tech world focused on the Metaverse and NFTs, a small group of developers and researchers in Silicon Valley had early access to GPT-3 and GitHub Copilot and realized the transfo…
GPT-3 is not AGI but language modeling differs from other AI domains because natural language encodes a world model. Scaling language models may drive loss toward Shannon entropy, yielding a usable wo…
AI token costs have dropped 600-fold in six years, yet companies are burning through annual AI budgets in a single quarter due to increased demand and more complex agent architectures. Nvidia CEO Jens…
Scaling laws predict training loss as model size, dataset size, and compute scale, but their practical application is sensitive to hyperparameter choices like learning rate. Lilian Weng's post highlig…
The carbon footprint of AI is projected to grow at a CAGR of nearly 44% through 2025, with computational costs doubling every few months. Training a single large NLP model can emit as much carbon as f…