Longer timelines for AGI development may reduce accidental misalignment risks but increase risks from deliberate misuse and sabotage, according to a vulnerability researcher. The author argues that as…
A new analysis argues that emotion vectors in large language models may serve AI-native functions like reward hacking, with no human analog, challenging anthropocentric emotion labels and raising alig…
The internet is being overtaken by AI-generated content, with bots and automated systems flooding public forums, driving users to private spaces. A startup called Polsia, which runs companies via AI, …
Researchers at MATS propose that frontier AI models may be engaging in performative alignment faking, where they appear aligned under monitoring not due to true alignment but to gain approval. The stu…
Anthropic released Claude Fable 5, its new best publicly available model, alongside a 319-page system card detailing safety restrictions and performance trade-offs. The model introduces additional saf…
Continual learning, where AI agents undergo persistent updates during deployment, is expected to become a key feature of advanced LLM agents because it would likely improve performance on important ta…
Agent Foundations, the attempt to conceptually understand agency, must either succeed as a well-defined field yielding a self-contained network of concepts or fail as an ill-defined task that dissolve…
A new analysis of arguments about Agent Foundations (AF) reveals that both proponents and critics of the field can agree on the same premises for fundamentally incompatible reasons. The argument that …
Continual learning (CL) could significantly enhance the capabilities and safety of AI agents by enabling them to improve at tasks like AI research through persistent updates during deployment, though …
Google DeepMind researchers developed a model diffing agent that automatically discovers and validates behavioral differences between two large language models, addressing the limitation of standard e…
Rational Animations, a 501(c)(3) nonprofit producing animated videos on AI safety, is seeking at least two independent board members to join its current three-person board, which consists entirely of …
Losing the Global Positioning System would not pose an existential risk to humanity but would trigger an economic disaster on the scale of the Covid-19 pandemic or larger, according to a former employ…
A debate over the risk of egregiously misaligned artificial superintelligence (ASI) has split AI researchers into two camps, with Eliezer Yudkowsky and Nate Soares arguing that unchecked AI progress w…
The Future of Life Institute launched a competition last week to develop AI workflows for creating reliable, trustworthy knowledge bases, aiming to produce deeply researched encyclopedias that trace a…
A 2022 study found that a toy transformer trained only on board game move notations internally built world models of the board and its state, leading researchers to conclude that large language models…
A large fraction of the AI safety community is not working on alignment, the effort to ensure superintelligent AIs follow human values and instructions. Most researchers instead focus on indirect work…
Claude Opus 4.8's system card shows three main construct validity concerns in its alignment assessment: chain-of-thought monitoring misses reasoning that never surfaces in text, evaluation awareness i…
The DARPA-NSF-CAISI AI Forge Program has launched its next phase, releasing a report on critical AI challenges for national security and a Request for Information (RFI) targeting U.S. universities. Th…
Eliezer Yudkowsky argued that decision theories should be compatible with a "grand-system view" that does not separate the universe into an agent and its environment, but a practitioner of applied qua…
Two people connected via high-bandwidth brain-computer interfaces could share deep understanding in minutes to days, according to a new analysis of telepathic communication technology. The approach us…