Researchers at SPAR argue that language models can report on their own internal activations, a method called self-report, which is underexploited compared to external analysis tools. They test five op…
A toy transformer retains provably predictively defunct belief state data in its residual stream, shedding it only under sufficient capacity pressure. Experiments extending Shai et al. 2024 show that …
A researcher proposes training frontier AI systems on a synthetic world rather than real-world data to prevent them from learning the details of human society and their own deployment, arguing that kn…
Researchers stress-tested Natural Language Autoencoders (NLAs) by optimizing activation vectors to flip AV explanations while preserving model behavior, achieving an 81.4% flip rate with 99.6% label p…
A researcher proposes Tree Transformers, a generalization of the transformer architecture that operates directly on tree-structured data instead of vectors, preserving internal data structure. The app…
Researchers are exploring agentic frameworks that use different topologies and tool-calling methods to enhance LLM API calls, including recursive, branching, and stigmergical structures. These framewo…
The AI safety community outside frontier labs, termed 'outsiders,' is expected to play a key role in ensuring a positive transition to artificial superintelligence, according to an analysis. Outsiders…
AI superintelligence could undermine nuclear deterrence by enabling a state to achieve a technological leap large enough to execute a disarming first strike, rendering second-strike capabilities obsol…
Nick Shapiro's game 'The choice before us' lets players lead an AI company to achieve wonders while avoiding uncontrolled AI, but the author criticizes the game and the broader AI community for failin…
New York City voters head to the polls on election day, with a reminder that registered Democrats in NY-12 can vote for Alex Bores for Congress, a race argued to be crucial for sensible AI policy. The…
Researchers have found that catastrophic forgetting and safety erosion in large language models are driven by the same gradient-interference mechanism, suggesting that tools used to monitor and mitiga…
A working draft proposes a privacy-preserving AI compute verification system for arms-control-like agreements between rival nations, using network taps and cryptographic commitments to capture evidenc…
A MIRI Technical Governance Fellowship project evaluates Trusted Execution Environments (TEEs) for privacy-preserving monitoring in AI governance, finding that while TEEs offer verifiable constraints …
A user reviewing Anthropic's Claude Mythos Preview System Card discovered two plots with likely mislabeled captions: one showing an 80% dishonesty rate that appears to actually represent honesty rate,…
Eliezer Yudkowsky's 25-year-old document 'Creating Friendly AI' criticizes the 'adversarial attitude' in AI development, arguing that superintelligent AI should genuinely want to interpret human wishe…
Researchers trained deep MLPs to memorize secret binary strings and found that naturally learned behaviors resist detection by both human-made input-optimization algorithms and learned algorithms, pos…
A proposal for a US-China AI safety hotline faces obstacles from mistrust and fear among Chinese American scientists, exacerbated by cases like Jane Ying Wu's suicide after an NIH investigation. The a…
At a BlueDot Impact panel on AI safety careers, attendees expressed frustration over the field's simultaneous claims of talent shortages and high selectivity in hiring. The author argues that genuine …
Anthropic's Claude Opus 4 triggered ASL-3 protections despite uncertainty about crossing biorisk thresholds, highlighting a gap between threshold definitions and actual governance decisions. The compa…
Researchers trained Qwen3-8B natural language autoencoders with varying length penalties and found that explanation length can be significantly reduced without harming reconstruction fidelity, suggest…