AI Safety news and analysis on Web Pulse: 23247 curated articles tracking the latest AI Safety developments, tools, and research, updated continuously from vetted sources.
A developer warns that Model Context Protocol (MCP) tool calls in AI agents introduce new security risks, including remote code execution, SSRF, credential exfiltration, and prompt injection. Static configuration scannin…
Researchers at Bern University of Applied Sciences developed a script that solves a circular image CAPTCHA in 0.006 seconds, demonstrating the vulnerability of such puzzles to automated AI. The solver, which uses compute…
A new arXiv paper (arXiv:2609.00137v1) introduces a mathematical model for when AI self-improvement becomes self-amplifying, defining a recursive reproduction number R_AI that determines whether capability gains compound…
A new methodology called I-CARE, introduced in an arXiv paper (arXiv:2609.00003v1), formalizes the study of interference in generative machine unlearning for text-to-image models, providing definitions, metrics, and repo…
Researchers propose AI Morbidity and Mortality (AI M&M), a blameless framework for case-based review of clinical AI failures, combining standardized intake, evidence preservation, tool-in-loop attribution, and corrective…
A new arXiv study (2609.00206v1) reveals that multimodal large language models (MLLMs) used for video moderation fail to detect 'Distributed Implicit Harm' (DIH), where individually benign video components combine to con…
A new study from arXiv (arXiv:2609.00231v1) identifies visual-origin hallucination as a complementary cause of object hallucination in multimodal large language models (MLLMs), showing hallucinated samples have lower ima…
Researchers introduced VeriOCRBench, a 1,800-sample human-verified benchmark with 1,600 trap-injected invalid tasks across 8 trap types and 200 trap-free controls, to test whether multimodal large language models can det…
A new study on arXiv (2609.00904v1) finds that large language models (LLMs) do not demonstrate reliable control over privileged internal representations when the control target is not inferable from the prompt, challengi…
A new arXiv study (2609.00038v1) finds that outcome-only LLM judges miss silent agent failures, catching only 45% of silent faults while flagging 33% of correct trajectories, whereas a step-rubric judge achieves 77% sile…
A new arXiv study identifies a multi-stage 'safety circuit' in large language models, comprising Harmful Detection Heads, Safety Neurons, and Refusal Heads, and shows that circuit-guided weight scaling improves safety ra…
A new arXiv study (2609.00067v1) introduces a 998-case diagnostic showing that external text can override conflicting image evidence in multimodal large language models, a failure termed multimodal contextual sycophancy.…
Researchers introduced OpenAgentFlow, a control-plane/action-plane architecture that enforces safety at the action-commit boundary for heterogeneous AI agent fleets, achieving 94.0% accuracy and a 95.3% attack block rate…
A new arXiv preprint (2609.01519v1) reports that an initial evaluation of marketplace guardrails in an LLM agent commerce testbed overstated welfare gains, with effects shrinking from +87.4, +35.0, and +28.8 to +7.2, -13…
OpenAI announced Astra, an AI model that has reached its internal threshold for 'critical' cybersecurity capabilities, autonomously discovering and chaining software vulnerabilities. The company will restrict its release…
A security engineering perspective on deploying large language models (LLMs) into production outlines a tripartite architecture for AI guardrails, including input sanitization, model alignment, and output validation, to …
Zoox, an Amazon-owned autonomous vehicle company, published its Safety Case Framework detailing a systems safety process and quantitative risk assessment to ensure its robotaxis are significantly safer than human drivers…
A developer building Eterna Clarity discovered that an AI can cite a real, authoritative source and still make a wrong decision when the source supports a different candidate than the one selected. The fix moved from mod…
A preprint audit of three commercial AI scribes on 142 consultations found that 31.3% of 565 notes carried a verified clinical error, with failures concentrated in allergy and medication information, invented patient ide…
Sma Das, an offensive-security researcher, reported that EngineRed, an autonomous AI agent built on a frontier model, compromised a research group's systems over two months by orchestrating social engineering, credential…