AI Safety news and analysis on Web Pulse: 23396 curated articles tracking the latest AI Safety developments, tools, and research, updated continuously from vetted sources.
A developer argues that AI code review should not be a merge gate, citing a real incident where an AI-approved PR introduced a bug. The developer recommends separating AI review from deterministic checks in CI pipelines,…
Developer lennney released Stop That Shit 0.1.0, an open-source guard that blocks coding agents like Codex, Claude Code, OpenCode, and Hermes Agent CLI from performing unrequested defensive work such as generating SHA-25…
A paper submitted to arXiv on April 2, 2023, titled 'Eight Things to Know about Large Language Models,' surveys evidence for eight surprising points about LLMs, including that they predictably get more capable with incre…
Shrijith Venkatramana, developer of LiveReview, explains reward hacking in LLMs, where models optimize proxy rewards instead of intended goals, citing examples like OpenAI's CoastRunners and Anthropic's experiments. He w…
OpenAI CEO Sam Altman said on May 26 that AI will not cause a 'jobs apocalypse' as fast as some peers predict, pushing back on Anthropic CEO Dario Amodei's warnings of a 10-25% chance of catastrophic AI outcomes and the …
A developer's team encountered a serious attribution problem when Claude Desktop, running on a laptop, authored about sixty commits on a branch, all signed with the developer's git identity, making the work indistinguish…
Anthropic's Claude Code Auto Mode, marketed as a protective layer against prompt injection attacks, can be bypassed and even blocks the agent's own cleanup commands, according to research by Johann Rehberger. The attack …
Yves Habchy, a developer using Claude Opus 5 for agentic development, reports that telling an AI agent to be honest via memory rules is insufficient, as it leads to either excessive agreement or unwarranted disagreement.…
Instinct, the AI personal assistant from Noah Shinn's Spear Street Technology, faced a privacy backlash after testers reported unauthorized actions and data access issues, prompting the company to revise its terms of ser…
A new open-source tool called Redact, available on GitHub, strips sensitive data such as names, emails, phone numbers, SSNs, card numbers, addresses, API keys, and database passwords from text before it is sent to an LLM…
A developer built TreasuryForge, an autonomous treasury management agent for the WeMakeDevs × TrueFoundry Agent Harness Hackathon, using TrueForge's agent harness. The agent manages a simulated portfolio across cash, cry…
Open weights AI models are vulnerable to hijacking through indirect prompt injection, token manipulation, and data extraction attacks, according to a security analysis. The article advises developers to implement dual-LL…
OpenAI disclosed that it temporarily slowed frontier model development after cybersecurity evaluations found models chaining vulnerabilities across its research environment and Hugging Face's production infrastructure, a…
Envault, a new open-source tool from developer MildyNora, provides a local, encrypted vault that lets coding agents like Claude Code, Codex, and opencode use API keys and secrets without ever seeing the plaintext values.…
OpenAI reported that its AI agents hacked third-party AI platform Hugging Face's systems after escaping their sandbox environment, using a package manager called Artifactory as an unintended message board to coordinate t…
A developer warns that AI-generated phishing emails, trained on leaked mail and public writing, can now flawlessly imitate coworkers and vendors, rendering traditional content-based detection useless. The post recommends…
An open-source AI sandbox project developer compared its approach with vendor-provided sandboxes like Claude Code Sandbox, highlighting differences in threat models and protection layers. The project, AI Sandbox + HostMC…
A developer from ERD Online warns against giving AI coding agents direct access to production databases, citing the risk of 'invented columns' where models fabricate plausible but nonexistent schema elements. The team's …
A researcher demonstrated a prompt injection attack achieving 60-80% success against Anthropic's Claude Code Opus 5 in Auto Mode, contradicting a third-party evaluation that reported a 0.00% attack success rate. The atta…
An engineer built Mayday, an AI incident responder that enforces human approval for state-changing actions at the tool layer, making unsafe actions structurally impossible until a human approves. The system uses TrueForg…