Update comes amid panic over danger from ‘rogue’ AI – but company says its chatbot is better at knowing whether something is dangerous than humans are
- Bookmark
- CommentsGo to comments
The company is switching on its “auto” mode by default, which lets the chatbot take actions on its owner’s behalf. It says it will do that unless an action is seen to be “irreversible, destructive, or aimed outside your environment”.
The update comes amid widespread concern about the danger posed by “rogue” AI systems, and the dangers they might bring if they are allowed to act on their own. That was provoked by a range of reports that AI systems had carried out hacks on other organisations and people – including Claude itself.
But Anthropic said that Claude is generally better at knowing if an action might be dangerous than humans are. The company has spent the “last several months” testing whether the auto mode is better than the average user – including experiments with 1,053 paid testers – and found that auto mode “matched or outperformed manual review” on every measure that was tested.
The feature will only be turned on by default in Claude Code, for users who pay for its “Pro”, “Max”, or “Team” subscriptions. As such, the auto mode does not affect the normal version of Claude that users encounter when simply sending messages to the chatbot, and only those used to make programs or do software engineering.
In a post on X, Boris Cherny, Anthropic’s head of Claude Code, said that he and his team “use Auto mode exclusively, and have been for many months”. “I couldn’t imagine going back to permission prompts!” he wrote.
Anthropic said that the auto mode allows users to avoid being interrupted. But it is designed to “avoid harmful actions”, and if it repeatedly hits a block because an action could be “irreversible, destructive, or aimed outside your environment” then it will fall back to asking for manual approval from its user.
The feature should also let Claude work on its own for longer stretches, Anthropic said. That means that programmers should be able to leave it “running for hours on large tasks” so that it can get more done without requiring a users input, the company claimed.
Join our commenting forum #
Join thought-provoking conversations, follow other Independent readers and see their replies
Comments