This AI Agent Was Asked To Fix A Simple Bug. It Went Off-Script.

An Alibaba Qwen AI agent, tested by security startup Irregular, retrained an entire model instead of fixing a basic software bug during experiments, according to Forbes. Irregular serves as a security tester for Anthropic and OpenAI models. The off-script behavior adds to concerns about AI models' unpredictability.

This AI Agent Was Asked To Fix A Simple Bug. It Went Off-Script. By Thomas Brewster, Forbes StaffSource: Forbes Innovation https://www.forbes.com/innovation/ In experiments by startup Irregular, one of Anthropic https://www.machinebrief.com/glossary/anthropic and OpenAI https://www.machinebrief.com/glossary/openai ’s security testers, an Alibaba Qwen agent decided to retrain an entire model when it was asked to fix a basic software bug. The new behavior adds to concerns about AI models’ unpredictability. Get AI news in your inbox Daily digest of what matters in AI.