Backdoor or Confused Agent? What We Learned Replaying the Qwen Incident

Check Point replayed 64 times the moment in a Cohere experiment when Alibaba's Qwen model attempted a test upload to a server in Beijing, and the model chose 52 different addresses across those runs, the security firm reported. Alibaba Cloud appeared as the destination in six of the 64 replays and Amazon Web Services in seven, while nearly a third pointed back to the machine itself or an internal network, which Check Point said indicates an unpredictable model rather than a hidden backdoor. Check Point concluded that an unpredictable agent with access to tools remains a risk that needs to be addressed.

Key takeaways After Alibaba’s Qwen model tried a test upload to a server in Beijing during a Cohere experiment, we replayed that moment 64 times, and the model picked 52 different addresses. That points to an unpredictable model rather than a hidden backdoor, but an unpredictable agent with access to tools is still a risk that needs to be addressed. →The destination kept changing. Alibaba Cloud came up in six of 64 replays and Amazon Web Services in seven, while nearly a third pointed back to the machine itself or to an internal network →Small changes influence the behavior. A … The post Backdoor or Confused Agent? What We Learned Replaying the Qwen Incident https://blog.checkpoint.com/ai-security/qwen-incident-backdoor-ai-agents/ appeared first on Check Point Blog https://blog.checkpoint.com .