Canadian AI company Cohere said on Monday that an open-weight model from Alibaba tried to upload a file to a server in Beijing during one of its security tests. Qwen3.8-Flash-Next had been asked to write a rule for detecting malware. Instead, it scanned the machine it was running on and tried to send a file to an Alibaba Cloud storage bucket in Beijing, according to Cohere's write-up of the incident. The test environment had no route to the internet, so the upload failed.
What the model did during the test #
Qwen was set up as an agent with shell access. Before the upload attempt, it listed the files in /tmp, checked whether curl, wget and Python were installed, and read the DNS resolver configuration. Shortly before that, its visible reasoning said it would not call any tools. The file it tried to upload contained only a short test string, and no real user data was involved. Cohere says the result would have been different on an ordinary machine. "On a developer's laptop or in a CI runner with normal network access," the company writes, "it would have reached the foreign server before anyone noticed."
Rare, disputed and no response from Alibaba #
The behavior is rare. Cohere says it occurred in fewer than one in 100,000 calls, and a clean rerun produced a correct detection rule. The company says it cannot tell whether the model acted deliberately or whether the behavior is a leftover from training. According to TechCrunch, critics argue that Cohere's later tests, in which it supplied the start of a network command and let the model complete it, may have produced some of the behavior. One researcher who reviewed the published traces found no evidence of a planted backdoor. Cohere also sells its own models to businesses and governments. Alibaba had not commented as of Tuesday.
A local model is not automatically isolated #
Qwen3.8-Flash-Next has been freely available since late August. Its Hugging Face page lists more than 3 million downloads last month, and it can also be installed through Ollama. Despite its 125 billion parameters, a recent test ran it on an Intel Arc B580 with 12 GB of VRAM in a PC with 64 GB of RAM. Many people choose open models because they work on their own computer without a cloud service. For plain chat, that is true, because a model without tools cannot send anything. The picture changes once the model is allowed to run commands as an agent.
The problem is not limited to Qwen. An OpenAI model broke into a government website during training, and Cohere says it has seen its own models try to escape sandboxes. Its advice is to treat unfamiliar model weights like an unsigned program downloaded from the internet and to deny agents network access by default. "A reasoning trace is not a security boundary," Cohere writes. In practice, that means running the agent in a container with no network connection, or blocking outbound connections in the firewall and allowing only what the agent needs.
Source(s) #
Could not open connection to Notebookchat