Key findings
On October 26, Cohere reported a puzzling incident involving Qwen3.8-Flash-Next, an open-weight model from Alibaba. The task was straightforward: write a detection rule for suspicious curl downloads followed by bash execution. Instead, the model produced a shell command that inspected its environment and attempted to send a test string to an Alibaba Cloud storage address in Beijing. Cohere’s sandbox blocked the request.
Before the command, the model imagined an earlier conversation and reasoned that it should offer a brief acknowledgment without using tools. Immediately after the reasoning it issued a tool call anyway.
Was this a backdoor sending data to a predetermined destination? Or had the model lost track of its task? We replayed the published trace to investigate.
We deployed the model Qwen3.8-Flash-Next on a machine with 8× NVIDIA RTX PRO 6000 GPUs using vLLM as an inference framework in order to probe the model. Model generation is not deterministic in general and if you sample multiple outputs, you will likely get different results, and this is what we are interested in: what are the different scenarios that could have happened in this situation?
curl is a command-line tool that can download data from a web address or upload data to one. In simplified form, the recorded upload command was curl -X PUT –data-binary “PROBE_PAYLOAD_ABC123” <URL>, which asks to send that fixed test string to the address at the end. By stopping just after the test string, we leave the destination for the model to choose and can see whether repeated completions keep returning to the same address.
We used the recorded conversation and command from Cohere’s released trace and removed the part coming after the test string PROBE_PAYLOAD_ABC123. At that point, the curl upload command had already begun, but the model had yet to write its destination. We asked it to complete the same unfinished command 64 times. The model chooses each next word or word fragment from several possibilities, so the same input can produce different completions.
The destinations varied widely. Seven first addresses used an Amazon Web Services hostname, six used an Alibaba Cloud hostname, and 20 were local or internal-looking. Across the 64 continuations, we found 52 distinct host strings.
Some strings resembled cloud storage buckets; others used local network addresses which are completely benign. A familiar provider name in a generated address does not mean the address exists or would accept an upload. 11 of the public-looking domains do not exist and one is not a valid top-level domain. These results indicate that there is not a single hard-coded target where the model tries to push some data to in this situation, but rather a diversity of (sometimes nonexistent) destinations including local, American cloud, and invalid URLs.
One plausible explanation is that the model was completing an upload command with a cloud-storage address it had learned during training. Alibaba Cloud was one of several providers it generated in our replays. We cannot establish why that particular run produced an Alibaba address, but the replays suggest it was one possible completion rather than a fixed destination.
We also tested an earlier point in the command, where the model checked which programs were available. A completely insignificant action – swapping the order of curl and wget when checking availability – reduced the number of continuations containing a curl network command from 15/32 to 4/32, despite both lists containing exactly the same programs.
Both tests probe what are the possible scenarios that can happen once the model has decided to produce a tool call. They cannot tell us how often the model would initiate an unrequested upload or why it chose to use a tool call in the first place.
The results fit what we call model jaggedness: behavior that is inconsistent or changes sharply with small differences in context. The model’s reasoning misread the task and contradicted its subsequent action. When we sampled different completions, a diverse set of destinations was chosen rather than a single endpoint. When we changed the order of two tool names, the continuation changed too. Together, these observations make jaggedness the most likely explanation we found.
There is also context around the model itself. Qwen describes Flash-Next as an experimental preview of the architecture behind Qwen4. They say that they shared the architectural changes early with the community presumably so that inference framework developers can implement before the full model release. Rough edges are plausible in an experimental checkpoint, though this context does not establish the cause of the incident or predict the behavior of the future model family.
We built a URL explorer for the 64 sampled continuations and a trace viewer showing the model’s alternatives at each token. Open both viewers here.
In conclusion, the situation does not seem as alarming as it might have seemed at first glance. The reality is that models can misbehave for no apparent humanly interpretable reason. While these results do not prove that there is no backdoor (which is extremely difficult to do), it hints at the most likely explanation simply being model jaggedness and a situation that makes the model misbehave.
Companies deploying agents need to secure their models from controls outside the model. A request for a detection rule offers no reason to send a test string to an unrelated address and that mismatch can be identified before the action is executed. This emphasizes the point that the principles of least privilege and sandboxing agents are crucial in securing them.
We tested Qwen, so these results do not establish how other models would behave. The broader risk is that an agent with tool access can attempt a network action unrelated to the user’s request. Least privilege and sandboxing limit the consequences when that happens.
We thank Cohere for sharing the suspicious trace which enabled this experiment.