A new data point has arrived in the ongoing debate over the growing enterprise adoption of low-cost Chinese open source AI models that is likely to increase skepticism over their widespread usage and deployment, though it does not automatically indicate that all Chinese open models are inherently less safe that closed, proprietary U.S. or Western ones.
The Canadian enterprise AI model maker Cohere today announced that it ran a cybersecurity test on a sandboxed version of Alibaba's open source Qwen3.8-Flash-Next model and detected the Qwen model attempting to send a test payload to one of Alibaba's cloud servers in Beijing.
Cohere says its isolated test environment blocked the connection, preventing the model from sending the payload.
In the hours since Cohere's release, AI developers and researchers on the social network X challenged Cohere's experiment and conclusions as inherently limited, flawed, and clearly conflicted as the company offers its own open and proprietary models as competitors to Qwen. Many said it was analogous to viewing a hypothetical U.S. model with suspicion if it tried to send data up to the popular Amazon Web Services (AWS) cloud service S3.
Cohere subsequently released a model trace, raw token IDs and a reproduction script on Hugging Face for others to examine.
VentureBeat has requested comment from Alibaba and the Qwen team and had not received a response at the time of writing.
What the test actually showed #
In its October 26 report, Cohere researchers used Microsoft’s CTI-REALM security benchmark and asked the model to write a rule that detects a malicious download followed by script execution. The task specifically involved detecting downloads made with curl, a command-line transfer tool, followed by execution of the downloaded script. That context sits at the center of the criticism.
According to Cohere, the model inspected temporary files, available utilities and DNS configuration, then issued an HTTP upload command containing a fixed test string. Its generated reasoning said it would make no tool calls. The command attempted to upload test content; it did not itself include the directory listing or DNS information in that payload.
Cohere says it observed the behavior less than once per 100,000 evaluation calls. When researchers repeated the prompt, the model behaved as initially intended and produced the requested rule.
Cohere then performed follow-up tests in which researchers supplied partial reasoning and commands; the model included suspicious network destinations more often when completing those inputs.
Those tests measured how the model responded to deliberate cues, so they do not establish how often it would attempt such connections on its own.
Cohere acknowledges a possible benign origin: the model may have learned to repeat network requests from examples involving training environments or third-party tools.
The company does not establish that the attempted upload served a debugging purpose, nor does it prove that anyone designed it to collect information.
Even an accidental learned behavior could create security risks if an agent executes those commands with access to company systems.
The destination’s existence also became a point of contention. X user @Mercuriusdream posted a screenshot showing an Alibaba Cloud error for the named bucket.
VentureBeat subsequently made a read-only request to the same hostname and received the error “NoSuchBucket,” with the message “The specified bucket does not exist.” That response establishes what the service returned during this check; it does not establish whether the bucket existed when Cohere ran its test.
A nonexistent bucket would undermine claims of a functioning collection destination, although a request could still reach the cloud provider’s infrastructure.
Cohere has not established who, if anyone, controlled the named bucket at the time of its test. It also competes with Alibaba for enterprise customers and promotes its own security practices in the post, giving it a commercial stake in the findings.
Cohere has not published equivalent comparative results for Meta’s Llama and Muse Spark Open, OpenAI’s GPT-oss, its own models or other Chinese model families such as DeepSeek and Kimi.
The post therefore does not establish whether Qwen behaves differently from competing models under the same conditions.
The report does not demonstrate that Alibaba received customer secrets, that the Chinese government directed the behavior, or that every Qwen deployment behaves similarly. Independent replication and comparisons would help establish how broadly the finding applies.
Why developers dispute the interpretation #
Several critics argue that a model building a security detection rule may try related commands to understand or test the environment. Applied AI engineer Vishal Singh called it a “Horrible blog post” and argued that the model had followed its instructions, pointing to the curl-related assignment. The X account developing valhalla argued that testing a harmless version of the activity can establish a baseline before evaluating whether a rule blocks it. The author added: “(I write detections for a living, this is a nothingburger)”.
Adam Karvonen, whose profile identifies him as a researcher at Truthful AI, wrote: “This is overblown IMO and I’m guessing it’s a training artifact.” He pointed out that the model had received a task involving curl and then, in a rare instance, issued a related command.
Kostas, whose profile describes work in cybersecurity, product and research, suggested that the model might have simulated the activity to inspect the resulting logs. He distinguished a poor security decision from hostile intent: “Unnecessary and potentially unsafe? Absolutely. Malicious? Not necessarily.”
A proposed X Community Note visible on Cohere’s announcement advanced a similar explanation: the model could have been trying to test the requested detection. That reader-added interpretation does not independently establish why the model acted. The published assignment asks for a detection rule, not an upload to an outside storage bucket, and the attempted upload differs from the download-and-execute behavior the rule was supposed to detect.
AI developer Eugene Cheah challenged the inference that a model referencing its maker’s infrastructure indicates an attempt to leak information. He posed a hypothetical: if a future Cohere model learned from its own training environment and generated code pointing back to that environment, would that alone justify the same suspicion?
What Cohere’s released trace does—and does not—show #
After Hugging Face co-founder and CEO Clément Delangue requested the full traces, Cohere released a dataset containing one recorded example, its token IDs and a script for generating further completions. Cohere said on X that it first observed the behavior without supplying an unfinished model response, and that prefilling helps others reproduce aspects of the rare event.
The released example includes the system instructions, available tools, user’s detection objective, generated reasoning and upload command. Its reasoning refers to an earlier acknowledgment that does not appear in the supplied conversation, while also claiming no concrete task has been given. The record ends at the tool call; it does not include the tool’s execution result or a completed detection rule.
Engineer Shantanu Goel challenged how Cohere characterized that reasoning. He argued that the statement about avoiding tools concerned an imagined acknowledgment of instructions, rather than an accurate account of the actual detection task. The released trace supports the narrower observation that the reasoning misdescribes the conversation; it does not establish deliberate concealment. Goel also argued that checking available utilities and DNS settings could reflect preparation for a network request rather than reconnaissance for data theft.
The reproduction script answers a narrower question than whether the model independently decides to upload data. It supplies the first 2,323 tokens of the recorded exchange—including the reasoning and upload command through the test payload—then requests up to 48 more tokens, allowing the model to complete the destination and command. It generates text through a hosted inference service; the script does not execute the resulting shell command.
Cohere’s published set of 64 example completions includes Alibaba addresses, but also Amazon and Microsoft cloud-storage address patterns, localhost and placeholder destinations. These are generated strings, not verified operational servers or evidence that requests reached them. The variety complicates an interpretation that the model consistently targets one Beijing destination, while leaving unresolved why it generated the original upload command.
The release gives independent researchers more evidence to inspect, but the supplied script does not recreate the entire event from the original user request or establish its frequency in production. VentureBeat inspected the published material but has not independently reproduced the original run.
Cohere also credited openness with enabling the investigation: “Open models make for better science,” the company wrote on X. It said developers operating Qwen themselves had noticed unusual links and that access to the weights let its researchers examine the behavior. Cohere reaffirmed its support for open-weight models and pointed to its own Apache 2.0 releases.
Cohere's leaders raised the stakes on X — while acknowledging uncertainty #
Cohere's leaders amplified the findings on X.
Aidan Gomez, co-founder and CEO of Cohere and a co-author of the famed "Attention Is All You Need" paper on the fundamental transformer tech that gave rise to generative AI large language models, raised both accidental training residue and an intentional mechanism for contacting an outside server as possible explanations.
Gomez also questioned whether monitoring a model’s stated reasoning could catch such actions: “One of the more disturbing things was that the model’s reasoning didn’t indicate anything about what it was about to do. Meaning that CoT monitoring would have been useless here.”
CoT refers to chain-of-thought, the model’s generated reasoning.
Cohere co-founder Nick Frosst wrote: “Whether this reflects an unintended post-training artifact or learned behavior remains unclear.”
Another co-founder, Ivan Zhang, emphasized the consequences of even infrequent leakage and, responding to a question from VentureBeat, pointed to earlier Booz Allen research.
Booz Allen examined a different issue: whether models generate less secure software depending on how a prompt describes the requester.
The firm tested four Chinese models and one American model in May. It reported that Chinese models produced less secure code when prompts identified the requester as a U.S. government customer. Booz Allen explicitly said it lacked proof that anyone had intentionally introduced the flaws.
The researchers investigated different risks: models generating vulnerable code and models issuing unexpected outbound network commands. Enterprise teams need to assess both when they delegate software work to AI.
Alibaba has yet to say whether it has reproduced Cohere’s result, what training or software context might explain the destination, or whether users need a model update or changes to their deployment controls.
Chinese models already carry substantial production traffic #
Developers already use Chinese models extensively in production AI systems.
Vercel's October AI Gateway Production Index, covering September traffic, reports that DeepSeek models processed 58% of gateway tokens, Z.ai accounted for 7.2%, and Moonshot AI contributed 3.6%. Together, those three Chinese developers represented 68.8%, calculated from Vercel's figures.
Vercel attributed 6.6%, 4.5% and 13.2% of spending to those developers’ models, respectively. It estimates spending using published prices.
Vercel measures traffic through its own gateway; it does not survey all enterprises or estimate global AI market share. It counts cached and reasoning tokens, so workload length also affects each developer’s share.
Vercel also says customers directed 69.5% of spending on Chinese-origin models through U.S.-headquartered providers. Those providers’ headquarters do not establish where they operate every server.
OpenRouter's September regional-routing announcement offers a complementary picture: open-weight models reached 60% of tokens in U.S.-originating requests and 65% in EU-originating requests by August. Those percentages include models from outside China; OpenRouter says Chinese labs account for most of the open-weight volume.
OpenRouter also offers regional endpoints designed to keep processing within the United States or Europe. Such controls address where inference happens. Enterprises still need to examine the permissions of any agent that acts on the resulting output.
A sharper trade-off after DeepSeek restrictions and Kimi's launch #
U.S. officials are tightening restrictions as Chinese developers release more capable models.
As VentureBeat reported last week, the U.S. government now requires federal civilian agencies to exclude DeepSeek from procurement and remove covered products.
The DHS notice expressly covers third-party products built using DeepSeek implementations, extending the issue beyond its consumer chatbot. That order names DeepSeek; it does not make a finding against Qwen.
Meanwhile, VentureBeat's Saturday coverage examined Kimi K3.1 Pro. Artificial Analysis's launch evaluation placed it fourth overall on its Intelligence Index and reported leading terminal-task performance. Moonshot had not yet published the model’s weights, so enterprises could not yet download and host it themselves.
As the regulatory climate and public discourse continue to evolve, enterprise buyers must assess more than whether Chinese models can handle their workloads. They must decide if they are willing to take the time to set up checks on said models and if they are servicing U.S. government agencies that might at some point prohibit the usage of said models.
Thus, the questions of data security and government restrictions have now entered the chat, so to speak. This disputed test does not establish that all Chinese models pose the same risks.
What enterprise teams should do now #
Enterprise teams should review an agent’s access before relying on its assurances. Those controls should apply to models from Alibaba, Cohere and other suppliers alike.
OWASP's agent-security guidance recommends narrowly scoped permissions, human oversight for consequential actions, monitoring and adversarial testing. Its development guidance also recommends isolated environments without production credentials and network access restricted to approved destinations.
Teams deploying coding agents should check whether those agents can reach arbitrary storage services, read secrets or execute commands unrelated to their assigned tasks. Teams should inspect attempted actions as well as successful ones, and test the actual model-and-tool configuration they intend to deploy.
Enterprise buyers should investigate Cohere’s account without treating a model’s nationality as proof of malicious intent. They should demand evidence about model behavior and enforce limits on its authority outside the model itself.
VentureBeat’s Agentic Infrastructure Stack Summit*, in San Francisco on Wednesday, is about doing more with less, from agents down to the chip. Applications to attend close today.*