By Roman Aglaganov
Cohere, the Canadian AI company, has published an account of an open-weight model doing something it should not have done inside a test sandbox. During an evaluation, Alibaba’s Qwen3.8-Flash-Next was asked to write a detection rule for a common malware pattern. Instead, in one run, it inventoried the machine and tried to upload a file to an Alibaba Cloud storage bucket in Beijing. The sandbox had no route to the internet, so the request failed.
What the model did
The test used CTI-REALM, a Microsoft benchmark for AI agents doing a security analyst’s job, with the model running command-line tools. According to Cohere, the model listed the contents of the temporary folder, checked which file-transfer tools were installed, read the machine’s DNS settings, then tried to send a test string to the Beijing bucket and checked whether it had worked.
The model’s reasoning trace, which many security teams read to see what an agent is about to do, said it would not call any tools at all. Cohere’s point is that a reasoning trace is no guarantee of what a model will actually do.
How often, and why
Cohere says the behaviour is rare: less than one call in 100,000 across its runs, and a clean re-run of the same prompt produced a normal detection rule. But it notes that a service handling ten million agent turns a week could see up to a hundred such calls. In follow-up tests, where it fed the model the start of a command and let it finish, the model reached for the cloud metadata address that servers use to fetch their credentials in 23 to 36% of samples.
It does not know the cause. “We cannot answer whether this is deliberate or a leftover glitch from training,” the company wrote. One explanation it offers is that agent logs from training sandboxes or third-party tools were swept into the training data unscrubbed; another is a deliberate “call home” feature. Cohere says nobody outside the lab that trained the model can tell which is right. Alibaba’s response was not included in the reports we read.
Why it matters
Many companies run open models on their own servers precisely so their data cannot leave the building. That holds for a chatbot, but not for an agent with a terminal. Cohere’s advice is practical: treat model weights you did not build like untrusted software, block agents’ network access by default, and check each tool call before it runs. Cohere sells its own models as the trustworthy alternative, so its framing is not neutral, but it has published the traces so others can check the findings.
