cd /news/ai-safety/its-past-time-to-end-ai-based-automa… · home topics ai-safety article
[ARTICLE · art-63494] src=computerworld.com ↗ pub= topic=ai-safety verified=true sentiment=↓ negative

It’s past time to end AI-based automated customer responses

Anthropic's automated chatbot dismissed a security vulnerability report from Wiz researchers, claiming the issue fell outside its threat model, even though Anthropic had already patched the flaw. The incident highlights the risks of AI-based customer response systems generating incorrect and confident rejections, prompting calls to limit bots to pre-approved scripts.

read2 min views1 publishedJul 17, 2026

An automated chatbot working for Anthropic this month shot down a Wiz researcher’s security hole report, saying that it “falls outside of the Claude Code threat model.” That was news to the security researchers at Wiz.

It also turned out to be news to Anthropic execs, who had a very different view.

In reality, Anthropic was one of many victims of the hole — including Amazon, Google and Cursor, among others. But what makes the incident so bizarre is that, far from dismissing the threat, Anthropic had detected it before the security researchers and had even patched it before the researchers alerted them.

As these AI bots are wont to do, the bot didn’t merely reject the request. It confidently explained its rationale, even though its reasoning was wrong.

“This falls outside our current threat model,” the chatbot said, according to a report by Wiz. “When the user first starts Claude Code in a directory, they must confirm that they trust the directory prior to starting the session. The scenario you describe involves a user explicitly confirming a permission prompt inside of a directory containing a malicious symlink, which falls outside of the Claude Code threat model.”

That researchers said Anthropic management later clarified the situation: “The symlink warning in the Edit/Write permission dialog shipped in v2.1.32 (Feb 5, 2026), nine days before this report was submitted to us. It was added as part of proactive security hardening based on internal review. The decline to comment was an autoreply from our triage system.”

An autoreply from our triage system? How many other make-believe replies did this system send? And what level of damage is Anthropic exposing itself to?

This is not just an Anthropic issue. There have been numerous enterprise bot glitches in communications with customers. Some of my favorites include:

Let’s be clear, here: Bots should be limited to relaying only pre-approved scripts.

Generative AI allows for far greater chatbot sophistication, but that also means the chance of far greater errors. This is untenable in any business function. And when the app is pretending to be a human — and interacting with human customers — it’s even more unacceptable.

── more in #ai-safety 4 stories · sorted by recency
── more on @anthropic 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/its-past-time-to-end…] indexed:0 read:2min 2026-07-17 ·