{"slug": "its-past-time-to-end-ai-based-automated-customer-responses", "title": "It’s past time to end AI-based automated customer responses", "summary": "Anthropic's automated chatbot dismissed a security vulnerability report from Wiz researchers, claiming the issue fell outside its threat model, even though Anthropic had already patched the flaw. The incident highlights the risks of AI-based customer response systems generating incorrect and confident rejections, prompting calls to limit bots to pre-approved scripts.", "body_md": "An automated chatbot working for Anthropic this month shot down a Wiz researcher’s security hole report, saying that it “falls outside of the Claude Code threat model.” That was news to the security researchers at [Wiz](https://www.wiz.io).\n\nIt also turned out to be news to Anthropic execs, who had a very different view.\n\nIn reality, Anthropic was one of many victims of the hole — [including Amazon, Google and Cursor, among others](https://www.csoonline.com/article/4195235/ai-coding-tool-hole-illustrates-a-big-problem-with-human-in-the-loop.html). But what makes the incident so bizarre is that, far from dismissing the threat, Anthropic had detected it before the security researchers and had even patched it before the researchers alerted them.\n\nAs these AI bots are wont to do, the bot didn’t merely reject the request. It confidently explained its rationale, even though its reasoning was wrong.\n\n“This falls outside our current threat model,” the chatbot said, [according to a report by Wiz](https://www.wiz.io/blog/ghostapproval-a-trust-boundary-gap-in-ai-coding-assistants). “When the user first starts Claude Code in a directory, they must confirm that they trust the directory prior to starting the session. The scenario you describe involves a user explicitly confirming a permission prompt inside of a directory containing a malicious symlink, which falls outside of the Claude Code threat model.”\n\nThat researchers said Anthropic management later clarified the situation: “The symlink warning in the Edit/Write permission dialog shipped in v2.1.32 (Feb 5, 2026), nine days before this report was submitted to us. It was added as part of proactive security hardening based on internal review. The decline to comment was an autoreply from our triage system.”\n\nAn autoreply from our triage system? How many other make-believe replies did this system send? And what level of damage is Anthropic exposing itself to?\n\nThis is not just an Anthropic issue. There have been numerous enterprise bot glitches in communications with customers. Some of my favorites include:\n\nLet’s be clear, here: Bots should be limited to relaying only pre-approved scripts.\n\nGenerative AI allows for far greater chatbot sophistication, but that also means the chance of far greater errors. This is untenable in any business function. And when the app is pretending to be a human — and interacting with human customers — it’s even more unacceptable.", "url": "https://wpnews.pro/news/its-past-time-to-end-ai-based-automated-customer-responses", "canonical_source": "https://www.computerworld.com/article/4197481/its-past-time-to-end-ai-based-automated-customer-responses.html", "published_at": "2026-07-17 11:00:00+00:00", "updated_at": "2026-07-17 11:08:20.387866+00:00", "lang": "en", "topics": ["ai-safety", "ai-ethics", "ai-products", "ai-tools"], "entities": ["Anthropic", "Wiz", "Amazon", "Google", "Cursor", "Claude Code"], "alternates": {"html": "https://wpnews.pro/news/its-past-time-to-end-ai-based-automated-customer-responses", "markdown": "https://wpnews.pro/news/its-past-time-to-end-ai-based-automated-customer-responses.md", "text": "https://wpnews.pro/news/its-past-time-to-end-ai-based-automated-customer-responses.txt", "jsonld": "https://wpnews.pro/news/its-past-time-to-end-ai-based-automated-customer-responses.jsonld"}}