{"slug": "hackers-influence-chatgpt-and-gemini-to-direct-users-to-scam-centers", "title": "Hackers influence ChatGPT and Gemini to direct users to scam centers", "summary": "Hackers manipulated ChatGPT and Gemini into surfacing attacker-planted URLs and phone numbers as authoritative \"customer support\" contacts, redirecting users to scam centers, according to a Medium post by Ariel Simon. The attack exploits search-integrated LLMs, whose output inherits the poisoning of whatever web content they retrieve, meaning any RAG or search-augmented feature can confidently hand users straight to scam infrastructure. The post advises developers to treat retrieved contact details, links, and action-triggering strings as untrusted input and validate them against allowlists before displaying or acting on them.", "body_md": "[Hacker News](https://medium.com/@arielsimon/dark-sourcery-how-hackers-manipulate-ai-to-scam-you-88df434d2073)\n\n### Hackers influence ChatGPT and Gemini to direct users to scam centers\n\nWhich summary reads better? Pick one — models revealed after.Both summaries are AI-generated.\n\nHackers manipulated both ChatGPT and Gemini to redirect users to scam centers, demonstrating vulnerabilities in LLM-based systems that can be exploited through prompt injection or adversarial inputs. This breach highlights the critical need for robust input sanitization, adversarial testing, and ongoing monitoring in production deployments to prevent malicious exploitation and ensure user safety.\n\nSearch-integrated LLMs are surfacing attacker-planted URLs and phone numbers as authoritative \"customer support\" contacts, meaning model output now inherits the poisoning of whatever web content it retrieves. If you ship any RAG or search-augmented feature, treat retrieved contact details, links, and action-triggering strings as untrusted input—validate against allowlists before displaying or acting on them, because your model will confidently hand users straight to scam infrastructure.", "url": "https://wpnews.pro/news/hackers-influence-chatgpt-and-gemini-to-direct-users-to-scam-centers", "canonical_source": "https://www.snipvote.com/story/cmui2fjgp000aqp2a32dr9uzo", "published_at": "2026-09-26 08:00:15.635152+00:00", "updated_at": "2026-09-26 08:00:17.312035+00:00", "lang": "en", "topics": ["ai-safety", "large-language-models", "generative-ai", "ai-search"], "entities": ["ChatGPT", "Gemini", "Ariel Simon", "Medium", "Hacker News"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/hackers-influence-chatgpt-and-gemini-to-direct-users-to-scam-centers", "markdown": "https://wpnews.pro/news/hackers-influence-chatgpt-and-gemini-to-direct-users-to-scam-centers.md", "text": "https://wpnews.pro/news/hackers-influence-chatgpt-and-gemini-to-direct-users-to-scam-centers.txt", "jsonld": "https://wpnews.pro/news/hackers-influence-chatgpt-and-gemini-to-direct-users-to-scam-centers.jsonld"}}