xAI files lawsuit against a South Carolina man for allegedly using its chatbot to create illegal AI deepfakes. #
Elon Musk's artificial intelligence company xAI has filed a lawsuit against a South Carolina man accused of using its chatbot, Grok, to create sexualised AI deepfakes involving children.
The lawsuit, filed in a federal court in Texas on Tuesday, alleges that Terry Harwood violated the platform's terms of service by attempting to generate child sexual abuse material and seeks financial damages as well as a permanent ban preventing him from using Grok.
The legal action comes after xAI has faced growing scrutiny over claims that Grok could be exploited to generate non-consensual sexualised deepfakes. The company says the latest lawsuit reflects a tougher stance against users who allegedly misuse its AI tools for criminal purposes.
Why xAI Is Taking Legal Action Against a Grok User #
According to the complaint, Harwood was arrested in February on charges related to the alleged sexual exploitation of minors before xAI filed its civil lawsuit.
The company alleges he uploaded non-sexual photographs of adults and children into Grok and attempted to generate sexually explicit AI-created images from them. It also claims he produced non-consensual sexual imagery involving adults.
Rather than pursuing criminal charges itself, xAI is suing for unspecified monetary damages while asking the court to permanently prohibit Harwood from accessing Grok in the future. The company argues that he breached its user agreement and deliberately used the chatbot for unlawful purposes.
In the filing, xAI stated that Harwood's alleged conduct amounted to 'a calculated scheme to weaponise Plaintiff's tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage.'
The case is believed to be among the first instances of an artificial intelligence company taking direct legal action against one of its own users over the alleged creation of AI-generated child sexual abuse material. While technology firms have previously removed accounts or reported users to law enforcement, civil lawsuits seeking damages against individual users remain uncommon.
xAI's Suspension of More Than 52,000 Grok Accounts #
The lawsuit also sheds light on how xAI says it responds to abuse on its platform.
According to court documents, the company enforces its policies by suspending or terminating accounts and reporting suspected child sexual abuse material to the US National Center for Missing & Exploited Children (NCMEC).
xAI said it had suspended 52,222 accounts during 2026 and submitted 73,604 reports to NCMEC. The company added that those reports had contributed to at least 244 arrests, although the filing does not specify how many resulted in convictions or whether all were directly linked to Grok.
Those figures were included as part of xAI's argument that it actively works to identify and report illegal activity rather than allowing it to continue unchecked.
Voices of Concern Over AI Misuse #
The lawsuit arrives as AI companies face increasing pressure over how generative tools can be misused to create realistic fake images and videos without consent. Lawmakers, child protection organisations and technology experts have raised concerns that increasingly sophisticated AI systems could make the production of abusive content easier if safeguards fail.
In this case, xAI argues that the alleged misuse went beyond violating company rules and created wider legal and reputational risks.
The company maintains that its reporting procedures and account enforcement demonstrate its efforts to prevent its technology from being used to produce child sexual abuse material, while the court will ultimately determine whether Harwood violated the platform's terms of service and whether xAI is entitled to the damages it is seeking.
Harwood has not publicly responded to the lawsuit. Contact information for him was not immediately available, while xAI did not immediately comment beyond its court complaint.
© Copyright IBTimes 2025. All rights reserved.