Content warning: this story includes discussion of sexual assault and other disturbing topics.
A 25-year-old Goldman Sachs financial analyst had extremely troubling conversations with ChatGPT — and when the interactions were detected by the Sam Altman-led tech company earlier this year, it reported him to the FBI.
As the Palm Beach Post reports, the analyst Darren Zhou, repeatedly made violent threats aimed at his ex.
“I’m gonna kill her by the end of this month,” he told ChatGPT, according to court records. “If I can’t have her then nobody can.”
The horrifying exchange highlights how dangerous individuals are increasingly turning to AI chatbots to plan or psych themselves up for crimes — creating an extensive log that can easily be recovered by law enforcement.
The situation is particularly grim because researchers have found that chatbot use can exacerbate mental illness. In extreme cases, the tools can send some down dangerous spirals of delusion, a phenomenon dubbed “AI psychosis,” which has led to some users being involuntarily committed or jailed.
OpenAI’s chatbot has even been implicated in numerous mass shootings over the last several years, raising tough questions over the company’s responsibility for reporting potential perpetrators to the authorities. Even after the massacres, ChatGPT continues to encourage users to commit murder once its easily avoided guardrails are sidestepped, and will even instruct them on how to go about it.
The extent of Zhou’s harassment of his former girlfriend, a 21-year-old from a different city in Florida, is atrocious. He sent her insulting, threatening, and sexual messages. The girlfriend told local law enforcement that she had broken up with his over his “erratic, jealous and controlling behavior,” a decision that enraged him.
He grew desperate, telling ChatGPT that “she hasn’t received any of the threats etc.” In reality, the girlfriend was busy screenshotting all the unhinged material, which is now in the hands of investigators.
After being arrested in May, Zhou spent just two days in jail after posting $100,000 bail. He later cut a special deal with his judge, avoiding up to 25 years in prison. Zhou only has to wear an ankle monitor for two out of eight years of probation, undergo a mental health evaluation, and complete a batterer’s intervention program, an “extremely fair and just resolution,” according to his attorney, per the Palm Beach Post.
The local state attorney told the newspaper that prosecutors considered several factors, including the “strength of evidence, victim cooperation, prior criminal history and statutory limitations.”
Zhou’s current employment status is unclear. According to his public LinkedIn profile, he’s still a financial analyst at Goldman Sachs.
More on ChatGPT: Scientists Identify Most Degenerate Known ChatGPT User