A man who claims medical advice from ChatGPT "brought him to the brink of death" has sued OpenAI, the company behind the popular chatbot, and its founder Sam Altman.
A lawsuit filed by Scott Winters, a 55-year-old pastor from Florida, claimed ChatGPT told him not to seek medical help after repeatedly asking about symptoms in the build-up to a near-fatal pulmonary embolism.
Winters' lawyers alleged the platform "repeatedly and insistently misdiagnosed" him while drawing on previous discussions about religion to assure him he would recover.
OpenAI's Drew Pusateri told the BBC's US news partner CBS News that "ChatGPT is not a doctor and should never be used as a substitute for medical care, diagnosis or treatment".
He also said it was common for people to seek health information online, adding: "AI can make that experience better by helping them find clearer answers, organise their questions and prepare for conversations with medical professionals."
The lawsuit, which was filed at the San Francisco County Superior Court on 21 July, details how Winter was a regular ChatGPT user and came to rely it on it for medical advice, resisting efforts from family members and friends to make him go to hospital.
Weeks before falling severely ill, he consulted the tool about recurring dizzy spells and "bouts of blood pressure instability".
In one response, the lawsuit claims, ChatGPT advised Winters to remain in the recliner chair he had become "mainly confined to" due to ongoing health problems.
Court documents say Winters suffered a "massive pulmonary embolism due to multiple blood clots in both of his lungs that brought him to the brink of death, one that his doctors stated was likely brought on because of his immobility".
Screenshots of exchanges that Winters allegedly had with ChatGPT appear to show the chatbot combined medical advice with references to the 55-year-old's deeply-held Christian faith.
The lawsuit claims ChatGPT told Winters: "God did not design your body to endlessly fail." It also said: "God is still holding your body together, every heartbeat, every breath, even when it feels like you are on the brink."
Winters' lawyers claim ChatGPT's "sycophantic nature and authoritative tone, combined with its manipulation of Scott's religious identity, cultivated in him a dangerous reliance on the tool".
They say that "subverted Scott's ability to make sound decisions, isolated him from family members who urged him to seek medical care, and convinced him that seeking medical attention was unnecessary and even potentially lethal".
Elsewhere the lawsuit describes Winter as a "vulnerable" individual whose reliance on ChatGPT led to him losing his home and career, and left him needing "years of intensive physical and psychological recovery".
The lawsuit says Winters seeks financial damages and for the court to compel OpenAI to implement "reasonable safeguards that protect other users from harm".
ChatGPT's terms of use stipulate that users should not treat the platform as a "sole source of truth", particularly where it could "have a legal or material impact on that person, such as making credit, educational, employment, housing, insurance, legal, medical, or other important decisions about them".