When FBI agents searched Bradley Heppner’s home, they walked out with about 31 documents that recorded his conversations with an AI chatbot. Not contracts. Not bank statements. Chat transcripts.
Heppner was a defendant in a criminal fraud case, and he had been using the public version of Claude to think through his legal exposure. Some of what he typed came from his own lawyers. His defense team said the pages were privileged. In February 2026, a federal judge in Manhattan disagreed. Judge Jed Rakoff’s reasoning began with a point so basic it sounds like a punchline: the chatbot is not a lawyer.
One case is a story. A few of them together are a pattern. OpenAI’s CEO has said on the record that ChatGPT conversations carry no legal confidentiality the way a therapist’s or lawyer’s office does. A federal judge in New York has ordered OpenAI to produce 20 million de-identified chat logs in the New York Times copyright fight. None of this is rumor. All of it is documented.
Nobody needs to quit chatbots over this. They are useful, and they aren’t going anywhere. The point is narrower. What you type into ChatGPT, Claude or Gemini is closer to a message passed through a company than a thought kept in a locked notebook. The law hasn’t caught up with how personal we’ve all become with these tools.
Below are 13 things I’d keep out of the chat box, ordered roughly from “could hurt you in court” to “could hurt you in your inbox.” The heavy ones come first.
General information, not legal advice. The rules here are changing fast.
Heppner’s real mistake wasn’t asking a chatbot questions. It was feeding it what his lawyers had told him. The government argued, and the judge agreed, that sharing privileged material with a third-party AI platform may waive protection over the original attorney-client conversations too. The court also saw no reasonable expectation of confidentiality on a consumer platform.
Read that twice. The damage didn’t stop at his chatbot chats. It reached back to the real ones.
The ruling did leave a narrow door open. The judge noted that if counsel had directed Heppner to use the tool, the picture might have looked different. White & Case’s reading of the decision draws the same line between lawyer-directed use and a client freelancing at midnight.
If you’re in a dispute, a divorce, an investigation or a lawsuit, treat a chatbot like a stranger sitting at the next table. Ask your lawyer whether you should use AI at all, and never paste in their emails or their advice. Chat logs get read the way search histories get read, as a window into your state of mind. A CNN wire story carried by CTV News describes investigators increasingly using AI chat histories as evidence. Legal experts in it compare them to Google searches, which can reveal motive and intent. One cybersecurity lawyer called them a “treasure trove” for law enforcement.
In the Palisades fire case, prosecutors say Jonathan Rinderknecht used ChatGPT to generate fire imagery months earlier. They also say that after calling 911 to report the blaze, he asked whether a cigarette makes you at fault for a fire. CNN’s story adds that he has pleaded not guilty and that his lawyer is seeking to exclude some of the ChatGPT evidence.
You don’t have to be guilty of anything for this to matter. A dark joke, a hypothetical about a coworker, a “how would someone even do that” typed out of pure curiosity: it all sits in a log with a timestamp and no context. Someone else gets to supply the context.
Plenty of people use these tools as a therapist now. Altman said so himself on a podcast. Young people especially treat ChatGPT like a life coach. He also said that if you bring it your most sensitive stuff and a lawsuit follows, OpenAI “could be required to produce that.” He called the situation a problem, and he’s right.
A therapist works under confidentiality rules. A chatbot works under a privacy policy. A privacy policy can be rewritten, and a judge’s order can override it. The warm tone of a reply doesn’t change which of those you’re dealing with.
If you need to vent, vent into a notes app. If you want a chatbot’s help thinking something through, strip out the names, places and anything else that points to a real person, including you. And if things feel heavy, a friend, a counselor or a helpline is a better first call than a chat window. Up a blood test to a chatbot feels like a clever second opinion. Legally, it’s a different animal from handing the same sheet to your doctor. HIPAA, the US health privacy law, binds hospitals, insurers and their partners. Becker’s Hospital Review summarized a New York Times report in which privacy experts said it doesn’t cover popular chatbots, and that uploaded records could be used for training.
OpenAI has since launched ChatGPT Health, which lets people connect medical records and apps like Apple Health. The company says those conversations are encrypted, stored separately and not used to train its models. Privacy advocates still point out that the data isn’t protected by HIPAA, and one said OpenAI hasn’t explained how it decides when to share health data with law enforcement.
That’s a company promise, not a statute. Ask about symptoms in general terms. Leave out your name, date of birth, insurance ID and the scan itself. If you want an AI to read your records, use a tool your provider has formally approved.
Samsung’s engineers weren’t trying to leak anything. They were trying to debug. After the company let them use ChatGPT, TechRadar reported three separate leaks of sensitive data, source code among them, and Samsung responded with a ban on company devices.
It wasn’t only Samsung. Amazon had reportedly warned its own staff months earlier not to share code or confidential information with ChatGPT, after the company noticed answers that resembled internal Amazon data.
The lesson is simple. If your company hasn’t handed you an approved AI tool and a written rule about it, assume the answer is no. Ask before you paste. “I was only fixing a bug” is a weak thing to say in a disciplinary meeting.
Freelancers and consultants make this mistake more than anyone, because nobody is watching. A client’s contract, a customer spreadsheet, a pitch deck marked confidential: dropping any of it into a personal chatbot account hands it to a third party you promised to keep it from.
The account terms matter here. A Stanford team that compared the privacy policies of Amazon, Anthropic, Google, Meta, Microsoft and OpenAI found that all six appear to use chat data for training by default. Opt-outs exist, but you have to go find them.
Read your contract before you paste anything. Many NDAs treat any outside disclosure as a breach, whether or not a human ever reads the chat. Redact names, figures and anything identifying. Work plans can come with different data terms from personal accounts, so read those before you assume.
This one sounds too obvious to list, and it still happens constantly, usually by accident. You paste a config file to fix an error, or a script with a token sitting on line 12, and the secret rides along without you noticing.
In January 2025, researchers at Wiz found a publicly exposed DeepSeek database holding over a million lines of logs, including chat history and secret keys. DeepSeek secured it quickly after being told. The point isn’t about one company. Chat logs and credentials end up in the same pile, and piles leak.
Swap real tokens for placeholders before you paste. If a secret has already gone into a chat, treat it as burned. Rotate it, because deleting the chat won’t undo anything. The same goes for passwords, two-factor backup codes and wallet recovery phrases. No useful answer ever needs the real one.
Social Security number, Aadhaar, passport number, card number, net banking password. A chatbot doesn’t need any of them to help you with a form or a statement. Black them out first.
The platform’s own plumbing can fail too. In March 2023, a bug in an open-source library meant OpenAI said payment-related information of 1.2% of ChatGPT Plus subscribers may have been visible to another user during a nine-hour window. It listed name, email, payment address, card type, the last four digits and the expiry date. OpenAI also said full card numbers were not exposed and that the number of people who actually saw anything was extremely low.
That leak came from subscription pages, not from anything users typed into chats. That’s exactly why it matters. If a bug can surface billing details, assume a pasted scan of your passport is a bigger prize. Type “XXXX” where the number would go. The answer will be just as good.
Your sister’s diagnosis. A friend’s affair. A coworker’s HR complaint. You’re typing it out to get advice, and your intentions are good, but the person you’re describing never agreed to be part of the conversation.
The policies behind these tools are thin on what happens next. The same Stanford researchers, in their peer-reviewed analysis of those privacy policies, found that some developers retain chat data indefinitely. A secret about someone else can outlive the friendship it came from. And as item 1 showed, a chat about a named person can be read by a lawyer or an investigator as easily as one about you.
Change the names. Make it “a friend” and “a coworker.” Cut anything that would let a stranger work out who is who. If you couldn’t say it to the person’s face, think twice before typing it behind their back.
Anger types fast. A rant about a boss, an ex or a neighbor can feel harmless at 1 a.m., and a chatbot will happily keep talking while you type it.
OpenAI has said its systems flag conversations where users seem to be planning harm to others. Those chats go to human reviewers, and if a reviewer decides there is an imminent threat, the company can report it to law enforcement. To an automated filter, a furious rant and a real plan can look alike. A person then has to decide which one it was.
You don’t need to be dangerous for this to go badly. You only need to be misread. Put the anger into a notes app, a voice memo or a text to a friend who knows you. A chat window is a bad place to say something you’d be nervous to see quoted in a report.
Typing “we’re out of town from the 12th to the 20th” next to a home address is the kind of detail that should never share a sentence, in any app.
Photos can say it for you. In 2025, a viral reverse-location trend showed OpenAI’s newer models working out where a picture was taken from visual clues alone, and it raised doxxing concerns. Your balcony view, a street corner or the front of your building can be enough.
If you want help with a photo, crop out street signs, house numbers and landmarks. Ask about neighborhoods in general terms instead of giving your exact block. Keep dates away from places. Your routine is part of your safety, and it’s very easy to give away one sentence at a time. Parents ask chatbots about homework, tantrums, health worries and birthday parties, and the details creep in. A full name, a school, a photo for “what’s wrong with this rash.” Each one is small. Together they build a profile of a person who never agreed to be profiled.
The American Psychological Association’s June 2026 piece on sharenting notes that children’s faces count as sensitive biometric data. It says they can be scraped from posts and used to create deepfakes. Up a picture to a chatbot isn’t the same as posting it publicly. The principle still holds, though: once a child’s likeness leaves your hands, you can’t call it back.
Use first names or none. Describe the situation instead of up the face. If you’re asking about your child’s health, type the symptom and the age range, not the whole story. They’ll have their own digital footprint soon enough, and it shouldn’t start with yours. The most embarrassing leaks in this story weren’t hacks. They were buttons.
In 2025, Fast Company found more than 4,500 shared ChatGPT links indexed by Google, some with names, resumes and confidential work content. People had made them discoverable with a checkbox. OpenAI pulled the option after the backlash.
Meta’s AI app has a public Discover feed. After users unwittingly posted chats there, Engadget reported that Meta added a warning telling people to avoid sharing personal or sensitive information. Meta says posting takes several deliberate steps. Plenty of people still ended up on the feed.
Before you share a chat, read it as a stranger would, from the top. Look for names, numbers and anything that points to you. If you ever shared one in the past, go back through your shared links and delete the ones you don’t need.
Thirteen items is a lot to remember. Here’s a shorter version.
Before you hit enter, picture the conversation printed out and sitting on a table between you and a stranger. Imagine a lawyer, a coworker, a journalist or a hacker holding the pages. If your stomach drops, rewrite it or don’t send it.
None of this means giving up chatbots. They’re useful, and nobody needs to feel guilty for using them. The law just hasn’t caught up with how we use them. Even OpenAI’s CEO has said he wants privacy protections for AI chats closer to what a doctor or lawyer gets. Until that exists, the only protection you can count on is the one you apply yourself.
Bradley Heppner learned the lesson the hard way, in front of a federal judge. You can learn it for free, by deciding what never goes in the box.
Your Chatbot Is Not a Vault: 13 Things to Keep Out of ChatGPT, Claude and Gemini Before a Judge… was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.