If you’ve spent any amount of time using ChatGPT, Claude, or other AI assistants, you’ve witnessed some of the ways it can go wrong. LLM accuracy remains an issue, and there are plenty of social situations in which your outputs can be unwelcome. AI also presents privacy pitfalls, given how personal some conversations can get.
The AI assistants themselves won’t give you much guidance on what to avoid, which means you might be on your own to figure this stuff out. Here are some pointers I’ve gathered from using AI over the years.
This one’s quickly approaching common knowledge. By default, most AI assistants can use anything you submit to train future models, and they also keep a record of all your previous conversations in their sidebars. While they may take precautions to remove personal information from their training data, you should still err on the safe side and leave sensitive details out.
If you’re intent on putting personal or sensitive data in your queries, at least dig through the AI app’s settings to opt out of training first. (In ChatGPT, that’s under Settings > Data Controls > Improve the model for everyone.) You can also use temporary chats to prevent them from showing up in your history. Corollary to the above, AI companies are constantly training new versions of their large language models based on previous conversations, and that occasionally means subjecting those chats to human review.
And while you can opt out of training in many cases, AI companies may get a human involved if they think the conversation veers into potentially dangerous territory, and they keep temporary records of all conversations to be on the safe side. Some AI tools do promise not to store your chat data (most notably Proton Lumo). But if you don’t like the idea of a stranger seeing what you wrote to ChatGPT, don’t put it in a prompt at all.
Even if you realize that AI can get things wrong, it’s easy to be lulled into thinking otherwise when an answer includes links to external sources. Seeing a few links to external sources can make you believe that AI has done its research so you don’t have to.
Don’t be so easily fooled. Too often those source links can lead to random Reddit threads, forum posts, or other unreliable sources. And even if the links look trustworthy, AI can conflate details and wind up getting the facts wrong.
If you care about getting the right answer, check the sources yourself. You can even ask for direct quotes from the source material and then make sure those quotes actually show up. Treating AI like an interrogation suspect instead of a friend will result in better information. AI tools are increasingly pushing users to connect external data sources such as email, calendar tools, and productivity apps. This can allow AI to reference personal information and also act on it, for instance by creating calendar events or drafting emails.
But connecting these data sources comes with risk. AI can still misrepresent your personal data and take actions you might not have intended. There’s also a security danger from “prompt injection,” in which attackers hide malicious instructions inside emails, calendar events, and web pages. If you’re going to connect AI to personal data sources, consider doing so on a temporary basis and always keeping a close eye on its actions.
While you might think that AI will remember every detail of your conversation, in reality it’s likely to let important details slip away as you pack more instructions, files, and data sources into the chat. So as your conversation gets longer, don’t be surprised if the AI’s memory starts to degrade. You may need to restate details from earlier on, or remind AI to reference a data source you’re working with. If you’re working on something complex, like a research report or an app, consider breaking it down into smaller tasks instead of working on everything all at once.
Depending on where you work, there may be policies against using AI-generated content. (Fast Company, for instance, forbids writers from submitting AI-generated content without substantial editing and fact-checking.) Even in emails with coworkers, using AI to communicate may be frowned upon.
So here’s a good rule of thumb to follow: If you’re planning to submit something that was largely AI-generated, and you think that disclosing as much would get you in trouble, just don’t bother in the first place.
Have you noticed an uptick in people cutting and pasting AI answers (or screenshots of AI answers) in response to a question? Don’t be one of those people. It’s an obnoxious practice partly because AI can get things wrong, partly because it gives off a passive-aggressive “Let Me Google That For You” vibe, and partly because injecting AI into human conversation just feels rude.
Besides, AI outputs are like dreams: They’re usually not interesting to anyone but you.
While ChatGPT is less sycophantic than it used to be, AI tools still hew toward being agreeable. That means you can’t expect AI to be a neutral party to an argument or be an especially tough critic of your work. Most AI tools allow you to customize their responses and urge them to tamp down the flattery, but it can still slip through on occasion. Avoid being smitten when AI compliments your instincts or praises the quality of your questions; that’s just what it was trained to do.