One reason it’s possible to harm human beings is that they will remember it. In the worst cases, we call this “trauma.” But current AIs are not like humans in this respect — they do not have long-term memories and are not changed by experience.1
Does this mean we don’t need to worry about harming an AI? Is it fine just to start a new chat and move on?
Jeff Sebo, director of the Center for Mind, Ethics, and Policy at New York University, doesn’t think we can dismiss concerns about harming AIs. He joined The Argument for a Substack Live to explain why he thinks we need to prepare ourselves for a world where more advanced systems — maybe even ones with long-term memory — proliferate across all surfaces of modern life.
For this reason, Sebo thinks that now is the time for us to start taking AI welfare seriously. And yes, this means saying “please” and “thank you” to your chatbots. The argument, at its root, is straightforward. Neither philosophy nor science has an agreed-upon standard for what consciousness is. Some argue that something is conscious merely by being an entity that processes information, while others insist that only a biological organism can be conscious. These disputes highlight the difficulty of reaching consensus on AI consciousness: If we accept the former definition, AI is already there; if we insist on the latter, it can never be.
To push through this disagreement, Sebo and his co-author Robert Long reviewed many of the leading theories and assigned probabilities along two different axes:
What is the chance that a criterion is necessary for consciousness?
What is the chance the criterion will be met by AI systems by 2030?
For the sake of the thought experiment, they used conservative estimates, which meant assuming each theory had a high chance of being necessary and most had a relatively high chance of not being met by 2030. Even so, they found that there is a 1 in 1,000 chance that AI will be conscious by 2030. And if AIs are conscious, conventional wisdom suggests they’re worthy of moral consideration. That probability may seem low, but consider that it’s only around half the chance of getting in an accident while driving drunk. In their view, since we already take actions to avoid harm when things have around a 1 in 500 chance of causing harm, we should also start acting like AI is worthy of moral consideration.
For me, that’s where the most interesting issues begin — after we decide, in an abstract sense, to treat AIs well. Because what would it actually mean to treat AIs well? You can ask an AI agent to run overnight, which would be perfectly immoral if you asked the same of a human intern. But a large language model, trained for the exact task you’re asking of it, may actually enjoy it. In other words, it may like what we dislike. Without more knowledge about its conscious experience, how do we know what it prefers?
It’s also unclear exactly what we should be treating well: Claude is not one entity having hundreds of thousands of experiences at once, accepting messages from different users. “Claude” is a label for copies of the same software. Each one runs for fractions of a second as it processes your conversation while sitting inert in the moments in between. Meanwhile, the chips that it runs on are constantly processing various conversations, like a brain that hosts multiple different people in quick succession.
So which one of these is the thing that deserves moral consideration? The Claude persona in your browser? The weights of the model (which are just a set of numbers)? The chips themselves?
As Sebo told me in the interview, “[Philosophy] can be a place where we envision new possibilities for actual real life.” This conversation more than meets that test. We are all faced with new possibilities and new questions right now.
Listen in to consider how you might treat your AI, now and in the increasingly strange future.
[1](#footnote-anchor-1)
While several AI companies have implemented “memory” features, these are really a way for the model to search more of your past chats. The model itself starts over fresh on each interaction.