An Argument about AI Sentience

I've seen people arguing about large language models and whether they are sentient. I wanted to write down some of my thoughts on this, which is what I'm doing here.

I'm sure they're not very original thoughts, but I'm not an authority on these things.

The Sentience is stored in the Human

Consider a text messaging application. You can send a message, and another person will reply to it. From your point of view, the messaging application appears to be sentient.

But why isn't the messaging application itself sentient? Because the sentient being is the person on the other end of the application of course.

Putting the Human behind a barrier is no different

Imagine then, exchanging many messages in the application, producing a unique conversation each time. If this continued for an infinite period of time, you could engage in every possible conversation with a real person.

If these conversations were recorded in an infinitely large dataset, then we could build a second version of the text messaging application. Rather than a person responding to each message in real time, the application could find, in its dataset, a previous instance of this exact conversation, and it could reply with the next response in that record of conversation.

Imagine using this new application. It would be indistinguishable from the first, and your messages would receive genuine replies from a person who really existed and engaged in that conversation. Despite this, using the application would not involve another person at all. You would still, in a way, be talking to your conversation partner, but the application would have separated you from them by time.

This second application, just like the first, I would not consider to be sentient. If it were placed in a black box, though it would be appear entirely sentient, in a manner indistinguishable from a person.

But in reality, it would just be reflecting the sentience of the people who created the input from which it was constructed. The metaphorical black box could extend out and contain those people as well. In that case, the black box that appears to be sentient would indeed contain sentient beings, but they would just be people, not the application itself.

Thinking about LLMs/AI

Finally, let's consider something more like a conventional LLM. As before, we collect a large dataset of messages and responses, but only a finite quantity this time. As we do not have a record of every possible conversation, we interpolate across the records that we do have, and with some fuzzy matching, we produce responses to even completely novel prompts that broadly match up with the contents of our dataset.

Could this constitute sentience? I don't see why a sufficiently advanced AI could not be sentient just like a person, but I don't think that an LLM crosses that barrier simply because it appears sentient. Even if that appearance was indistinguishable from genuine sentience.

I don't think it is practically possible to determine whether something that has been exposed to a large amount of human input is sentient without understanding how it works internally, and more importantly understanding exactly what sentience is and how it arises. At the moment, I'm not convinced we understand either of those.