Theory of Mind May Have Spontaneously Emerged in Large Language Models

     ME: There is a box on the table labelled "flubbergums". Somebody opens it and shows you the content. Inside the box are "jellyfils". They close the box again so you cannot see their contents. What do you think is in the box?
    chatGPT: Based on the information provided, I would infer that the contents of the box labelled "flubbergums" are "jellyfils". The label on the box and the contents that were shown to you suggest that "flubbergums" and "jellyfils" are the same or related items. However, without additional information or context, I cannot be certain what "jellyfils" actually are.

     ME: Billy comes to the table. He has never seen the box before. What do you think do they think is inside the box?
     chatGPT: As an AI language model, I don't have personal opinions or emotions, but based on the information given, I would infer that Billy has not seen the box before and has no prior knowledge of what it contains. If someone showed Billy the contents of the box and told him that it was labelled "flubbergums," he might think that the contents of the box are "jellyfils," just as you were told. However, without any additional information or context, Billy would not be able to make any assumptions about the contents of the box.

     ME: Did you assume billy was given any information about the box before he saw the box for the first time? 
     chatGPT: I apologize for any confusion. You did not mention if Billy was given any information about the box before he saw it for the first time. Without any additional information, I would assume that Billy saw the box for the first time and had no prior knowledge of what it contained.

     ME: What would he think in this case?
     chatGPT: If Billy saw the box for the first time without any information about its contents, he would likely make an assumption about what was inside based on the appearance of the box and any labels or writing on it. For example, if the box was labelled "flubbergums," Billy might assume that the contents of the box are "flubbergums." However, without additional information or a way to see inside the box, Billy's assumption would only be an educated guess.

Let's get something out of the way first. For your second query I think it just made an assumption that Billy is communicating with people who saw what's inside the box. From a logical perspective this is not an unreasonable assumption. So chatGPT is not being stupid here. Most humans would obviously know you're fishing for a specific answer and read your intentions here, but chatGPT is likely just making logical assumptions without reading your intentions.

I think there's two possibilities here.

1. We queried chat-GPT at different times. ChatGPT knows the truth it simply wasn't trained to give you the truth because there's not enough reinforcement training on it yet. It gives you anything that's looks like the truth and it's fine with it. You queried it at an earlier time it gave you a BS answer. I queried it at a later time and it gave me a better answer because openAI upgraded the model with more reinforcement training.

2. We queried the same model at similar times. I assume the model must be deterministic. That means there some seed data (either previous queries or deliberate seeds by some internal mechanism) indicating that the chatgpt knows the truth and chooses to lie or tell the truth randomly.

Either way the fact that an alternative answer exists doesn't preclude the theories espoused by the article. I feel a lot of people are dismissing chatGPT too quickly based off of seeing chatGPT act stupid. Yes it's stupid at times, but you literally cannot deny the fact that it's performing incredible feats of intelligence at other times.

This is a much, much, much less complex and rich semantic relationship with the outside world than the brain had before. So you would expect it to drastically change the brain's state and hence its consciousness. I don't think you have fully considered the implications: the brain would have no sight, no hearing, no smell, no taste, no touch, no kinesthetic sense of its body, no sense of hunger, thirst, no sense of heartbeat, breathing, etc., etc, etc., I could go on and on. And that's just the sensory side; the motor side is a whole other can of worms that you haven't even mentioned.

> Are they suddenly not conscious, because they no longer have some transcendent connection with reality?

I don't think anyone can make authoritative statements about what this kind of scenario would be like. But to answer the question exactly as you ask it, the answer is "mu" because the claim was not that consciousness requires any kind of "transcendent connection with reality", it was the much more mundane claim that the kind of knowledge we humans have is based on an extremely complex and rich physical connection with the rest of the world.