The distinction may not be obvious, but it is there.
The reason it is not obvious is that nearly everything you have heard about ChatGPT itself is wrong. The first thing people do to explain what ChatGPT is and does is to personify it. From then on, they are talking about ChatGPT personified, and not ChatGPT as it literally exists. The second thing people do is draw conclusions about the nature and behavior of ChatGPT itself from the narrative they are telling about ChatGPT personified. It's a case of mistaken identity.
ChatGPT has a "brain", but the context that "brain" interacts with is semantics not symbolics.
A human, when answering a question, interprets the symbols present in the language, then considers them logically. Finally, they formulate an answer, and express that answer with more symbols.
ChatGPT does none of that. ChatGPT doesn't even know what sentences, punctuation, or even words are. The only subjects ChatGPT has in mind are short groups of characters: the tokens from the lexical analysis step.
ChatGPT reads those tokens (groups of characters) in order, and generates an implicit model from them. That model is like a map: each token is a feature in the landscape.
When ChatGPT gets a prompt, it tokenizes it, then checks the map for the closest match. Then it starts at that location, and steps forward, writing out what it sees along the way.
That's everything that "ChatGPT as it literally exists" can do. So where does all the behavior come from?
It's the content in the map. It's in language itself. ChatGPT's behavior is limited to interacting with that map, but the effect of interacting with that map is where we get all the interesting behavior.
Language does not simply encode data: it also encodes instructions and logical relationships. By simply walking through text and feeling the semantic landscape, ChatGPT exhibits the behavior that was already encoded into the symbolic meaning of that text. It accomplished this implicitly without ever defining the meaning of any symbol. It doesn't even know what a symbol is in the first place!
So when ChatGPT exhibits the behavior of a person writing correct answers to an exam, it is not behaving like a person at all. It's not interpreting the questions or finding the answers. Instead, it is simply filling the hole in the story with the semantic landscape it sees nearby. If the result is to place answer after question, that is because that data is already present in the training text that ChatGPT was modeled around.
Because of this distinction, we can have a much better understanding of what ChatGPT is and isn't capable of. Because language itself holds the features of truth and lie, mistake and success, elegance and verbosity, love and hate, logic and fallacy, defined and abstract, ambiguous and unambiguous, etc. all equal, ChatGPT must rely on the implementation of language - what was written in the first place - to exhibit behaviors we want it to exhibit.
But there is a critical flaw in that. Language allows, and even depends on, ambiguity. The context that resolves ambiguity can exist in many semantic shapes, so a model cannot be guaranteed to choose the semantic content that contains the disambiguation.
We haven't solved the context dependence problem of natural language. We have only moved it. ChatGPT's success is dependent entirely on the content it is given. It cannot change its behavior to improve that system.