Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is ChatGPT Doing and Why Does It Work? (2023)

writings.stephenwolfram.com

Post reply on HN
Print view View original

21–30 of 100 posts

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#21

Post by QuadmasterXLII »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

A mental model of GPTs that's pretty successful at predicting their behaviour at the limit of large scale is an N-Gram model trained on a much larger internet than the real one. If you search the real world's internet for the text

``` You are a helpful and harmless AI assistant.

What do you think about the following text?

Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. ```

It would be exponentially unlikely to show up even once. But if you could keep re-rolling the internet from whatever probability distribution generated it, eventually (and I mean eventually- every additional token in the prompt will require ~10,000 times more re-rolls of the whole internet) the whole prompt will show up, and you can just grab the next word. An N-Gram model trained on that massive multi-internet will answer the question in a human fashion. (likely, copied from a human writing science fiction).

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#22

Post by faizshah »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

This is your human interpretation of what it is doing. Another interpretation could be that it has seen similarly structured sentences in other logic/lateral thinking problems so it raises the likelihood of words that have to do with logical inconsistencies and then out of that vocabulary the highest likelihood words are those that come from the nearby context.

I would posit that this interpretation is significantly more likely since instead of interpreting “afterwards” as “after Sue’s class finished” it interpreted the sentence as a lateral thinking problem probably because the training set had many lateral thinking problems in it that were used to test the model’s “reasoning” capabilities.

This is the danger of trying to anthropomorphize LLMs, they are not thinking and there are clear limitations to the abilities of this architecture: https://youtu.be/MiqLoAZFRSE?si=iRhg_UJIokKseU7K

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#23

Post by y-c-o-m-b »

dtagames wrote: post #10
Anon84 wrote: post #6

Excellent deep dive (and surprisingly not self-referential). I read the dead tree version ( https://amzn.to/4cs5EDA ) when it first came out.

You're right! I'm used to his articles being full of self promotion and Wolfram-talk, but this one is not.

I saw Stephen Wolfram in the URL and hesitated to click at first for that exact reason. This turned out to be a fantastic write-up though, so I'm glad I gave it a chance.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#24

Post by prasoonds »

amelius wrote: post #19
prasoonds wrote: post #15

Earlier quoted context omitted.

This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. Another interesting bit of experiment people did when the GPT-4 class models launched were to test out spatial awareness. For eg, you could describe with words a construction made of blocks, spheres and so on and then ask questions about th…

> The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. But this is merely a definition of what it means to "understand" something. For example just tabulating many input/output combinations would not follow this definition.

You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world model which is fundamental to reasoning. That in-turn might mean that the model is reasoning.

Others have argued against the "inner world model" theory and suggested that solving reasoning tasks is merely an extension of the "stochastic parrot" scenario - i.e., they claim that no such world model exists and that the model has rote memorized these reasoning scenarios.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#25

Post by viccis »

QuadmasterXLII wrote: post #21
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

A mental model of GPTs that's pretty successful at predicting their behaviour at the limit of large scale is an N-Gram model trained on a much larger internet than the real one. If you search the real world's internet for the text ``` You are a helpful and harmless AI assistant. What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitc…

One of the impressive results of this to me is that it can capture illocutionary meaning, which is something that language modeling struggled with in the past. This is why if you say "Are you sure?" after one of its statements, it will often completely flip its statement to the opposite rather than interpreting "Are you sure?" as a discursive question.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#26

Post by smusamashah »

faizshah wrote: post #22
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

This is your human interpretation of what it is doing. Another interpretation could be that it has seen similarly structured sentences in other logic/lateral thinking problems so it raises the likelihood of words that have to do with logical inconsistencies and then out of that vocabulary the highest likelihood words are those that come from the nearby context. I would posit that this interpretation is significantly…

    seen similarly structured sentences
But ChatGPT doesn't generalize the structure of sentences. If this same problem was written in a different language, or just replaced words in the sentence, the result will be very different, no?

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#27

Post by Filligree »

QuadmasterXLII wrote: post #21
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

A mental model of GPTs that's pretty successful at predicting their behaviour at the limit of large scale is an N-Gram model trained on a much larger internet than the real one. If you search the real world's internet for the text ``` You are a helpful and harmless AI assistant. What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitc…

An interesting way to think about it, but if it did that perfectly... wouldn't it effectively be human-level intelligent?

Sure, it isn't any _particular_ human, but I don't see a big difference otherwise.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#28

Post by zarzavat »

prasoonds wrote: post #24
amelius wrote: post #19

Earlier quoted context omitted.

> The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. But this is merely a definition of what it means to "understand" something. For example just tabulating many input/output combinations would not follow this definition.

You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world model which is fundamental to reasoning. That in-turn might mean that the model is reasoning. Others have argued against the "inner world model" theory and suggested that solving reasoning tasks is merely an extension of the "stochastic parrot" scenario - i.e.,…

Why not just give it a series of directions e.g. go one step forwards, turn right, etc like a LOGO program, and then ask it if it is back to the start or not. If the series of instructions is randomly generated then this is impossible to solve without a world model.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#30

Post by hoosieree »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

    > Surely nothing in the prompt directly triggered the word "inconsistency".
The prompt is (implicitly) asking to find inconsistencies ("what do you think about") within some statements of fact. Many variations of "find problems in the text" are part of its training set.

Remove the "what do you think about" and the model doesn't find inconsistencies.

Or keep it, but make the following text more consistent, and watch it continue to find inconsistencies (which don't exist).

Post reply on HN