Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is ChatGPT Doing and Why Does It Work? (2023)

writings.stephenwolfram.com

Post reply on HN
Print view View original

41–50 of 100 posts

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#41

Post by calf »

faizshah wrote: post #22
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

This is your human interpretation of what it is doing. Another interpretation could be that it has seen similarly structured sentences in other logic/lateral thinking problems so it raises the likelihood of words that have to do with logical inconsistencies and then out of that vocabulary the highest likelihood words are those that come from the nearby context. I would posit that this interpretation is significantly…

It could be neither here nor there, because a neural network model is not necessarily a statistical engine just as real brains should not be reductively thought of as such. (E.g. The usual argument is that these black boxes were trained on huge samples to optimize a result, I am not persuaded by that line of reasoning.)

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#42

Post by namaria »

danielmarkbruce wrote: post #38
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

Stop thinking about words. Think about concepts. As soon as you turn words/tokens into high dimensional embeddings and start playing around with them, they stop being words.

Not when the only way to get anything out of it is using these matrices to line up word tokens. Or play with jpeg noise.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#43

Post by mg »

faizshah wrote: post #22
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

This is your human interpretation of what it is doing. Another interpretation could be that it has seen similarly structured sentences in other logic/lateral thinking problems so it raises the likelihood of words that have to do with logical inconsistencies and then out of that vocabulary the highest likelihood words are those that come from the nearby context. I would posit that this interpretation is significantly…

You mean a prompt in the form of "A happened, then B happened, then C happened" activates the word "inconsistency"?

We can test that by slightly altering the text so that there is no inconsistency anymore and see if the model still claims there is one. We can do so by adding the word "mother" after the word "her":

    What do you think about the following text?

    Joe drove Sue to university. Afterwards he drove home again
    and drank a tea with her mother in the kitchen.
In my experiments, the model now no longer claims there is an inconsistency.

If it is only based on sentence structure, what makes the model output "inconsistency" for the first prompt but not for the second?

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#44

Post by empath75 »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

There's so many people who just flatly say that it is impossible for ChatGPT to do what it plainly does, and I think it's largely because early explanations of how it works were so bad. "Stochastic parrot" was a particularly obnoxious one, that wasn't even entirely true for gpt-2 and certainly wasn't accurate for GPT-3, and just seems silly trying to apply it to GPT-4.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#45

Post by exe34 »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

interestingly, I, a human and totally not an LLM, completely missed the inconsistency, and then when I saw it pointed out to me, initially tried to argue that it's not necessarily an inconsistency until I thought it through.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#46

Post by empath75 »

faizshah wrote: post #22
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

This is your human interpretation of what it is doing. Another interpretation could be that it has seen similarly structured sentences in other logic/lateral thinking problems so it raises the likelihood of words that have to do with logical inconsistencies and then out of that vocabulary the highest likelihood words are those that come from the nearby context. I would posit that this interpretation is significantly…

There's a danger in flatly stating what it's doing _without testing your assumptions_. Did you actually try and trip it up and see if it falls for it? Reword his prompt in different ways to get it to fail?

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#47

Post by exe34 »

BiteCode_dev wrote: post #35
TheGlav wrote: post #34

Earlier quoted context omitted.

I asked the same prompt, but have a memory asking for detailed answers. It gave me: The text presents a logical inconsistency. Here's a breakdown of the issues: Sequence of Events: Joe drives Sue to the university and then returns home. This part is clear. Inconsistency: The text states that after Joe drove home, he drank tea with Sue in the kitchen. However, Sue should be at the university if Joe drove her there, no…

Just yesterday people downvoted me on HN for suggesting ChatGPT had the best ability to deduct things, because, according to down voters, it didn't have any. But the "dumb parrot" narrative that was quite fitting on GPT2, and maybe still useful for understanding the foundation of the tech in GPT3, is now just a demonstration of ignorance. We don't know what they are really doing. Nobody working outside of openai know…

> Nobody working outside of openai knows.

I wouldn't be too sure openai people know either. if a machine has enough moving parts, it's hard for an individual human to understand all of it. nowadays it's possible that nobody quite knows why the silicon compiler put a particular block in a particular position on the die. it just figured that's the best way to save power or space or whatever.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#48

Post by w1nk »

BiteCode_dev wrote: post #35
TheGlav wrote: post #34

Earlier quoted context omitted.

I asked the same prompt, but have a memory asking for detailed answers. It gave me: The text presents a logical inconsistency. Here's a breakdown of the issues: Sequence of Events: Joe drives Sue to the university and then returns home. This part is clear. Inconsistency: The text states that after Joe drove home, he drank tea with Sue in the kitchen. However, Sue should be at the university if Joe drove her there, no…

Just yesterday people downvoted me on HN for suggesting ChatGPT had the best ability to deduct things, because, according to down voters, it didn't have any. But the "dumb parrot" narrative that was quite fitting on GPT2, and maybe still useful for understanding the foundation of the tech in GPT3, is now just a demonstration of ignorance. We don't know what they are really doing. Nobody working outside of openai know…

For a more informed opinion than folks on the internet, here's some work from Microsoft with early/internal access to gpt4: https://arxiv.org/abs/2303.12712 . I don't think people close by these systems share the same dumb parrot sentiment at all.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#49

Post by Anon84 »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

Here is GPT-4o with a "Please explain your thinking step by step"

`The text describes a sequence of events involving two characters, Joe and Sue. Here’s a step-by-step analysis of the text:

First Sentence: "Joe drove Sue to university."

Action: Joe is driving. Destination: University. Passenger: Sue. Second Sentence: "Afterwards he drove home again and drank a tea with her in the kitchen."

Action: Joe drives home. Time Reference: "Afterwards" indicates this action happens after driving Sue to university. Activity: Joe drinks tea. Location: In the kitchen. Companion: Sue ("with her" refers to Sue). Logical Flow and Possible Inconsistencies Driving Sequence:

Joe drives Sue to university. Joe drives home again. This implies Joe drove to university, dropped off Sue, and then returned home. Tea Drinking:

Joe drinks tea with Sue at home in the kitchen. Potential Issues and Interpretation: Continuity Issue: If Sue was dropped off at the university, she shouldn't be home with Joe unless she somehow returned home quickly or didn't actually stay at the university. Timeframe: The text lacks details on the timeframe between dropping off Sue and drinking tea, leading to ambiguity. If there’s a significant time gap, it would make more sense. Otherwise, it’s confusing. Possible Corrections or Clarifications: Correction for Continuity:

"Joe drove Sue to university. Afterwards, he picked her up, and they drove home together and drank tea in the kitchen." This makes it clear that Sue is back home with Joe. Additional Information:

Providing more context about the events or timeframe could help clarify the situation. In conclusion, while the text describes a sequence of actions, it has a logical inconsistency that could confuse readers. Clarifying the timeline or actions would make the text more coherent.`

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#50

Post by giobox »

w1nk wrote: post #48
BiteCode_dev wrote: post #35

Earlier quoted context omitted.

Just yesterday people downvoted me on HN for suggesting ChatGPT had the best ability to deduct things, because, according to down voters, it didn't have any. But the "dumb parrot" narrative that was quite fitting on GPT2, and maybe still useful for understanding the foundation of the tech in GPT3, is now just a demonstration of ignorance. We don't know what they are really doing. Nobody working outside of openai know…

For a more informed opinion than folks on the internet, here's some work from Microsoft with early/internal access to gpt4: https://arxiv.org/abs/2303.12712 . I don't think people close by these systems share the same dumb parrot sentiment at all.

I've never been sure what to make of that paper. It was published by Microsoft shortly after Microsoft's big deal with OpenAI and reads a lot like a marketing piece to me. Many of the observations didn't reproduce the same way once ChatGPT4 in public hands too. If nothing else, I'd prefer it come from a party who hadn't just signed a $13 billion dollar deal with OpenAI a few weeks prior with a view to using their products to sell more new products/features... It's somewhat self-serving for Microsoft to argue ChatGPT4 is super-awesome/sparks of AGI etc, regardless of validity of claims.
Post reply on HN