Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is ChatGPT Doing and Why Does It Work? (2023)

writings.stephenwolfram.com

Post reply on HN
Print view View original

71–80 of 100 posts

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#71

Post by mriziq »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

There is a world model. Please see Wattenberg Viegas 2023 https://arxiv.org/abs/2305.02469

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#73

Post by hoosieree »

misnome wrote: post #52
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

Does this work without “Clever Hans” prompting it with the implication that there is something to notice?

No. Remove the "explain" or "what do you think" or "think critically about" types of phrases from the prompt and you get this kind of thing:

My prompt:

    Joe drove Sue to the university. Afterwards, he drove home again and drank tea with her in the kitchen.
GPT-4o Response:

    Your sentence is correct, but for a slightly smoother flow, you could rephrase it as: Joe drove Sue to the university. Afterwards, he returned home and drank tea with her in the kitchen.
It will never "notice" anything off about the input unless you ask it to. I'm consistently impressed how many supposedly tech-savvy people believe GPTs turn into magic orbs if you make them big enough.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#74

Post by hoosieree »

williamcotton wrote: post #66
hoosieree wrote: post #30

Earlier quoted context omitted.

> Surely nothing in the prompt directly triggered the word "inconsistency". The prompt is (implicitly) asking to find inconsistencies ("what do you think about") within some statements of fact. Many variations of "find problems in the text" are part of its training set. Remove the "what do you think about" and the model doesn't find inconsistencies. Or keep it, but make the following text more consistent, and watch i…

https://news.ycombinator.com/item?id=40721115

"Explain" is a variation of "what do you think about", so you get similar results.

https://news.ycombinator.com/item?id=40723601

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#75

Post by Jensson »

mg wrote: post #43
faizshah wrote: post #22

Earlier quoted context omitted.

This is your human interpretation of what it is doing. Another interpretation could be that it has seen similarly structured sentences in other logic/lateral thinking problems so it raises the likelihood of words that have to do with logical inconsistencies and then out of that vocabulary the highest likelihood words are those that come from the nearby context. I would posit that this interpretation is significantly…

You mean a prompt in the form of "A happened, then B happened, then C happened" activates the word "inconsistency"? We can test that by slightly altering the text so that there is no inconsistency anymore and see if the model still claims there is one. We can do so by adding the word "mother" after the word "her": What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again…

Remember that OpenAI has given it a massive amount of extra tasks to read that is more like such logical puzzles etc. Assume it has read billions of such lines, easily generated by a simple python script and many such scenarios, then it is much less hard to understand how these models thinks, and why some models thinks betters than others, they just have better such case generators.

An LLM can do some very basic fitting into such cases, so it doesn't need to have seen the exact scenario, but it has seem many similar scenarios.

Edit: So the hard part of making a smart LLM is to figure out a good set of such scenarios and problems to feed the LLM to cover most of the things people will ask the LLM.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#76

Post by Jensson »

Filligree wrote: post #27
QuadmasterXLII wrote: post #21

Earlier quoted context omitted.

A mental model of GPTs that's pretty successful at predicting their behaviour at the limit of large scale is an N-Gram model trained on a much larger internet than the real one. If you search the real world's internet for the text ``` You are a helpful and harmless AI assistant. What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitc…

An interesting way to think about it, but if it did that perfectly... wouldn't it effectively be human-level intelligent? Sure, it isn't any _particular_ human, but I don't see a big difference otherwise.

No, since that larger internet is still not infinitely large, it still fails in dumb ways once your question is a tiny bit stranger.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#77

Post by Jensson »

williamcotton wrote: post #67
misnome wrote: post #52

Earlier quoted context omitted.

Does this work without “Clever Hans” prompting it with the implication that there is something to notice?

Yes.

I just tried with 4o and it didn't notice anything, it just rewrote the statements with different words.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#79

Post by prasoonds »

zarzavat wrote: post #28
prasoonds wrote: post #24

Earlier quoted context omitted.

You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world model which is fundamental to reasoning. That in-turn might mean that the model is reasoning. Others have argued against the "inner world model" theory and suggested that solving reasoning tasks is merely an extension of the "stochastic parrot" scenario - i.e.,…

Why not just give it a series of directions e.g. go one step forwards, turn right, etc like a LOGO program, and then ask it if it is back to the start or not. If the series of instructions is randomly generated then this is impossible to solve without a world model.

People have tried this! And for not-complex problems, the newer models do give the right answer and yet usually fail on the complex tasks.

Prompting does help though - if you ask the model to check it's own answer, it can often catch errors it made and improve. But the interesting part is precisely how does the model succeed at all at any spatial reasoning task? Surely it hasn't seen all spatial reasoning tasks in its training set. So, what's the other hypothesis? As you say, people suggest that there could be a world model that is constructed internally by the LLM.

However, as you can see right here in this thread that people disagree that you need a world model for solving such tasks.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#80

Post by prasoonds »

nalrcnG wrote: post #33
prasoonds wrote: post #15

Earlier quoted context omitted.

This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. Another interesting bit of experiment people did when the GPT-4 class models launched were to test out spatial awareness. For eg, you could describe with words a construction made of blocks, spheres and so on and then ask questions about th…

But where do yo get access to these premium models? I only have free access to CoPilot (which claimed at one point to be GPT4) and it cannot solve any of these issues reliably. It spectacularly fails at slight variations of the goat/cabbage/lion/river problem, it cannot solve spatial or mathematical questions reliably. Do some of the researchers that hype up ChatGPT get access to a special version? I'm not inclined t…

Like the sibling comment says, you typically have to pay for the premium model (and believe me, they are miles ahead of the free stuff - free copilot uses gpt3.5 which is a full 2 generations behind).

The other way is to get an API key from OpenAI or Anthropic and then, you can pay per token which is incredibly cheap - load it up with $5 and it can easily last you months if you're not using it much.

As a dev, I cannot recommend these better models enough. The ROI on my $10-$20 that I spend every month is easily in hundreds to thousands of dollars.

Post reply on HN