Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is ChatGPT Doing and Why Does It Work? (2023)

writings.stephenwolfram.com

Post reply on HN
Print view View original

31–40 of 100 posts

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#31

Post by faizshah »

smusamashah wrote: post #26
faizshah wrote: post #22

Earlier quoted context omitted.

This is your human interpretation of what it is doing. Another interpretation could be that it has seen similarly structured sentences in other logic/lateral thinking problems so it raises the likelihood of words that have to do with logical inconsistencies and then out of that vocabulary the highest likelihood words are those that come from the nearby context. I would posit that this interpretation is significantly…

seen similarly structured sentences But ChatGPT doesn't generalize the structure of sentences. If this same problem was written in a different language, or just replaced words in the sentence, the result will be very different, no?

That’s right, if LLMs were really thinking/forming world models etc. we would expect them to be robust against word choice or phrasing. But in practice anyone using RAG can tell you that that is not the case.

I’m just a practitioner so my language might be imprecise but when I say similarly structured sentences what I mean is, and this is my interpretation based on my experience with using Agents and LLMs, that the shape of the context as in the phrasing and the word choice highly bias the outputs of LLMs.

In my own observations at work, those who interpret LLMs to be thinking often produce bad agents. LLM are not good at open ended questions, if you ask an LLM “improve this code” you will often get bad results that just look passable. But if you interpret LLMs as probabilistic models highly biased by their context then you would add a lot more context and specific instructions in the prompt in order to get the Agent to produce the right output.

Side note, this is also why I like the AICI approach: https://github.com/microsoft/aici A lot of us think it is silly how the phrasing and word choice can produce dramatically different results in RAG applications. If you could run a program (like AICI) that post processes the output and picks the next word in a more structured way instead of writing more creative prompts that just makes a lot more sense to me.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#32

Post by jacobr1 »

flebron wrote: post #20
brcmthrowaway wrote: post #12

Earlier quoted context omitted.

Is RLHF same as fine tuning?

No. In the common use of the word fine-tuning, one is in the supervised learning scenario. One has an input prompt, and an output sentence. One teaches the model to say that output in response to that prompt. In the reinforcement learning scenario, one has a prompt, and a way of rewarding the model for different outputs. One can have, for instance, a reward model, that assigns a reward for a given model output. One c…

One wrinkle, is that it is now common to fine-tune on previously derived RL datasets, with the tested inputs and preferred sample outputs as the training data.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#33

Post by nalrcnG »

prasoonds wrote: post #15
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. Another interesting bit of experiment people did when the GPT-4 class models launched were to test out spatial awareness. For eg, you could describe with words a construction made of blocks, spheres and so on and then ask questions about th…

But where do yo get access to these premium models? I only have free access to CoPilot (which claimed at one point to be GPT4) and it cannot solve any of these issues reliably.

It spectacularly fails at slight variations of the goat/cabbage/lion/river problem, it cannot solve spatial or mathematical questions reliably.

Do some of the researchers that hype up ChatGPT get access to a special version? I'm not inclined to buy a subscription to find out and the AI Reddits aren't that positive either.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#34

Post by TheGlav »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

I asked the same prompt, but have a memory asking for detailed answers. It gave me:

    The text presents a logical inconsistency. Here's a breakdown of the issues:

    Sequence of Events: Joe drives Sue to the university and then returns home. This part is clear.
    Inconsistency: The text states that after Joe drove home, he drank tea with Sue in the kitchen. However, Sue should be at the university if Joe drove her there, not at home.
    To correct this, consider one of the following revisions:

    Sue returns home: If Sue returns home with Joe:

    "Joe drove Sue to the university. Afterwards, he drove back home with her, and they drank tea together in the kitchen."
    Joe drinks tea alone: If Sue stays at the university:

    "Joe drove Sue to the university. Afterwards, he drove home again and drank tea alone in the kitchen."
    Either option removes the inconsistency and clarifies the sequence of events.
This is pretty impressive.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#35

Post by BiteCode_dev »

TheGlav wrote: post #34
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

I asked the same prompt, but have a memory asking for detailed answers. It gave me: The text presents a logical inconsistency. Here's a breakdown of the issues: Sequence of Events: Joe drives Sue to the university and then returns home. This part is clear. Inconsistency: The text states that after Joe drove home, he drank tea with Sue in the kitchen. However, Sue should be at the university if Joe drove her there, no…

Just yesterday people downvoted me on HN for suggesting ChatGPT had the best ability to deduct things, because, according to down voters, it didn't have any.

But the "dumb parrot" narrative that was quite fitting on GPT2, and maybe still useful for understanding the foundation of the tech in GPT3, is now just a demonstration of ignorance.

We don't know what they are really doing. Nobody working outside of openai knows.

But if you use it long enough, it will be obvious we have passed the fancy completion phase, there is a little logic in there. Not a lot, but it makes all the results much better already.

And definitely much better than the competition.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#36

Post by seanhunter »

nalrcnG wrote: post #33
prasoonds wrote: post #15

Earlier quoted context omitted.

This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. Another interesting bit of experiment people did when the GPT-4 class models launched were to test out spatial awareness. For eg, you could describe with words a construction made of blocks, spheres and so on and then ask questions about th…

But where do yo get access to these premium models? I only have free access to CoPilot (which claimed at one point to be GPT4) and it cannot solve any of these issues reliably. It spectacularly fails at slight variations of the goat/cabbage/lion/river problem, it cannot solve spatial or mathematical questions reliably. Do some of the researchers that hype up ChatGPT get access to a special version? I'm not inclined t…

How people get access to the premium models is typically to sign up[1] for an account and pay a subscription. GPT-4o is significantly different from original GPT-4 which is again very significantly different from GPT-3.5. This is all moving fast.

So if you don't want to pay for a subscription I think you can get some free use of anthropic's most capable model (Opus) - I don't know the status of what you can get for free from openai.

The opinion of AI reddit is only really going to get you so far because the capabilities of the models are wildly different for different use cases, so you really need to be able to try it out for yourself and see if it can do what you need it to do.

[1] Somewhere like https://openai.com/chatgpt/pricing/ https://claude.ai/ or similar

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#37

Post by jonah »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

I asked Google's Gemini and got the following:

    The text contains a logical inconsistency.

    The first sentence states that Joe drove Sue to university, implying Sue is now at university.

    The second sentence says Joe drank tea with Sue in the kitchen afterwards, which suggests Sue is back at home with Joe.

    This creates a contradiction. It's impossible for Sue to be both at university and in the kitchen at the same time.
Then, I asked a follow-on:

    But what if Joe drove home with Sue even though it doesn't say that? (It doesn't say he drove home alone either.)
To which it replied:

    You're absolutely right. The text we have is ambiguous. Here's why:

    * Original interpretation: We naturally assume Joe dropped Sue off and then returned home alone. This is because it's the most common scenario and aligns with the typical purpose of driving someone to a specific location (like a university).

    * Alternative interpretation: As you point out, the text doesn't explicitly state Joe drove back alone. It's entirely possible he gave Sue a ride home after her university visit.

    In this case, the seemingly illogical detail of drinking tea in the kitchen becomes perfectly reasonable.

    This is a great example of how  limited information can lead to multiple interpretations.

    The next sentence or some additional context could clarify the situation. For instance:

    * "Joe dropped Sue off at her dorm and then drove home alone to make himself a cup of tea." (Original interpretation)

    * "After dropping Sue off, Joe drove them both back to their place and they enjoyed a cup of tea together in the kitchen." (Alternative interpretation)

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#38

Post by danielmarkbruce »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

Stop thinking about words. Think about concepts. As soon as you turn words/tokens into high dimensional embeddings and start playing around with them, they stop being words.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#39

Post by namaria »

mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

Occam's Razor says that between the model having somehow developed a persistent internal world model representation and can now reason about it - or is it just big enough in terms of training data, knobs and memory allocation that it can capture more and more language corner cases... Well likely the latter is true.

Re: What Is ChatGPT Doing and Why Does It Work? (2023)

#40

Post by namaria »

prasoonds wrote: post #15
mg wrote: post #5

The better the models get, the harder it is for me to form a mental model of what goes on inside of them. An example of a prompt for which I don't have a good mental model why it works: What do you think about the following text? Joe drove Sue to university. Afterwards he drove home again and drank a tea with her in the kitchen. Older models behaved similar to Markov chains and completely missed that something is log…

This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. Another interesting bit of experiment people did when the GPT-4 class models launched were to test out spatial awareness. For eg, you could describe with words a construction made of blocks, spheres and so on and then ask questions about th…

Stochastic parrot is way too simplistic a dismissal. It's hand wavy.

But I don't get what's so impressive about the nuanced language of a language model that has been given datacenter amounts of compute and virtually all of written word ever put digitalized. Yeah it's the first actually functioning natural language interface. At what cost though. It's completely out of proportion with the benefits and only bubble level 'investing' can justify this.

Post reply on HN