Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does GPT obtain its ability? Tracing emergent abilities of language models

yaofu.notion.site

Post reply on HN
Print view View original

121–130 of 205 posts

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#121

Post by [deleted] »

skybrian wrote: post #58
resource0x wrote: post #48

Earlier quoted context omitted.

Sometimes the meaning of the response is totally different. [ 4 days ago ] The son of my father, but not my brother. Who is he? If a person is the son of the speaker's father but is not the speaker's brother, then that person is the speaker's nephew. A nephew is the son of a person's sibling, so if the speaker's father has a son who is not the speaker's brother, that person is the speaker's nephew. For example, if th…

Yes. GPT doesn't really deal in meanings. Much like autocomplete, it doesn't know what the end of a sentence will be when it starts it. If it randomly chooses different words at the start of a sentence, it may pretend to have a different belief by the end. In reality it doesn't have beliefs any more than a library does. It implicitly contains beliefs (since it's been trained on them) and it can imitate them, but whic…

[deleted]

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#122

Post by rocqua »

vikp wrote: post #31
srajabi wrote: post #2

Amazing insight, particularly section 6. "- The two important but different abilities of GPT-3.5 are *knowledge* and *reasoning*. Generally, it would be ideal if we could *offload the knowledge part to the outside retrieval system and let the language model only focus on reasoning.* This is because: - The model’s internal knowledge is always cut off at a certain time. The model always needs up-to-date knowledge to an…

It's unclear to me how you could separate knowledge and reasoning: - Reasoning typically requires base knowledge to work from. A side effect of training reasoning is embedding knowledge into the model parameters. - Even if you offload the search portion (either through outputting special tokens that are postprocessed, or applying the model in multiple steps with postprocessing), you still need embedded knowledge for…

Humans (or at least I) have the ability to look up knowledge. What about an architecture where the main model has a background conversation with a knowledge model, just like I need to look up things I remember exist, but forget details off. Heck, such a knowledge model would have great value for people aswell.

Using written conversation as an interface between language models feels natural and completely bonkers at the same time.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#123

Post by yunwal »

anon291 wrote: post #111
Digory wrote: post #108

This just confirms I'm not minimally competent in this conversation. Is there a "college freshman" explainer? GPT seems to be doing something incredibly different than prior AI. Is it really a Bayesian "next word" chooser at incredible scale?

You're looking for the paper 'attention is all you need'. Gpt is not a bayesian next word chooser. It does something different.

Not really. Attention is all you need describes a new mechanism used in transformer networks, but the model is still a Bayesian word chooser

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#124

Post by yafbum »

srajabi wrote: post #2

Amazing insight, particularly section 6. "- The two important but different abilities of GPT-3.5 are *knowledge* and *reasoning*. Generally, it would be ideal if we could *offload the knowledge part to the outside retrieval system and let the language model only focus on reasoning.* This is because: - The model’s internal knowledge is always cut off at a certain time. The model always needs up-to-date knowledge to an…

I feel like in my tests, reasoning is currently clearly weaker than knowledge. When asked to provide demonstrations of simple mathematical theorems, I've observed ChatGPT repeatedly confuse assumptions and conclusions, and even get it to "demonstrate" facts that it knows are wrong when asked directly, like 1 = 2.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#125

Post by Gh0stRAT »

amelius wrote: post #13
sebzim4500 wrote: post #9

Earlier quoted context omitted.

It would have got it right if you got it to show its working. That's the trick to doing computations with LLMs, the computation per token is constant so you want to split the computation over many tokens. > I have 2 cats and 1 horse. How many legs do they have in total? show your working. > If you have 2 cats and 1 horse, they would have a total of 10 legs. Cats typically have 4 legs, so 2 cats would have 2 x 4 = >8…

> That's the trick to doing computations with LLMs, the computation per token is constant so you want to split the computation over many tokens. Could you explain this a bit more? What are LLMs? Why is the number of tokens different in your approach?

LLM = Large Language Model. Asking it to show its work will cause it to generate more tokens (you could think of it as the logical leaps between each step being smaller the more detail you go into)

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#126

Post by Donckele »

somewhat_drunk wrote: post #22
1123581321 wrote: post #12

Earlier quoted context omitted.

I've found it gets the math right most of the time in stories. For example: > Write a story about a veterinarian counting the number of total legs on two cats and a horse. He reports the total number of legs to his assistant so he can write them down. As the veterinarian entered the examination room, he was greeted by two playful cats and a majestic horse. He smiled and began his examination, carefully checking the h…

Fucking amazing.

LOL, that outburst fits in with your HN username.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#127

Post by skybrian »

flatline wrote: post #88
skybrian wrote: post #58

Earlier quoted context omitted.

Yes. GPT doesn't really deal in meanings. Much like autocomplete, it doesn't know what the end of a sentence will be when it starts it. If it randomly chooses different words at the start of a sentence, it may pretend to have a different belief by the end. In reality it doesn't have beliefs any more than a library does. It implicitly contains beliefs (since it's been trained on them) and it can imitate them, but whic…

And yet it is often able to make surprising references to previous text. This is not just a markov chain, and is capable of what the author describes as chain of thought. I think there are deeper relationships encoded in the model that allow it to keep to a consistent narrative for a very long time. Its beliefs may change between queries but do not, generally, within the context of a single conversation.

The attention mechanism lets it look backwards to "understand" what was said before and predict what could possibly come next. Whatever consistency it has is due to studying the preceding text.

Thinking ahead is different. All it needs to do is calculate the probability that there is any reasonable completion starting with a particular word. It doesn't need to decide what it's going to say beyond that; it can decide later.

Have you ever played a game where players take turns adding one more word to a sentence? When it's your turn and you're choosing the next word, you don't need to think ahead very much. Also, you don't necessarily need have the same thing in mind as the player who went before you.

In improv there is a "yes, and" where you are always building on what happened before. These algorithms are doing improv all the time.

The algorithm doesn't know or care who wrote the words that came before. It will find a continuation regardless.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#128

Post by amilios »

esprehn wrote: post #61
resource0x wrote: post #43

Can anyone explain why the GPT response to the same question 4 days ago is different from today's: [ 4 days ago] > Can I learn to sing like Pavarotti? It is possible to learn to sing like Pavarotti, but it will require a lot of practice, dedication, and training. Pavarotti was a renowned opera singer known for his incredible vocal range, powerful voice, and technical mastery. He had many years of training and experie…

This is by design (and implemented as explained in the sibling comments). The reason for the randomness is to avoid returning the same thing all the time. For example if you ask it "write me a poem" you don't want it to return the exact same one each time. You can also set the "temperature" parameter to 0 to remove the non determinism (though I'm not sure that's exposed in the web interface).

It's not, not in the ChatGPT web interface at least. It's just a chat prompt, that's it. Can't control any sampling parameters at all, not temperature, nor top-P or top-K-based selection, nothing. I guess OpenAI is aiming to keep things simple for now. And to be fair, this choice probably does open the demo up to more users who would otherwise be confused at all the knobs and dials. Although they could just hide it by default or something

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#129

Post by astrange »

typon wrote: post #119
astrange wrote: post #106

Earlier quoted context omitted.

It'd be pretty surprising if you could quantize a text model and have it still work. It has to be using those lower bits to store text; it's not like you can round a letter up or down.

It's not storing any text? The weights are floating point numbers - the "text" is in some extremely high dimensional embedding space.

Of course it's storing text. GPT was trained for less than one epoch; they just continually throw new text in there and it mostly just remembers it (= learns it = compresses it). It's not simply "a high dimensional embedding" because words aren't differentiable; you'll get different words if you round off your "coordinates".

If you go to https://beta.openai.com/playground/ and prompt it "Read me the book Alice in Wonderland" it will quote you word for word the original book.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#130

Post by lovecg »

kazinator wrote: post #36
vikp wrote: post #31

Earlier quoted context omitted.

It's unclear to me how you could separate knowledge and reasoning: - Reasoning typically requires base knowledge to work from. A side effect of training reasoning is embedding knowledge into the model parameters. - Even if you offload the search portion (either through outputting special tokens that are postprocessed, or applying the model in multiple steps with postprocessing), you still need embedded knowledge for…

Reasoning is that which knows that it lacks some necessary knowledge, whereas knowledge isn't aware that it lacks some necessary reasoning.

Fascinating - makes me think of the distinction between dreaming and being awake. Only in the latter state one can tell the difference.
Post reply on HN