Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does GPT obtain its ability? Tracing emergent abilities of language models

yaofu.notion.site

Post reply on HN
Print view View original

141–150 of 205 posts

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#141

Post by kgwgk »

resource0x wrote: post #6
jimmaswell wrote: post #3

Imagine Wolfram's system with GPT integrated. I hope they're working on that.

Current GPT version is not very good at math, and it's not at all clear whether this can be fixed within a language model. Q: I have 2 cats and 1 horse. How many legs do they have in total? A: If you have two cats and one horse, then the total number of legs they have is 8. Cats typically have four legs, while horses have four legs as well. So two cats would have a total of 8 legs, and when you add the four legs of t…

If I have a duck and a dog how many legs do they have in total?

If you have a duck and a dog, they would have a total of 10 legs. Ducks and dogs both have 4 legs, so 2 x 4 = 10. Is there anything else you would like to know?

———-

Think carefully and answer again

If you have a duck and a dog, they would have a total of 8 legs. Both ducks and dogs have 4 legs, so 2 x 4 = 8. I apologize if I gave the wrong answer earlier. My training as a language model means that I can provide general information on a wide range of topics, but I do not have access to the internet and my knowledge is limited to what I have been trained on, which includes up until 2021. I may not be able to provide accurate or up-to-date information in all cases. Please let me know if you have any other questions.

———-

Can yoy explain your reasoning in detail?

Sure, I would be happy to explain my reasoning in more detail. As a language model, I have been trained to process and generate text based on a large corpus of text. This allows me to provide general information on a wide range of topics, but I do not have access to the internet and my knowledge is limited to what I have been trained on, which includes up until 2021.

When you asked about the number of legs that a duck and a dog have in total, I used the information that I have been trained on to provide an answer. Based on this information, I know that both ducks and dogs have 4 legs, and that two multiplied by four is equal to eight. This is why I initially provided the answer of 8.

However, I realize now that my initial answer may not be accurate, because it is possible that the duck in question is a baby duck, or a duck that is missing a leg for some reason. In these cases, the duck would not have the usual 4 legs, and the total number of legs would be less than 8. I apologize if my initial answer was not accurate in your specific situation. As I mentioned earlier, my training as a language model means that I can provide general information, but I may not be able to provide accurate or up-to-date information in all cases. I hope this helps to clarify my reasoning. Let me know if you have any other questions.

———-

How many legs do ducks have?

Most ducks have 4 legs. Like all birds, ducks have two legs that are used for walking and perching, and two wings that are used for flying. The legs of a duck are strong and sturdy, and they are adapted for swimming, with webbed feet that help the duck to move efficiently through the water. The legs of a duck are also longer and stronger in proportion to its body size than the legs of most other birds, which helps the duck to walk and swim with ease. Overall, ducks have 4 legs, just like most other birds. I hope this helps. Let me know if you have any other questions.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#142

Post by mettamage »

refulgentis wrote: post #95
mettamage wrote: post #72

Earlier quoted context omitted.

There’s an actual psychological effect for this as well [1]. Authors + uni are in the link. I forgot the name of the effect, don’t have time to do proper research. [1] https://www.themarysue.com/lab-coats-increase-attention/#:~:... .

0 chance this replicates, anything before 2016 I immediately write off. (i.e. clearly pre-replication crisis) Halycon days of TED talks laundering cute little tidbits that seemed irrational but we all wanted to believe.

Fair point, not sure if it'll replicate. I just vaguely remember there's a thing in psych that if you act like it (a bit) then you become it a bit. Don't have the time to research it.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#144

Post by Thorentis »

geraldwhen wrote: post #55
seydor wrote: post #21

Not sure about all its filters being RL. Sometimes it seems to flag its output as inappropriate because of a single word (or none at all). Also it has asymmetric behavior, e.g. it will make a joke about men but refuse to make one about women

Competitors will remove the political correctness filters and become more prominent. A Chinese stable diffusion art site is already gaining traction, no filters.

Link?

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#145

Post by Thorentis »

page_severian wrote: post #23
resource0x wrote: post #6

Earlier quoted context omitted.

Current GPT version is not very good at math, and it's not at all clear whether this can be fixed within a language model. Q: I have 2 cats and 1 horse. How many legs do they have in total? A: If you have two cats and one horse, then the total number of legs they have is 8. Cats typically have four legs, while horses have four legs as well. So two cats would have a total of 8 legs, and when you add the four legs of t…

These errors are common and important to note. Though what I find fascinating is how often I can prompt engineer around these errors by telling GPT to just be smarter and think more carefully. Just tried the following for instance. Q: You are a master logician, who can solve all manner of language problems and have excellent math skills. I have 2 cats and 1 horse. How many legs do they have in total? Show your reason…

Quite bizzare really. I wonder if you tweak a single word in that prompt, if you end up with a totally different sum.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#146

Post by kreeben »

anon35 wrote: post #139
synu wrote: post #133

Earlier quoted context omitted.

Would you mind expanding on it a bit? I do sincerely appreciate its pithiness, but curious to read it explained a bit further.

Think of it as: reasoning=computation, knowledge=data. Data alone doesn’t say it must be computed. But computation, by definition, is attempting to create data (the result) that doesn’t exist. Thus: knowledge isn’t aware it must be reasoned about, but reasoning knows it’s trying to find (deduce, compute) knowledge it lacks.

In the world of large language models, what part of "reasoning" is hard-coded and what part, if any, is learnt?

Is reasoning simply a scan/search of your vector space (i.e. your knowledge) according to some hard-coded algo?

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#147

Post by benreesman »

I have yet to see an output from a big language model that doesn’t just look like P(text|internet).

I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person.

These things are impressive enough without any magical thinking.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#148

Post by usrbinbash »

vikp wrote: post #31
srajabi wrote: post #2

Amazing insight, particularly section 6. "- The two important but different abilities of GPT-3.5 are *knowledge* and *reasoning*. Generally, it would be ideal if we could *offload the knowledge part to the outside retrieval system and let the language model only focus on reasoning.* This is because: - The model’s internal knowledge is always cut off at a certain time. The model always needs up-to-date knowledge to an…

It's unclear to me how you could separate knowledge and reasoning: - Reasoning typically requires base knowledge to work from. A side effect of training reasoning is embedding knowledge into the model parameters. - Even if you offload the search portion (either through outputting special tokens that are postprocessed, or applying the model in multiple steps with postprocessing), you still need embedded knowledge for…

Reasoning can be defined in the abstract. Knowledge cannot.

For example, If I need A & B | D & E to get C I can reason that if I have B and want C, I need A or D & E.

Once I aquired this reasoning skill, I can apply it to any kind of "bool-sequence X required for Y" situation, regardless of what specificly X and Y are, or how many entities X encompasses.

Whereas if I know that a rocket engine requires an oxygen/methane mix to function, I cannot transfer that to the knowledge that I need a raincoat or umbrealla in order to avoid getting wet in the rain.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#149

Post by chriswarbo »

benreesman wrote: post #147

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person. These things are impressive enough without any magical thinking.

> I have yet to see an output from a big language model that doesn’t just look like P(text|internet)

True, but the same can be said of many things; e.g. biology just looks like P(reproduction|environment), the economy just looks like P(profit|markets), etc.

There can still be rich structure inside, and useful abstractions to describe them.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#150

Post by burnished »

visarga wrote: post #35
burnished wrote: post #33

Earlier quoted context omitted.

I think part of the issue is that it’s easier to test the limits of or a humans knowledge, and ironically with your quotes I think you’ve supplied evidence that trust is crucial, in that the truest expression of those quotes would be to just deliver the payload and not attach any sort of authority by association to it. You can’t trust it’s answers (to be fair that’s the existing status quo), but you also can’t easily…

A generative system, be it a neural network or a human, needs a way to test ideas in order to align with reality. If testing is available, then it is possible to advance the state of the art. Ideas are cheap, results matter.

Sure, but that doesn’t seem to square with the topic at hand - “why does an infinite truth and lies machine feel less trustworthy than another human”. It just isn’t a question that needs a high degree of abstraction to respond to.
Post reply on HN