Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does GPT obtain its ability? Tracing emergent abilities of language models

yaofu.notion.site

Post reply on HN
Print view View original

91–100 of 205 posts

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#91

Post by ssgh »

rcarr wrote: post #86
ssgh wrote: post #85

Earlier quoted context omitted.

So if we asked GPT to write a book, it would be hallucinating a chain of words without sticking to any coherent plot. However, we could use a "multi-resolution" approach even with today's version of GPT: at the top level we ask it to write a brief plot for the entire novel, at the next level we'll use this plot as the context and ask to outlines sub-plots of the 3 books in our novel, at the third level we'll use the…

It’s interesting because in the writing world there’s a spectrum with plotters at one end and pantsers at the other. Plotters work similarly to what you’ve suggested, starting with a plot and working their way down to the actual writing. Pantsers just start writing ‘by the seat of their pants’ and see what emerges. Stephen King is famously in the latter camp. Most people fall somewhere in between, having a rough plot…

The pantsers can also fit the model I've described. In this case GPT would keep in memory a sliding window of past N=1024 words, like it does today, but in addition to that it would remember the past N paragraph-tokens (symbols that are blurry versions of all the words in that paragraph), the past N chapter-tokens and so on. When generating words, GPT would first generate the next chapter-token, then the next paragraph-token and finally the next word-token.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#92

Post by rileTheNuuters »

srajabi wrote: post #2

Amazing insight, particularly section 6. "- The two important but different abilities of GPT-3.5 are *knowledge* and *reasoning*. Generally, it would be ideal if we could *offload the knowledge part to the outside retrieval system and let the language model only focus on reasoning.* This is because: - The model’s internal knowledge is always cut off at a certain time. The model always needs up-to-date knowledge to an…

How is that amazing?

Restated as “model cannot include info it has not observed” it’s pretty much run of the mill, decades old physics.

It is still a machine under the hood bound by the known laws vetted by experiment.

x86 machines have not taken us beyond the known laws of the shared physical space.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#93

Post by lettergram »

ummonk wrote: post #90
srajabi wrote: post #2

Amazing insight, particularly section 6. "- The two important but different abilities of GPT-3.5 are *knowledge* and *reasoning*. Generally, it would be ideal if we could *offload the knowledge part to the outside retrieval system and let the language model only focus on reasoning.* This is because: - The model’s internal knowledge is always cut off at a certain time. The model always needs up-to-date knowledge to an…

How is hosting the knowledge in a large cloud database any different than hosting the model itself in the cloud? Why the need to run "reasoning" locally?

You could have a locally trained variant that uses a tuned model or set of models, plus local data

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#94

Post by kyle_grove »

rdedev wrote: post #26
adamsmith143 wrote: post #4

Earlier quoted context omitted.

The problem with ChatGPT's "knowledge" is that it isn't trustworthy. It will happily output very confident sounding nonsense, or blatantly incorrect statements. We need a way to verify how accurate it's outputs are

I just had a run in with this yesterday. I asked it to explain box embeddings. It's a pretty niche topic so I didn't expect it to give the right answer. But the answer it gave sounded so confident but it was so wrong. It took a not al vector embeddings approach but replaced that with box. I tried correcting it but it refused to budge and still sounded confident.

Sounds like many humans I know.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#95

Post by refulgentis »

mettamage wrote: post #72
skybrian wrote: post #47

Earlier quoted context omitted.

Yes, this certainly helps. I find it ironic that you can get somewhat better results for nonfiction by giving it more clues about what it's supposed to be pretending to be. It's always pretending, though.

There’s an actual psychological effect for this as well [1]. Authors + uni are in the link. I forgot the name of the effect, don’t have time to do proper research. [1] https://www.themarysue.com/lab-coats-increase-attention/#:~:... .

0 chance this replicates, anything before 2016 I immediately write off. (i.e. clearly pre-replication crisis)

Halycon days of TED talks laundering cute little tidbits that seemed irrational but we all wanted to believe.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#96

Post by squeaky-clean »

Gigachad wrote: post #82
PartiallyTyped wrote: post #79

Earlier quoted context omitted.

The required knowledge is around entity recognition, with "Obama" referring to the 44th POTUS, and not somebody else who happens to have the same surname (and there are multiple of them actually, at least 4 given his family).

Fair, although I’d think it acceptable if the response to your prompt was “which one? I found 4”

It would have way more than 4 people in the search results though. GP said there's at least 4 because there's him, his wife and his kids.

Even knowing Obama is a person is a knowledge-based leap. (To us humans) it's obvious the question means Barack Obama because he's the most notable subject for that name. But how do you prevent your AI from responding that the "Obama JS library is 5 years old"

https://github.com/rgbkrk/obama

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#97

Post by pmayrgundter »

tunnuz wrote: post #51

> The ability of complex reasoning with chain-of-thought is likely to be a magical side product of training on code Seems reasonable, and magical indeed. Can any expert on the topic comment on this hypothesis?

Similar question, tho by my read, chain-of-thought seems to be standalone and the primary breakthrough. I asked the author[1].

I've worked in AI&ML since the 90s and both of these do seem much more intuitively promising than simply making larger nets. This seems borne out by the discussion in his previous post, that in Oct '21 OpenAI ran GPT-3 on it, solving 35% and they estimated:

"it appears likely that the 175 [billion link neural network] model would require at least two additional orders of magnitude of training data to reach an 80% solve rate.”

But instead 85% was achieved within a year using chain-of-thought, not a larger model.[2]

[1]https://twitter.com/PMayrgundter/status/1603224294124920832 [2]https://yaofu.notion.site/A-Closer-Look-at-Large-Language-Mo...

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#98

Post by squeaky-clean »

resource0x wrote: post #6
jimmaswell wrote: post #3

Imagine Wolfram's system with GPT integrated. I hope they're working on that.

Current GPT version is not very good at math, and it's not at all clear whether this can be fixed within a language model. Q: I have 2 cats and 1 horse. How many legs do they have in total? A: If you have two cats and one horse, then the total number of legs they have is 8. Cats typically have four legs, while horses have four legs as well. So two cats would have a total of 8 legs, and when you add the four legs of t…

That's why you'd use ChatGPT to parse the question and wolfram for the actual math.

"2 * (number of legs on a cat) + (number of legs on a horse)"

Result: 12

https://www.wolframalpha.com/input?i=2+*+%28number+of+legs+o...

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#99

Post by rvbissell »

kazinator wrote: post #36
vikp wrote: post #31

Earlier quoted context omitted.

It's unclear to me how you could separate knowledge and reasoning: - Reasoning typically requires base knowledge to work from. A side effect of training reasoning is embedding knowledge into the model parameters. - Even if you offload the search portion (either through outputting special tokens that are postprocessed, or applying the model in multiple steps with postprocessing), you still need embedded knowledge for…

Reasoning is that which knows that it lacks some necessary knowledge, whereas knowledge isn't aware that it lacks some necessary reasoning.

This has to be one of the most insightful sentences I've ever read.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#100

Post by pixl97 »

squeaky-clean wrote: post #96
Gigachad wrote: post #82

Earlier quoted context omitted.

Fair, although I’d think it acceptable if the response to your prompt was “which one? I found 4”

It would have way more than 4 people in the search results though. GP said there's at least 4 because there's him, his wife and his kids. Even knowing Obama is a person is a knowledge-based leap. (To us humans) it's obvious the question means Barack Obama because he's the most notable subject for that name. But how do you prevent your AI from responding that the "Obama JS library is 5 years old" https://github.com/rg…

If it's based on frequency of training data, the ex-president will have far more hits in the training corpus.

Now, on less talked about topics it doesn't sound any different than what happens with people

Q "How old is Tim?"

A "Which Tim are you talking about, you didn't give me crap to work with?"

Post reply on HN