Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does GPT obtain its ability? Tracing emergent abilities of language models

yaofu.notion.site

Post reply on HN
Print view View original

151–160 of 205 posts

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#151

Post by ajuc »

aeternum wrote: post #27
adamsmith143 wrote: post #4

Earlier quoted context omitted.

The problem with ChatGPT's "knowledge" is that it isn't trustworthy. It will happily output very confident sounding nonsense, or blatantly incorrect statements. We need a way to verify how accurate it's outputs are

Whose knowledge is trustworthy? We've somehow come to associate certain institutions or scientific authorities with truth when that is about the furthest from real science: "Have no respect whatsoever for authority; forget who said it and instead look what he starts with, where he ends up, and ask yourself, Is it reasonable?" -Richard P. Feynman "One of the great commandments of science is, "Mistrust arguments from a…

I, too, always recreate double blind experiments before I take the drugs my doctor gives me :)

I also double-check the transistors in my computer work correctly before I run any code on them, and of course I re-derive the physics to be able to do that :)

In practice you are an expert in a very small domain (if any) and in all the other domains you have no choice but to accept somebody's authority.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#152

Post by nathias »

roywiggins wrote: post #32
nathias wrote: post #7

Earlier quoted context omitted.

so, much like other knowledge sources?

Most knowledge sources don't make up totally fictional citations to nonexistent sources. Or, if they do, nobody uses them for anything serious. Even Wikipedia citations will get removed if they point to URLs that never existed.

if we focus on the best sources, even in studies a lot of research can't be replicated, and if we focus on te most common ones like newspapers and tv, I'd say most of it is made up or might as well be

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#153

Post by Andrew_nenakhov »

page_severian wrote: post #23
resource0x wrote: post #6

Earlier quoted context omitted.

Current GPT version is not very good at math, and it's not at all clear whether this can be fixed within a language model. Q: I have 2 cats and 1 horse. How many legs do they have in total? A: If you have two cats and one horse, then the total number of legs they have is 8. Cats typically have four legs, while horses have four legs as well. So two cats would have a total of 8 legs, and when you add the four legs of t…

These errors are common and important to note. Though what I find fascinating is how often I can prompt engineer around these errors by telling GPT to just be smarter and think more carefully. Just tried the following for instance. Q: You are a master logician, who can solve all manner of language problems and have excellent math skills. I have 2 cats and 1 horse. How many legs do they have in total? Show your reason…

Suddenly, all those movies and tv shows where characters tell robots to concentrate start to make sense.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#154

Post by benreesman »

chriswarbo wrote: post #149
benreesman wrote: post #147

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person. These things are impressive enough without any magical thinking.

> I have yet to see an output from a big language model that doesn’t just look like P(text|internet) True, but the same can be said of many things; e.g. biology just looks like P(reproduction|environment), the economy just looks like P(profit|markets), etc. There can still be rich structure inside, and useful abstractions to describe them.

Yeah and I hope I didn’t come off like I was trying to knock the technical achievement: it’s remarkable along multiple dissensions: at a minimum technical, infrastructural, mathematical (you don’t throw 25-50k A100s for months at something without running some serious numbers first).

It’s possible that I’ve just fallen too far under the influence of Deutsch and Marletto, but as someone who has worked on systems like this I’m rather skeptical that one of these things is going to break the gridlock between quantum theory and general relatively any time soon.

There’s no reason why one couldn’t principle but I’ve babysat enough big ML systems that I tend to think in terms of “how do we keep this thing from shitting itself” rather than “damn this thing is going to win a Fields medal if I turn my back on it”.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#155

Post by synu »

anon35 wrote: post #139
synu wrote: post #133

Earlier quoted context omitted.

Would you mind expanding on it a bit? I do sincerely appreciate its pithiness, but curious to read it explained a bit further.

Think of it as: reasoning=computation, knowledge=data. Data alone doesn’t say it must be computed. But computation, by definition, is attempting to create data (the result) that doesn’t exist. Thus: knowledge isn’t aware it must be reasoned about, but reasoning knows it’s trying to find (deduce, compute) knowledge it lacks.

Thanks for that!

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#156

Post by ImprobableTruth »

benreesman wrote: post #147

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person. These things are impressive enough without any magical thinking.

The article links multiple papers on chain of thought reasoning. There are tasks that language models struggle with, but when you ask it to explain its reasoning, certain large language models do much better than the scaling for the normal prompt would suggest. Calling this an 'ascribed quality' is crazy, it's just an observation and says nothing about the internals. Hell, you could even test it yourself if you don't trust the papers.

Saying that it just looks like P(text|internet) is a tautology, it's a text predictor trained on the internet. This doesn't tell you anything about why phenomena like the above occur, or why it occurs only in large language models and only in some of them.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#157

Post by loa_in_ »

benreesman wrote: post #147

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person. These things are impressive enough without any magical thinking.

Gpt-3 plays the Wikipedia game pretty well. Ask it to follow and list all articles between one article and other by inter-article links. I liked it because most have Wikipedia corpus already learned.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#158

Post by Closi »

jfoutz wrote: post #102
vikp wrote: post #31

Earlier quoted context omitted.

It's unclear to me how you could separate knowledge and reasoning: - Reasoning typically requires base knowledge to work from. A side effect of training reasoning is embedding knowledge into the model parameters. - Even if you offload the search portion (either through outputting special tokens that are postprocessed, or applying the model in multiple steps with postprocessing), you still need embedded knowledge for…

> It's unclear to me how you could separate knowledge and reasoning yeah, me too. There are a few cases I'd point your attention to. 1, preschool toys, kids somehow manage to put the square peg in the square hole. I mean, they may chew on them or push them around, but there's a "moment of magic" when they make it all click together. Maybe there's some implicit knowledge there, I know I played games like that, but I d…

Different people might think diffrent, but when solving complex problems I do think I have seperate "think about / gather the facts" and "formulate the solution" phases.

I don't think about the totality of facts in the world - I think my brain is mentally extracting the facts that are relevant to the problem and then reason about those facts.

There is certinaly back/forth though, but I think I go "here is a bit of information, how does that apply? ok but what about this fact? ok here is how that would apply considering something else..." but I think this is still a gather -> solve -> gather -> solve

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#159

Post by benreesman »

ImprobableTruth wrote: post #156
benreesman wrote: post #147

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person. These things are impressive enough without any magical thinking.

The article links multiple papers on chain of thought reasoning. There are tasks that language models struggle with, but when you ask it to explain its reasoning, certain large language models do much better than the scaling for the normal prompt would suggest. Calling this an 'ascribed quality' is crazy, it's just an observation and says nothing about the internals. Hell, you could even test it yourself if you don't…

I hope my comment didn’t imply that I’m anything short of impressed: it’s truly a feat.

But it’s one thing to regurgitate a causal structure that’s been said over and over and quite another to propose a novel and falsifiable hypothesis.

Telling a novel-ish just so story is well within the reach of a precocious 8 year old, and that’s selling the bright year old short.

Let’s see one of things propose a novel hypothesis before we get our drawers moist.

Re: How does GPT obtain its ability? Tracing emergent abilities of language models

#160

Post by [deleted] »

benreesman wrote: post #147

I have yet to see an output from a big language model that doesn’t just look like P(text|internet). I understand that it’s very easy to ascribe all kinds of qualities to these things, but when the corpus is the Internet, the log likelihood of it sounding like a person is not so different from the corpus sounding like a person. These things are impressive enough without any magical thinking.

[deleted]
Post reply on HN