Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

151–160 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#151

Post by ClumsyPilot »

hackinthebochs wrote: post #115
ssivark wrote: post #86

Earlier quoted context omitted.

Statistics doesn’t preclude understanding, but statistics are definitely not enough. For example, uncertainties/probabilities/statistics is original to whether the model incorporates causal/reasoning structure. Any tractable amount of data with the former can’t approximate an ounce of the latter. All breakages will be attributed to “distribution shifts” of the underlying statistical distribution, or other pretty word…

>Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure]. I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. In a predictive language model, at some point the best way to model a text corpus that indirectly references the…

Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that.

The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature in 2 hours you might have travelled across town, across continents, or to the moon. Statistical approach to such problems is dumb.

Re: GPT-3 has no idea what it’s talking about

#152

Post by hackinthebochs »

datameta wrote: post #92
hackinthebochs wrote: post #70

Earlier quoted context omitted.

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

GPT-3 is a language engine, not a reasoning machine. What is understanding, neurologically? At what point can we measure whether an organism or piece of technology is able to be aware of concepts? Does a honeybee (arguably the most intelligent insect) understand that pollen is a necessary component of honey? Or that it is using geometry to convey directions using angles of the sun? Why should we expect a piece of tec…

>Why should we expect a piece of technology with magnitudes less sophistication to be able to mimic higher order lifeforms?

Honeybees apparently have 1 million neurons compared to GPT-3's 175 billion parameters. Granted, there isn't a 1 to 1 correspondence between biological neuron and parameter. And considering much of the honeybees neuron's would be directed towards biological function and control mechanisms, whereas all of GPT-3's parameters are directed towards text prediction, the relevant expressive power of GPT-3 is plausibly much larger.

Re: GPT-3 has no idea what it’s talking about

#153

Post by lumost »

throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. This could realistically only occur if they cost-effectively fine-tune away the more egregious problems in beta - or convince enough investors that their next model with a 100 million dollar price tag will be able to handle something approximating AGI for realistic applications.

This is the same game plan that Self-driving car companies have been playing. The product is only an investment round away, if we just happen to spend more money on bigger models using more data. This will either end with a price tag in the billions that investors are unwilling to pay, or successful monopolies. Allowing additional researchers to perform extensive analysis of the technique is likely to just reveal systematic flaws which increase the risk that the next round of research will produce a successful product, or limit the companies ability to create a monopoly following success.

This isn't necessarily a bad thing for advancing the state of the art, but it does introduce a whole lot of BS into the current state of research.

Re: GPT-3 has no idea what it’s talking about

#154

Post by rytill »

throwawaygh wrote: post #126
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

Simple Markov Chains of the sort you might assign as an undergrad programming assignment can write impressive poetry/captions if you tweak the inputs and cherry-pick outputs. There’s a whole Reply All episode of tech journo types being wowed by 90s text generation tech. Nothing wrong with that; it is what it is. But, do markov chains do few-shot learning? What’s actually unclear to me that there is much economic/scie…

What’s the difference between careful prompt design and any other type of careful design?

Re: GPT-3 has no idea what it’s talking about

#155

Post by dodobirdlord »

ClumsyPilot wrote: post #139
hackinthebochs wrote: post #64

Earlier quoted context omitted.

>If it did have a real understanding, prompt construction wouldn't matter as much This is only true if we assume GPT was never trained on satire or intentionally absurd text. But there's no reason to think this. Because it continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. If you treat GPT as a conversation bot and expect it to call you out when you give it a…

> if continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. Well then you can justify it outputting anything at all.

This is why it’s fair to call some prompt designs bad. With a bad prompt you can’t really judge anything from the response. But better prompt design can uncover interesting behaviors. GPT-3 will generally not reject a prompt as nonsense or unreasonable, and will instead do a straight-faced continuation, probably because it has been trained on plenty of jokes, satires, dream sequences, etc. But if you specifically give it space in the prompt to reject nonsense, it’s actually quite good at rejecting nonsense.

https://arr.am/2020/07/25/gpt-3-uncertainty-prompts

Re: GPT-3 has no idea what it’s talking about

#156

Post by monkpit »

throwawaygh wrote: post #150
hnuser123456 wrote: post #147

Earlier quoted context omitted.

OpenAI never claims GPT-3 is an AGI, do they? If they don't, why should they pander to people who criticize it for not being one? Are there other claims they make, that have been/could/should be refuted? It's simply the most advanced text generator by far. Nothing more, nothing less.

> pander The goal should be pushing science forward, not maximizing brand value. Enabling top scientists in the field to reproduce and probe your work is not "pandering". It's participating in the scientific process.

It’s easy to sit here and say what their goals “should” be, but it doesn’t change the fact that they never claimed it’s an AGI.

Re: GPT-3 has no idea what it’s talking about

#157

Post by DonHopkins »

perl4ever wrote: post #21
kingkawn wrote: post #15

Earlier quoted context omitted.

“every once in a while these systems degenerate into surreal non sequitur nonsense.” Exactly as our minds do

>Exactly as our minds do This rhetorically obscures the fact that when humans do produce similar stuff, it's a recognized sort of pathology that is obviously distinct from normal functioning. https://en.wikipedia.org/wiki/Derailment_(thought_disorder) Example: "I think someone's infiltrated my copies of the cases. We've got to case the joint. I don't believe in joints, but they do hold your body together." https://en…

Could somebody with GPT-3 access please ask it what words come after "person, woman, man, camera"?

Re: GPT-3 has no idea what it’s talking about

#158

Post by [deleted] »

Veedrac wrote: post #135
Barrin92 wrote: post #128

Earlier quoted context omitted.

>because GPT-3 can do simple math It can't actually, and again this is an example of the same issue. This was discussed earlier here[1]. Sometimes it produces correct arithmetic results on addition or subtraction of very small numbers, but again this is likely simply an artifact of training data. On virtually everything else it's accuracy drops to guesswork, and it doesn't even consistently get operations right that…

GPT-3's failure at larger addition sizes is almost fully due to BPE, which is incredibly pathological (392 is a ‘digit’, 393 is not; GPT-3 is also never told about the BPE scheme). When using commas, GPT-3 does OK at larger sizes. Not perfect, but certainly better than should be expected of it, given how bad BPEs are. http://gptprompts.wikidot.com/logic:math

[deleted]

Re: GPT-3 has no idea what it’s talking about

#159

Post by hackinthebochs »

ClumsyPilot wrote: post #151
hackinthebochs wrote: post #115

Earlier quoted context omitted.

>Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure]. I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. In a predictive language model, at some point the best way to model a text corpus that indirectly references the…

Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that. The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature i…

But this isn't pointing to a fundamental limitation of statistical models, only a limitation of the text corpus. If you had a billion pages of text written about some town and the text included descriptions of travel distances and locations, the model should eventually develop a good representation of the town and relative locations. But of course without such a seed of spatial information, it will just make up plausible data. A human would behave similarly when forced to write a story while lacking critical information.

>Statistical approach to such problems is dumb.

Well, expecting your model to extract a spatial representation of the world from text is a dumb approach indeed. We interact with the spatial information much more directly. But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.

Re: GPT-3 has no idea what it’s talking about

#160

Post by mlb_hn »

Veedrac wrote: post #135
Barrin92 wrote: post #128

Earlier quoted context omitted.

>because GPT-3 can do simple math It can't actually, and again this is an example of the same issue. This was discussed earlier here[1]. Sometimes it produces correct arithmetic results on addition or subtraction of very small numbers, but again this is likely simply an artifact of training data. On virtually everything else it's accuracy drops to guesswork, and it doesn't even consistently get operations right that…

GPT-3's failure at larger addition sizes is almost fully due to BPE, which is incredibly pathological (392 is a ‘digit’, 393 is not; GPT-3 is also never told about the BPE scheme). When using commas, GPT-3 does OK at larger sizes. Not perfect, but certainly better than should be expected of it, given how bad BPEs are. http://gptprompts.wikidot.com/logic:math

My thinking there wasn't because of BPEs, I think it's a graph traversal issue.
Post reply on HN