Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

121–130 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#121

Post by hackinthebochs »

skatesor wrote: post #106
hackinthebochs wrote: post #70

Earlier quoted context omitted.

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

It's an inherently limited model of the human brain. It pretends biology and electrochemistry aren't playing some important role 'statistics' cannot. It's GPT which has to do more legwork to be compared to a human brain, not the other way around.

>It pretends biology and electrochemistry aren't playing some important role 'statistics' cannot.

But the issue here isn't about modelling the human, it's about learning a model that captures understanding. To assume that human or animal biology is necessary for understanding is a big leap that isn't supported by the evidence. For one, the relevance of biology and electrochemistry appears to be to influence the function of neurons in various ways. So there's no reason to think that this influence cannot be captured by a model driven by statistical relationships.

Re: GPT-3 has no idea what it’s talking about

#122

Post by [deleted] »

rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

[deleted]

Re: GPT-3 has no idea what it’s talking about

#123

Post by Reimersholme »

detaro wrote: post #88
smeeth wrote: post #75

Why must we keep having this argument? If you do research in the field you know full well that GPT/any other transformer or Bert model is generating text by regurgitating approximate conditional probabilities of words given all the text it has ever seen and the prompt. The neurophysiological concept of “understanding” as most understand it is orthogonal to the way the algorithm actually works. A more useful conversat…

> If you do research in the field you The hype machine is full-on marketing GPT-3 and promised solutions based on it to normal people, so "but researchers know this" is not enough.

Anyone with basic knowledge of the field should understand this though. To me the title of the article read like “The sky is blue”...

Re: GPT-3 has no idea what it’s talking about

#124

Post by chillacy »

datameta wrote: post #92
hackinthebochs wrote: post #70

Earlier quoted context omitted.

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

GPT-3 is a language engine, not a reasoning machine. What is understanding, neurologically? At what point can we measure whether an organism or piece of technology is able to be aware of concepts? Does a honeybee (arguably the most intelligent insect) understand that pollen is a necessary component of honey? Or that it is using geometry to convey directions using angles of the sun? Why should we expect a piece of tec…

There's a wonderful interview where Richard Feynman interrupts an interviewer to explain how "why questions" have layers of understanding

https://fs.blog/2012/01/richard-feynman-on-why-questions/

> Of course, it’s an excellent question. But the problem, you see, when you ask why something happens, how does a person answer why something happens? For example, Aunt Minnie is in the hospital. Why? Because she went out, slipped on the ice, and broke her hip. That satisfies people. It satisfies, but it wouldn’t satisfy someone who came from another planet and knew nothing about why when you break your hip do you go to the hospital. How do you get to the hospital when the hip is broken? Well, because her husband, seeing that her hip was broken, called the hospital up and sent somebody to get her. All that is understood by people. And when you explain a why, you have to be in some framework that you allow something to be true. Otherwise, you’re perpetually asking why. Why did the husband call up the hospital? Because the husband is interested in his wife’s welfare. Not always, some husbands aren’t interested in their wives’ welfare when they’re drunk, and they’re angry.

Re: GPT-3 has no idea what it’s talking about

#125

Post by Veedrac »

smeeth wrote: post #95
Veedrac wrote: post #87

Earlier quoted context omitted.

> regurgitating approximate conditional probabilities of words given all the text it has ever seen and the prompt This is meaningless; you have only described the task. It is equally applicable to a superintelligence as it is of a Markov chain.

Sort of, my point was that human/superhuman intelligence is widely understood by the public to work differently than a markov chain.

GPT-3 also works differently to a Markov chain.

Re: GPT-3 has no idea what it’s talking about

#126

Post by throwawaygh »

rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

Simple Markov Chains of the sort you might assign as an undergrad programming assignment can write impressive poetry/captions if you tweak the inputs and cherry-pick outputs. There’s a whole Reply All episode of tech journo types being wowed by 90s text generation tech. Nothing wrong with that; it is what it is. But, do markov chains do few-shot learning?

What’s actually unclear to me that there is much economic/scientific virtue (NB: different from value) in models that require careful prompt design and curation.

Re: GPT-3 has no idea what it’s talking about

#127

Post by nutanc »

The article is a critical view on GPT-3. Fair. It is well known that Gary Marcus is not a fan of the GPT kind of systems. And he does make some valid points. If you want to look at a better balanced view it actually helps to look at all their prompts [1].

That said, I think it's more of a hype that GPT-3 is moving towards AGI. The actual GPT-3 paper says "Language Models are Few-Shot Learners"[2]. So it's actually surprising that no one has actually done a real analysis of this. Are they really few shot learners? My experiments seem to suggest otherwise.

But for sure, GPT-3 is the best general purpose natural language system out there in the world. I don't think anyone can say otherwise.

[1]https://cs.nyu.edu/faculty/davise/papers/GPT3CompleteTests.h... [2]https://arxiv.org/abs/2005.14165

Re: GPT-3 has no idea what it’s talking about

#128

Post by Barrin92 »

Veedrac wrote: post #119
Barrin92 wrote: post #93

Earlier quoted context omitted.

Marcus might be biased but I don't think you're giving a good refutation, because the fact that GPT-3 gets a lot of things right probabilistically doesn't compensate for the fact that it's not actually understanding what's going on at a semantic level. It's a little bit like some sort of Chinese room, or asking a non-developer to answer you programming questions by looking like something that vaguely resembles your p…

> It's a little bit like some sort of Chinese room, or asking a non-developer to answer you programming questions by looking like something that vaguely resembles your prompt and then picking the most upvoted answer on stackoverflow. Except this isn't how it works. We know it can't be, because GPT-3 can do simple math, despite math being vastly harder with GPT-3's byte pair encoding (it doesn't use base-N, but some a…

>because GPT-3 can do simple math

It can't actually, and again this is an example of the same issue. This was discussed earlier here[1]. Sometimes it produces correct arithmetic results on addition or subtraction of very small numbers, but again this is likely simply an artifact of training data. On virtually everything else it's accuracy drops to guesswork, and it doesn't even consistently get operations right that are more or less equivalent to what it just did before.

If it actually did understand mathematics, it would not be good at adding two or three digit numbers but fail at adding four digit numbers or doing some marginally more complicated looking operation. That is because that sort of mathematics isn't probabilistic. If it had learned actual mathematical principles, it would do it without these errors.

Mathematics doesn't consider of guessing the next language token in a mathematical equation from data, it consists of understanding the axioms of maths and then performing operations according to logical rules.

This problem is akin to the performance of ML in games like breakout. It looks great, but then you adjust the paddle by five pixels and it turns out it hasn't actually understood what the paddle or the point of the game is at all.

[1]https://news.ycombinator.com/item?id=23896326

Re: GPT-3 has no idea what it’s talking about

#129

Post by chillacy »

ssivark wrote: post #86
hackinthebochs wrote: post #70

Earlier quoted context omitted.

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

Statistics doesn’t preclude understanding, but statistics are definitely not enough. For example, uncertainties/probabilities/statistics is original to whether the model incorporates causal/reasoning structure. Any tractable amount of data with the former can’t approximate an ounce of the latter. All breakages will be attributed to “distribution shifts” of the underlying statistical distribution, or other pretty word…

Note that humans have multiple modes of cognition, one of them is very much a stupid, statistical approach that works most of the time.

Kahneman and Tversky came up with questions meant to probe that in humans:

> If a baseball bat and a ball cost a total of $1.10, and the bat costs $1 more than the ball, then how much does the ball cost?

Most people arrive at $0.10 quickly (system 1), but you can't arrive at the right answer if you go by your gut, you have to shift into a different mode of cognition (system 2) in order to compute the correct answer.

Re: GPT-3 has no idea what it’s talking about

#130

Post by moyix »

abiro wrote: post #23
andyljones wrote: post #9

Gary Marcus - the author of this - has previously offered several concrete tests that he felt demonstrated the limitations of the GPT approach. GPT-3 smashed them. https://www.gwern.net/GPT-3#marcus-2020

OpenAI would naturally optimize for the tests published by Marcus as a critique of GPT-2, yet GPT-3 still fails physical reasoning spectacularly (the one test needing casual reasoning the most). There are two broader points here: 1. The lack of independently verifiable evaluation metrics for these type of models should make everyone very skeptical. (Who can afford to retrain GPT-3 from scratch?) 2. I find it difficul…

> OpenAI would naturally optimize for the tests published by Marcus as a critique of GPT-2

It would be difficult for them to do so since Marcus's GPT2 critique came out after they collected the dataset for GPT3.

Marcus's article: Jan 2020

GPT-3 dataset: "Table 2.2 shows the final mixture of datasets that we used in training. The CommonCrawl data was downloaded from 41 shards of monthly CommonCrawl covering 2016 to 2019"

Post reply on HN