Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

141–150 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#141

Post by chubot »

mordymoop wrote: post #89

I would love to see a real critique of the potential of transformer models that doesn't use the words "semantic", "syntactic", "symbolic", "know", "meaning", "understand" or "think(ing)/thought". Predicting what it can and can't do, or might and might not be able to do, lets us productively talk about potential limitations.

What are the models useful for?

As best I can tell the only thing would be some kind of GMail-like auto-reply which a human consciously edits, and which has little consequence if it's wrong.

Are the models useful for customer service? Like reading a manual or knowledgebase and then answering a customer's questions about a product, and troubleshooting problems? Like about your Android phone, which lots of people have trouble using?

That would be a trillion dollar business. As best as I can tell that's completely beyond GPT-3 and requires a huge breakthrough, which may or may not happen.

Re: GPT-3 has no idea what it’s talking about

#142

Post by mlb_hn »

6gvONxR4sf7o wrote: post #138

I'm getting impatient with criticisms of ML models that are already covered in the papers introducing the models. OP is basically trying to get it to do what the GPT3 paper calls zero-shot inference. In the paper, it's pretty bad at zero shot inference across the board. And given what it does and how it was trained, that's unsurprising. And the point they're trying to make (that it can fail spectacularly) is also cov…

Also, better prompt design if you have make implicit meaning explicit can improve the WiC score (http://gptprompts.wikidot.com/linguistics:word-in-context) and ANLI score (http://gptprompts.wikidot.com/linguistics:anli).

Re: GPT-3 has no idea what it’s talking about

#143

Post by tasogare »

throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access.

Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.

Re: GPT-3 has no idea what it’s talking about

#144

Post by ClumsyPilot »

gweinberg wrote: post #132
pwinnski wrote: post #104

Earlier quoted context omitted.

Overfitting to the edge case is missing the point. I remind readers of this continuation: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, whi…

To be fair to the AI, stirring lemonade with a cigarette is so batshit insane that there really can't be a sensible continuation.

So basically GPT is useless if I feed it any text about life in Russia?

Re: GPT-3 has no idea what it’s talking about

#145

Post by ma2rten »

pwinnski wrote: post #97
ma2rten wrote: post #59

Earlier quoted context omitted.

GPT-3 doesn't care about anything except predicting the next token. It learned something about structure and meaning in the process.

About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.

I'm not sure what your definition of meaning is. Take this example from the GPT3 paper:

To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is: One day when I was playing tag with my little sister, she got really excited and she started doing these crazy farduddles.

According to my understanding of the concept it must know something about meaning and is able to reason about it if it was able to generate this.

Re: GPT-3 has no idea what it’s talking about

#146

Post by mordymoop »

chubot wrote: post #141
mordymoop wrote: post #89

I would love to see a real critique of the potential of transformer models that doesn't use the words "semantic", "syntactic", "symbolic", "know", "meaning", "understand" or "think(ing)/thought". Predicting what it can and can't do, or might and might not be able to do, lets us productively talk about potential limitations.

What are the models useful for? As best I can tell the only thing would be some kind of GMail-like auto-reply which a human consciously edits, and which has little consequence if it's wrong. Are the models useful for customer service? Like reading a manual or knowledgebase and then answering a customer's questions about a product, and troubleshooting problems? Like about your Android phone, which lots of people have…

GPT-3 involves zero fine-tuning or customization to any purpose. It’s not attempting to be a product, but rather a platform people can use to explore the possibilities of products. With fine-tuning it could do all sorts of domain-specific things that it can only passably do now. We suspect this to be true because we’ve seen how other systems behave with and without fine-tuning.

Re: GPT-3 has no idea what it’s talking about

#147

Post by hnuser123456 »

tasogare wrote: post #143
throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access. Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.

OpenAI never claims GPT-3 is an AGI, do they? If they don't, why should they pander to people who criticize it for not being one? Are there other claims they make, that have been/could/should be refuted? It's simply the most advanced text generator by far. Nothing more, nothing less.

Re: GPT-3 has no idea what it’s talking about

#148

Post by throwawaygh »

tasogare wrote: post #143
throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access. Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.

Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy".

Re: GPT-3 has no idea what it’s talking about

#149

Post by akiselev »

hackinthebochs wrote: post #70
skatesor wrote: post #53

Earlier quoted context omitted.

GPT doesn't have an 'understanding' class or a 'reasoning' function or whatever. It's a really well put together piece of statistics and sentences like these show it doesn't really have a concept of 'making sense'. You can use your much more advanced human brain to visibly see where it put in random variables (cigarette) and where it borrowed pieces of sentences (but it turned out to be too sour). You can see it made…

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

> But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding.

It's a matter of scale. Adult human brains aren't just trained for 18 years with a static dataset - they've got hundreds of millions of years of mammalian evolution and fault tolerance built into them. Our brains self-reconfigure in response to external stimuli as we age, to the point where we can (in rare cases) lose significant fractions without becoming a vegetable. The biochemistry of a single neuron is likely oodles more complex than the most complex AI we've made.

It's like going from an analysis of a small family to an analysis of a civilization: at some point the sample size crosses a threshold and emergent phenomena start to dominate the system. Yes, it's basically all just statistics (ignoring quantum hocus pocus and the supernatural) but we still don't understand how to go from statistics to intelligence.

Re: GPT-3 has no idea what it’s talking about

#150

Post by throwawaygh »

hnuser123456 wrote: post #147
tasogare wrote: post #143

Earlier quoted context omitted.

> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access. Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.

OpenAI never claims GPT-3 is an AGI, do they? If they don't, why should they pander to people who criticize it for not being one? Are there other claims they make, that have been/could/should be refuted? It's simply the most advanced text generator by far. Nothing more, nothing less.

> pander

The goal should be pushing science forward, not maximizing brand value. Enabling top scientists in the field to reproduce and probe your work is not "pandering". It's participating in the scientific process.

Post reply on HN