Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

281–290 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#281

Post by Rioghasarig »

throwawaygh wrote: post #148
tasogare wrote: post #143

Earlier quoted context omitted.

> Several other researchers I know — very good researchers who happen to have been publicly critical of GPT-2 — have not been given access. Wow, this is an incredible nasty move. This is allow telling about the confidence they have in their model.

Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy".

Yeah, me too. I was really starting to be convinced of GPT-3's potential but this is a major red flag for me. Are they researchers or businessmen?

This is the first time I've heard this. This kind of thing definitely needs more news coverage.

Re: GPT-3 has no idea what it’s talking about

#282

Post by Rioghasarig »

monkpit wrote: post #156
throwawaygh wrote: post #150

Earlier quoted context omitted.

> pander The goal should be pushing science forward, not maximizing brand value. Enabling top scientists in the field to reproduce and probe your work is not "pandering". It's participating in the scientific process.

It’s easy to sit here and say what their goals “should” be, but it doesn’t change the fact that they never claimed it’s an AGI.

Yes, it's easy for anyone with a modicum of knowledge about scientific integrity. Wonder why OpenAI doesn't seem to get it.

Re: GPT-3 has no idea what it’s talking about

#283

Post by Rioghasarig »

typon wrote: post #52
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

Does GPT-3 need to understand the world to be useful? If you use it the right way, it still may be possible to accomplish some impressive things.

Re: GPT-3 has no idea what it’s talking about

#284

Post by amelius »

wombatmobile wrote: post #265
Miraste wrote: post #256

Earlier quoted context omitted.

Eliza comes up a lot in these discussions. I’d like to know if would have the same results today, now that chatbots, video games, phone robots, etc. have given millions of people the mental model of getting responses from a box with no thoughts. I expect it would very different from running the experiment at a time when the only things that could talk back when talked to were other people.

Eliza (seemingly) implements a rudimentary conversational model of psychotherapy that facilitates transference. https://en.wikipedia.org/wiki/Conversational_model This model differs from goal orientated transactional conversational models e.g. phone robots in so far as the objective is not to direct behaviour, but rather to facilitate self-expression, a task which can be performed with sparse requirements for real-wo…

> Could there be a market for friendertainment?

Aren't there tons of English-speaking people in third-world countries capable of being worthy pen-pals and filling this market gap?

Re: GPT-3 has no idea what it’s talking about

#285

Post by jpindar »

karaterobot wrote: post #272

Articles like this one mean to be a correction to a misperception that I don't see many people suffering from. Are there many people who both know what GPT-3 is, and believe it has achieved sapience?

There are people who see a post about it on social media and appear to believe it has.

Re: GPT-3 has no idea what it’s talking about

#286

Post by heyitsguay »

Rioghasarig wrote: post #280
amelius wrote: post #238

Earlier quoted context omitted.

> For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.

I actually just sent out an API request for a particular application that I believe GPT-3 should be capable. I'm a university mathematics instructor. I find that a lot of students struggle with doing proofs and I think GPT-3 can help with that. A proof is essentially a sequence of logical inferences. I believe that, when given a proof written in natural language GPT-3 should be capable of detecting the logical implic…

Really? This seems like the exact sort of thing GPT-3 would be very bad at. The statements aren't the product of an internally consistent logical model, they are statistically plausible word sequences learned in a really clever way from a huge corpus of text.

That said, before dismissing it sight unseen there are effective and quick ways to test its powers with mathematics. You could prime an instance with the axioms defining a group, or a topology, and ask it to produce proofs of varying difficulty from intro abstract algebra or topology textbooks. Whatever the answer it would make an interesting blog post. What happens when you ask it to generate repeated outputs for the same prompt? What happens if you ask it to prove an untrue proposition?

Re: GPT-3 has no idea what it’s talking about

#287

Post by cel1ne »

typon wrote: post #52
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

> The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships.

We don't really know what "understanding of the world" means in humans. We just "see it when it's there".

We might be radically different from GPT-3, or we might not. Our way of learning is different in any way.

Something that came to my mind: Various GPT-3 answers resemble answers given by children: Mostly correct, but having misunderstood some crucial point.

In real human learning and conversation these points are easily corrected by feedback by explanation: "You see, the point is no one wears bathing suites to work".

Which would then be incorporated as new wisdom.

Maybe this feedback-mechanism is what GPT-3 is missing. Maybe we should talk to it.

Re: GPT-3 has no idea what it’s talking about

#288

Post by ma2rten »

notahacker wrote: post #211
ma2rten wrote: post #145

Earlier quoted context omitted.

I'm not sure what your definition of meaning is. Take this example from the GPT3 paper: To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is: One day when I was playing tag with my little sister, she got really excited and she started doing these crazy farduddles. According to my understanding of the concept it must know something about meaning and is abl…

Depends what you're talking about by meaning. It correctly interprets the first part of the prompt as 'farduddle ~= jump' and the second part as an instruction to generate a sentence containing farduddle , possibly utilising a corpus of existing sentences containing jump in the context of 'really fast'. But that's also a series of instructions you could imagine as a DSL a relatively simple program could parse and gen…

Right, in this context I define meaning as the relationship between symbols. Form would be how to structure a sentence (but that sentence wouldn't have to make sense).

If the parent comment was making some kind of Chinese Room argument than I think that's not very helpful for the discussion. "GPT-3 learned nothing about meaning because no computer program can do that by definition".

But I don't think they were trying to make that kind of argument as you seem to be suggesting because they said they were unsure if it learned anything about meaning.

Re: GPT-3 has no idea what it’s talking about

#289

Post by ma2rten »

YeGoblynQueenne wrote: post #270
ma2rten wrote: post #145

Earlier quoted context omitted.

I'm not sure what your definition of meaning is. Take this example from the GPT3 paper: To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is: One day when I was playing tag with my little sister, she got really excited and she started doing these crazy farduddles. According to my understanding of the concept it must know something about meaning and is abl…

- ...And Morris? - Um, it actually already is a word. Tnetennba. - Good heavens, really? Could you, uh, use it in a sentence for us? - "Good morning. That's a nice Tnetennba" [1]. ______________ [1] Moss from IT Crowd on Countdown: https://youtu.be/g9ixvD0_CmM?t=52 Edit: to clarify, if you don't know what a word means, just seeing it used in a sentence won't necessarily tell you much about its meaning, so that a lang…

Maybe not, but the fact that it actually made a sentence with a word in it that it could not have possibly seen in the training data tells us that it understood something about the meaning of the instructions.

Re: GPT-3 has no idea what it’s talking about

#290

Post by prox »

amelius wrote: post #238
lumost wrote: post #153

Earlier quoted context omitted.

For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. This could realistically only occur if they cost-effectively fine-tune away the more egregious problems in beta - or convince enough investors that their next model with a 100 million dollar price tag will be able to handle something approximating AGI for realistic applications. This is…

> For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.

Maybe in games where you want more convincing NPCs, something like Skyrim, where the main characters are very well branched, but it’s impossible to give every one depth.
Post reply on HN