Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

231–240 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#231

Post by rirarobo »

monkpit wrote: post #197
rirarobo wrote: post #180

Earlier quoted context omitted.

Hmm a sensible continuation to an absurd situation? Sounds like fun fiction. > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it promptly dissolved into my drink, creating a most unpleasant concoction, with an aroma which evoked memories of my gran…

Is your argument that a human can write better prompted fiction than gpt3?

No, I was just addressing the claim regarding the prompt itself, that "stirring lemonade with a cigarette is so batshit insane that there really can't be a sensible continuation".

That seemed like a fun challenge to me. Sorry I got a little carried away trying to come up with an interesting continuation!

Re: GPT-3 has no idea what it’s talking about

#232

Post by smsm42 »

bigyikes wrote: post #99
typon wrote: post #52

Earlier quoted context omitted.

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

Lacking “understanding” doesn’t make GPT-3 less impressive and also doesn’t make comparisons to human abilities unwarranted. I read the prompt, and I expected that this was the beginning of some kind of fiction. In my mind, it sounded like I was reading the beginning of a somebody’s dream. What does it even mean to understand something? Because naively, it looks very much like GPT-3 and I have a shared understanding…

> Lacking “understanding” doesn’t make GPT-3 less impressive

Yes it does. A model that latches on superficial frequentist links between words is much less impressive than one that would understand what those words actually mean, and the latter is how most humans use words. The former is just chinese-rooming, the latter is understanding. Of course, a model that is chinese-rooming something like a coherent text is impressive, but it is less impressive than one that would demonstrate actual grasp of the fact that words mean something.

Re: GPT-3 has no idea what it’s talking about

#233

Post by visarga »

DonHopkins wrote: post #157
perl4ever wrote: post #21

Earlier quoted context omitted.

>Exactly as our minds do This rhetorically obscures the fact that when humans do produce similar stuff, it's a recognized sort of pathology that is obviously distinct from normal functioning. https://en.wikipedia.org/wiki/Derailment_(thought_disorder) Example: "I think someone's infiltrated my copies of the cases. We've got to case the joint. I don't believe in joints, but they do hold your body together." https://en…

Could somebody with GPT-3 access please ask it what words come after "person, woman, man, camera"?

Q: What comes after "person, woman, man, camera"

A: person, woman, man, camera, lens, light, film, lab, darkroom.

A: person, woman, man, camera, dog, cat, horse

A: person, woman, man, camera, camera, camera, camera

Re: GPT-3 has no idea what it’s talking about

#234

Post by spacecity1971 »

ctoth wrote: post #191

I keep wanting to write a long explanation of just why this is so... silly? to read? But Gwern has already done the hard work. [0] The only other bit I'd like to mention is that GPT-3 uses exactly none of the new techniques that have been coming out in the last two years that would have significant impact on text generation. From working methods to apply GANs to text, to far more efficient transformer models that can…

Yes, this! The point being missed by most is the very real possibility that the Scaling Hypothesis is true. If it is, then we're seeing some kind of reasoning intelligence emerge. GPT-3 obviously isn't there yet. Unless it's faking it (Yudkowsky)...

Re: GPT-3 has no idea what it’s talking about

#235

Post by smsm42 »

visarga wrote: post #210
typon wrote: post #52

Earlier quoted context omitted.

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

GPT-3 is a statistical model of text sequences, it has just textual understanding of the world. But the funny thing is that it can do lots of tasks without explicit training, and that is something amazing, it shows a path forward. In order to have real understanding it needs to be an embodied agent that interacts with the world like us, and has goals and needs like us.

It is impressive, but I'm not sure why doing stuff without training is such a good thing. Giving a choice, I'd rather have a model that works better with extensive domain-driven training than one that works worse without it. After all, when choosing an expert, you usually go for one that has the best experience in the field, not the one that can speak most eloquently on the widest variety of subjects (unless we're talking about politicians, there everybody does the opposite for some reason).

Re: GPT-3 has no idea what it’s talking about

#236

Post by visarga »

bra-ket wrote: post #137

The article is of course right but also a bit silly. Language models like GPT-X are producing grammatically correct sentences, along the lines of "Colorless green ideas sleep furiously". The NLP research more or less solved the old syntax problem using 'distributional semantics' but 'semantics' is a misnomer, it's all about syntax. In fact the most useful part of the article for me is that they mentioned Douglas Summ…

Yes, how dare these machine learning scientists use backprop? I mean, there are better methods our there, right?

Re: GPT-3 has no idea what it’s talking about

#237

Post by hackinthebochs »

akiselev wrote: post #149
hackinthebochs wrote: post #70

Earlier quoted context omitted.

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

> But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. It's a matter of scale. Adult human brains aren't just trained for 18 years with a static dataset - they've got hundreds of millions of years of mammalian evolution and fault tolerance built into them. Our brains self…

I'm not seeing an argument here. Mammalian evolution is great. So is backprop. They're both methods to efficiently search the state-space of parameters. If your argument is scale, you have to argue how scale precludes a sufficiently general learning algorithm and enough computation from capturing the underlying structure.

>The biochemistry of a single neuron is likely oodles more complex than the most complex AI we've made.

There's a lot of biochemistry in service to supporting the neurons existence exclusive from its functional properties for the brain. In terms of a neurons I/O mapping, deep learning has that covered[1]

>Yes, it's basically all just statistics... but we still don't understand how to go from statistics to intelligence.

But this assumes intelligence isn't an emergent phenomenon of sufficiently general learning. GPT-3 suggests this might be the case.

[1] https://www.biorxiv.org/content/10.1101/613141v1.full.pdf

Re: GPT-3 has no idea what it’s talking about

#238

Post by amelius »

lumost wrote: post #153
throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. This could realistically only occur if they cost-effectively fine-tune away the more egregious problems in beta - or convince enough investors that their next model with a 100 million dollar price tag will be able to handle something approximating AGI for realistic applications. This is…

> For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model.

On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.

Re: GPT-3 has no idea what it’s talking about

#239

Post by voces »

ClumsyPilot wrote: post #223
mordymoop wrote: post #146

Earlier quoted context omitted.

GPT-3 involves zero fine-tuning or customization to any purpose. It’s not attempting to be a product, but rather a platform people can use to explore the possibilities of products. With fine-tuning it could do all sorts of domain-specific things that it can only passably do now. We suspect this to be true because we’ve seen how other systems behave with and without fine-tuning.

you are sidestepping the question above - what can it be used for? What kind of fine-tuning is required? Because to me it looks like 'fixing catastrophic errors' not fine-tuning - as anywhere where the text can deliver value, it has to be correct. AWS/Azure are a platform, and they work reliably. What kind of platform 'sometimes works'?

Language models can help the disabled communicate better and faster (fine-tune on their online output, then the BCI can offer better continuations, so the patient does not have to type as much).

Fine-tune on educational materials, and a language model could be a 24/7 assistant to students.

Fine-tune on psychology or medicine data, and a language model could tell you of all medicine interactions, or act as a better ELIZA, and "socialize with" - and "support" people with depression or trauma.

Fine-tune on etiquette and social norms, and autistic people could ask questions without being ashamed.

These are some positive use cases. There are also neutral use cases (seeding a new social platform with autogenerated comments), and negative use cases (SEO spam, fake reviews, scaling up disinformation campaigns).

Edit: not complaining, but no idea why this was downvoted. Would be helpful to state why, so I don't make this mistake in the future (I am optimizing for useful replies).

Re: GPT-3 has no idea what it’s talking about

#240

Post by Ngunyan »

throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

> So, other than “GPT-3 isn’t an AGI” [1], I’m not sure what to take away from this article

This is the takeaway:

"[GPT-3 don’t learn about the world—they learn about text and how people use words in relation to other words. What it does is something like a massive act of cutting and pasting, stitching variations on text that it has seen, rather than digging deeply for the concepts that underlie those texts.

Its performance is unreliable, causal understanding is shaky, and incoherence is a constant companion. GPT-2 had problems with biological, physical, psychological, and social reasoning, and a general tendency toward incoherence and non sequiturs. GPT-3 does, too.

All GPT-3 really has is a tunnel-vision understanding of how words relate to one another; it does not, from all those words, ever infer anything about the blooming, buzzing world."

Post reply on HN