Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

211–220 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#211

Post by notahacker »

ma2rten wrote: post #145
pwinnski wrote: post #97

Earlier quoted context omitted.

About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.

I'm not sure what your definition of meaning is. Take this example from the GPT3 paper: To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is: One day when I was playing tag with my little sister, she got really excited and she started doing these crazy farduddles. According to my understanding of the concept it must know something about meaning and is abl…

Depends what you're talking about by meaning.

It correctly interprets the first part of the prompt as 'farduddle ~= jump' and the second part as an instruction to generate a sentence containing farduddle, possibly utilising a corpus of existing sentences containing jump in the context of 'really fast'. But that's also a series of instructions you could imagine as a DSL a relatively simple program could parse and generate a satisfactory response to. Which I believe the OP is classing as 'structure' since it's just performing translations based on familiar syntax. Understanding the concept of 'jumping' is a step further, before we get into the more philosophical stuff about qualia and whether things that can't jump can ever truly understand the experience of jumping...

Re: GPT-3 has no idea what it’s talking about

#212

Post by voces »

Pretty meta, but I thought it was relevant here. We are familiar with Brandolini's law:

> The amount of energy needed to refute bullshit is an order of magnitude bigger than to produce it.

This can be illustrated with math or logic statements. To refute the program "1 + 1 = 3" you need to, at minimum, state "1 + 1 != 3", and such a program is always lengthier. A fuller refutation could be "1 + 1 != 3, 1 + 1 = 2", more than twice as long as the bullshit statement.

What's happening here is sort of an inverse Brandolini's law: 35 world-class computer scientists use a massive amount of programming and compute to come up with a new language model trained on massive amounts of data. The trained weights don't even fit into memory. Impressive NLP progress.

Then Gary Marcus comes around and states "Not AGI!". Not one of the computer scientists stated that they delivered AGI. But some tech journalists did. So OpenAI is guilty by association. Even though Altman came out to temper the hype and expectations. That's like proving the Poincaré conjecture, and someone dissing your research, because "1 + 1 != 3".

Re: GPT-3 has no idea what it’s talking about

#213

Post by zaptrem »

coolspot wrote: post #208
sillysaurusx wrote: post #195

Earlier quoted context omitted.

Seems spot-on. One trick to estimate a startup's burn rate is to multiply their number of employees by $200k. It's not too accurate, but it's within the ballpark. So how many employees does OpenAI have? Supposing they have 500, that's a burn rate of $100M/yr. 250 employees, $50M/yr. 100 employees, $20M/yr.

I believe significant share of GPT-3 cost is machine-hours that were spent training this model - months of hundreds top-tier NVidia machines. Edit: estimates range from $2-$5MM to $15MM https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_t...

I thought MS was giving them Azure GPU instances for free?

Re: GPT-3 has no idea what it’s talking about

#214

Post by visarga »

pwinnski wrote: post #97
ma2rten wrote: post #59

Earlier quoted context omitted.

GPT-3 doesn't care about anything except predicting the next token. It learned something about structure and meaning in the process.

About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.

The old debate about syntax and semantics. I would say syntax, when embodied in the world gains meaning. Meaning comes from the context of the agent and its goals.

Re: GPT-3 has no idea what it’s talking about

#215

Post by samatman »

wizzwizz4 wrote: post #181
tck42 wrote: post #39

Earlier quoted context omitted.

Purely my opinion, but this is a static function right? Wouldn't anything conscious require some sort of feedback loop, where observations, either internal or external, cause an update to the model for you to even start considering if it's conscious or not?

Are humans with total inability to form short-term memories not conscious, then? During its training period, there was a feedback loop like you describe.

That's a difficult question to answer but I would have to say "no".

When I had my wisdom teeth out, it was under "deep sedation". They use local anaesthetics, and an additional cocktail which produces sedation, but also, prevents the formation of memories.

I have one memory that got through, of a molar being ripped out: but for the most part, my sense of that experience is that I wasn't conscious.

The thing is, if you ask someone under deep sedation to raise their right hand, they'll do it. It's like asking if you were conscious during a dream which you've completely forgotten: ...kinda? not in the usual sense in which we mean the word though.

Re: GPT-3 has no idea what it’s talking about

#216

Post by visarga »

canjobear wrote: post #189
chillacy wrote: post #134

Earlier quoted context omitted.

This is the thesis of Bender 2020 https://www.aclweb.org/anthology/2020.acl-main.463.pdf when evaluating GPT-2. They distinguish between meaning and form. We don't _really_ know what the physical manifestations of meaning and form are in the brain... they're just concepts we invented. If anything, GPT-3 is suggesting that either: 1. Tasks which were previously thought to require meaning actually turn out only to requ…

It seems like Bender & Koller's argument can just as easily prove that humans don't have representations of meaning either. Our brains get input from multiple "languages"---conventional natural languages as well as the neural codes that represent our sensory inputs. All we have access to is the form of these inputs, and so we can never learn the underlying "meaning".

Meaning comes from the game, not from the brain. It's the relation between agent and environment that creates meaning.

Re: GPT-3 has no idea what it’s talking about

#217

Post by visarga »

pwinnski wrote: post #104
kzrdude wrote: post #96

Earlier quoted context omitted.

I think it would be fair to say that either outcome would be an understandable continuation of the story.

Overfitting to the edge case is missing the point. I remind readers of this continuation: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, whi…

Tried a GPT-3 completion:

> At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because ...

> ... the cigarette burned a hole in the tablecloth. I didn’t have another one, so I used my finger to stir it and then licked my finger.

> After that, I went into the living room where everyone was dancing. They were all dressed up—even the dog—but they sure could dance! And they looked like they were having fun. But when I tried to join them, they wouldn’t let me because I wasn’t wearing shoes or socks.

> I went back into the kitchen to get a drink. But when I turned on the water, it sprayed all over me and my clothes got soaked. So I took off all my clothes and put them in the dryer. But then I realized that there was no more room in the dryer for anyone else’s clothes, so we had to take everything out of it and hang it up to dry.

Seems pretty reasonable to me.

Re: GPT-3 has no idea what it’s talking about

#218

Post by evrydayhustling »

typon wrote: post #52
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

Just because something doesn’t display understanding by responding to your expectation doesn’t mean it doesn’t possess understanding . If you ran into my office with these prompts, the response to each would be “what the hell are you doing in my office?” All behavior is contextualized, and GPT-3’s native context is predicting continuous text, not answering questions.

It’s a distracting anthropomorphism to even attempt ascribing “understanding” to a model like GPT-3. An assessment of its useful capabilities should be through an honest effort to get it to do something - and should of course include consideration of the effort/intelligence required to do so. Marcus knows enough to know this set up is inappropriate, so the article reads as disingenuous.

Re: GPT-3 has no idea what it’s talking about

#219

Post by Veedrac »

abiro wrote: post #105
SpicyLemonZest wrote: post #49

Earlier quoted context omitted.

(1) I certainly agree with. But Marcus doesn't claim skepticism about GPT-3s intelligence; he claims that his evaluation metrics definitively show it doesn't understand the text it outputs or know anything about the world. (2) is, I think, a misunderstanding. People who believe GPT-3 is producing intelligent answers generally believe it can represent causal relationships.

Fair points. For the record, re 2: The GPT family of models (and all neural networks for that matter) can estimate P(X | Y), but have no way of computing whether X -> Y or X <- Y.

A computation can represent causality without being made of causality-neurons.

Re: GPT-3 has no idea what it’s talking about

#220

Post by chillacy »

visarga wrote: post #216
canjobear wrote: post #189

Earlier quoted context omitted.

It seems like Bender & Koller's argument can just as easily prove that humans don't have representations of meaning either. Our brains get input from multiple "languages"---conventional natural languages as well as the neural codes that represent our sensory inputs. All we have access to is the form of these inputs, and so we can never learn the underlying "meaning".

Meaning comes from the game, not from the brain. It's the relation between agent and environment that creates meaning.

The paper makes reference to the Symbol Grounding Problem, but I have not found the SGP's distinction between form and meaning to be completely convincing without some evidence of a physical, observable process.

At the end when you look long enough it seems to call into question the very nature of consciousness.

Post reply on HN