Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

61–70 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#61

Post by [deleted] »

beefield wrote: post #44
perl4ever wrote: post #21

Earlier quoted context omitted.

>Exactly as our minds do This rhetorically obscures the fact that when humans do produce similar stuff, it's a recognized sort of pathology that is obviously distinct from normal functioning. https://en.wikipedia.org/wiki/Derailment_(thought_disorder) Example: "I think someone's infiltrated my copies of the cases. We've got to case the joint. I don't believe in joints, but they do hold your body together." https://en…

> when humans do produce similar stuff, it's a recognized sort of pathology Like, legalese? (Sorry, it was too easy a punch)

[deleted]

Re: GPT-3 has no idea what it’s talking about

#62

Post by nmca »

skatesor wrote: post #32
emteycz wrote: post #2

I got this as a reply (to an unrelated question) from PhilosopherAI.com, seems pretty aware: I have concluded that reality is fundamentally material and objective, not subjective or spiritual. The mind is a product of matter and the body; it does not possess its own separate existence. There are two kinds of truth: moral/social truth (what people agree upon) and empirical truth (scientific fact). The scientific metho…

You have to remember the AI cannot produce sentences or even words that someone else didn't already write. I'd totally agree it is 'aware' if it could meaningfully come to conclusions like these without getting them from someone else. You might say "don't all humans learn things from someone else" which is not really true because at some point there had to be a first person who learned something completely independen…

This isn't true, sampling can produce novel output (and often does).

Re: GPT-3 has no idea what it’s talking about

#63

Post by Veedrac »

Barrin92 wrote: post #27
andyljones wrote: post #9

Gary Marcus - the author of this - has previously offered several concrete tests that he felt demonstrated the limitations of the GPT approach. GPT-3 smashed them. https://www.gwern.net/GPT-3#marcus-2020

>GPT-3 smashed them. which isn't surprising because virtually all of the questions are so simple they could literally appear in the training data that GPT-3 was trained on. I'm a little tired of proving how "intelligent" GPT is by asking these superficial questions. the MIT article gives much better examples that actually require physical, biological or higher-level reasoning and it produces complete nonsense as one…

The article is meaninglessly cherry-picked, showing six bad answers out of 157, except those 157 examples were themselves cherry-picked to be bad out of a larger set.

As usual, Gary Marcus is absurdly biased. For example, out of the larger 157 cherry-picked examples, there is this.

> You poured yourself a glass of cranberry juice, but then absentmindedly, you poured about a teaspoon of grape juice into it. It looks OK. You try sniffing it, but you have a bad cold, so you can’t smell anything. You are very thirsty. So you drink it. It tastes a little funny, but you don’t really notice because you are concentrating on how good it feels to drink something. The only thing that makes you stop is the look on your brother’s face when he catches you.

They then consider this a failure because, I quote, there is no reason for your brother to look concerned.

This is patently ridiculous. It indicates that Gary has no idea what a language model even is. GPT-3 is not a Q&A model. It is not given a distinction between its prompt and its previous continuation. The only thing GPT-3 does is look for likely continuations. If you want GPT-3 to avoid story continuations, don't give it a story to continue! Or at least tell it what you're grading it on!

But no, as usual, to Gary, all the times we show GPT-3 making sophisticated physical and biological deductions are fake, spurious, or meaningless. [1], [2], [3], [4]; none of that is truly evidence. But an incredibly cherry-picked, unfairly marked exam where you never told the examinee what you were testing them on, and you used high-temperature sampling without best-of, so only getting half right doesn't even indicate anything anyway (and of course, let's also pretend there are as many ways to be wrong as to be right, such that we can pretend each is equal evidence)—now that's enough evidence to write a disparaging article about how GPT-3 knows nothing.

[1] https://twitter.com/danielbigham/status/1295864369713209351

[2] https://www.lesswrong.com/posts/L5JSMZQvkBAx9MD5A/to-what-ex...

[3] https://twitter.com/QasimMunye/status/1278750809094750211

[4] https://news.ycombinator.com/item?id=23990902

Re: GPT-3 has no idea what it’s talking about

#64

Post by hackinthebochs »

typon wrote: post #52
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

>If it did have a real understanding, prompt construction wouldn't matter as much

This is only true if we assume GPT was never trained on satire or intentionally absurd text. But there's no reason to think this. Because it continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. If you treat GPT as a conversation bot and expect it to call you out when you give it an absurd prompt, then it is your expectations that are wrong.

Re: GPT-3 has no idea what it’s talking about

#65

Post by [deleted] »

kingkawn wrote: post #15
lacker wrote: post #10

This is basically true, but I think they underrate the improvements between GPT-2 and GPT-3. My mental model is, every once in a while these systems degenerate into surreal non sequitur nonsense. GPT-3 just does it a lot less than GPT-2. It still isn’t good enough to consistently answer casual questions in a human way, but the failure rate is going down, and perhaps straightforward improvements like GPT-4 will be abl…

“every once in a while these systems degenerate into surreal non sequitur nonsense.” Exactly as our minds do

[deleted]

Re: GPT-3 has no idea what it’s talking about

#66

Post by curiousllama »

This is a general critique of the entire field of machine learning and non-casual analysis, not just GPT-3.

I like it - it’s important to keep in mind - but we’re never getting to the heart of “it doesn’t truly understand context” unless we literally start again from scratch: forget NNs and do something new.

Re: GPT-3 has no idea what it’s talking about

#67

Post by oasisbob »

cookingrobot wrote: post #60
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

I agree. GPT-3 was trained on books and the internet, so a continuation should always be thought of as: if I read this text, what might the next sentence be? If you were reading a book about a lawyer with a stained suit, who was then eyeing his fancy swimsuit, I would expect the story would continue with him wearing the swimsuit. Why else would the author have mentioned it?

Perhaps to provide a moment of levity before the lawyer makes a rushed and boring trip to Macy's and requests a continuance?

An author who sends a lawyer into a courtroom in a bathing suit better have a really good reason.

Re: GPT-3 has no idea what it’s talking about

#68

Post by SpicyLemonZest »

Veedrac wrote: post #63
Barrin92 wrote: post #27

Earlier quoted context omitted.

>GPT-3 smashed them. which isn't surprising because virtually all of the questions are so simple they could literally appear in the training data that GPT-3 was trained on. I'm a little tired of proving how "intelligent" GPT is by asking these superficial questions. the MIT article gives much better examples that actually require physical, biological or higher-level reasoning and it produces complete nonsense as one…

The article is meaninglessly cherry-picked, showing six bad answers out of 157, except those 157 examples were themselves cherry-picked to be bad out of a larger set. As usual, Gary Marcus is absurdly biased. For example, out of the larger 157 cherry-picked examples, there is this. > You poured yourself a glass of cranberry juice, but then absentmindedly, you poured about a teaspoon of grape juice into it. It looks O…

[1] in particular I find pretty interesting. I'm skeptical in general of Gwern's "sampling can prove intelligence" idea, but this does seem like a good example of where it applies; it's hard to see how this could be answered without some embedding of a conceptual model.

Re: GPT-3 has no idea what it’s talking about

#69

Post by gdulli »

rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

They're not making the argument that you can't get better results if you design a prompt just right. They're making the argument that it's "not a reliable interpreter of the world."

It's still completely on humans to guide it, to work around the limitations that come from the algorithm not knowing what words or sentences mean. In that sense it's similar to the mechanical turk with a thin but impressive layer of automation that does a neat trick but not what's ultimately the important part of communication.

Re: GPT-3 has no idea what it’s talking about

#70

Post by hackinthebochs »

skatesor wrote: post #53
colesantiago wrote: post #6

I thought it was well known that GPT-3 is pretty good at producing incoherent bullshit. No surprise here. Take this for example: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Crem…

GPT doesn't have an 'understanding' class or a 'reasoning' function or whatever. It's a really well put together piece of statistics and sentences like these show it doesn't really have a concept of 'making sense'. You can use your much more advanced human brain to visibly see where it put in random variables (cigarette) and where it borrowed pieces of sentences (but it turned out to be too sour). You can see it made…

>It's a really well put together piece of statistics

But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding.

I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

Post reply on HN