Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

21–30 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#21

Post by perl4ever »

kingkawn wrote: post #15
lacker wrote: post #10

This is basically true, but I think they underrate the improvements between GPT-2 and GPT-3. My mental model is, every once in a while these systems degenerate into surreal non sequitur nonsense. GPT-3 just does it a lot less than GPT-2. It still isn’t good enough to consistently answer casual questions in a human way, but the failure rate is going down, and perhaps straightforward improvements like GPT-4 will be abl…

“every once in a while these systems degenerate into surreal non sequitur nonsense.” Exactly as our minds do

>Exactly as our minds do

This rhetorically obscures the fact that when humans do produce similar stuff, it's a recognized sort of pathology that is obviously distinct from normal functioning.

https://en.wikipedia.org/wiki/Derailment_(thought_disorder)

Example: "I think someone's infiltrated my copies of the cases. We've got to case the joint. I don't believe in joints, but they do hold your body together."

https://en.wikipedia.org/wiki/Word_salad

Whatever the difference between this and normal language, call it "X", and whether or not it's amenable to implementing in software in principle, GPT-3 clearly does not have "X" at all.

Maybe it would be fruitful to fund study of mental/neurological disorders more, just to understand the mind better.

Re: GPT-3 has no idea what it’s talking about

#22

Post by mordymoop »

SpicyLemonZest wrote: post #3

I don't get it. Their methodology says > These experiments are not, by any means, either a representative or a systematic sample of anything. We designed them explicitly to be difficult for current natural language processing technology. Moreover, we pre-tested them on the "AI Dungeon" game which is powered by some version of GPT-3, and we excluded those for which "AI Dungeon" gave reasonable answers. (We did not kee…

I’ve repeatedly seen the following sequence, from acquaintances and randos on twitter:

1) GPT-3 is actually really dumb. The people posting impressive results are either cherry picking top 1% best results or just lying.

2) (time passes, they figure it out) Okay, I get it, you have to prompt it right, then it really does work well. But that’s so burdensome!

3) (rewriting history) Of course you have to prompt it right, how else is it supposed to know exactly what sort of task I expect it to be doing?

—

If, for example, you see GPT-3 say something extremely stupid rather than just a bit off, you’ve almost certainly primed it to think it’s supposed to be saying stupid, random things, by giving it a nonsense prompt.

Re: GPT-3 has no idea what it’s talking about

#23

Post by abiro »

andyljones wrote: post #9

Gary Marcus - the author of this - has previously offered several concrete tests that he felt demonstrated the limitations of the GPT approach. GPT-3 smashed them. https://www.gwern.net/GPT-3#marcus-2020

OpenAI would naturally optimize for the tests published by Marcus as a critique of GPT-2, yet GPT-3 still fails physical reasoning spectacularly (the one test needing casual reasoning the most).

There are two broader points here:

1. The lack of independently verifiable evaluation metrics for these type of models should make everyone very skeptical. (Who can afford to retrain GPT-3 from scratch?)

2. I find it difficult to believe that smart people still insist that a model incapable of representing causal relationships can produce intelligent answers.

Re: GPT-3 has no idea what it’s talking about

#25

Post by shadowmore »

colesantiago wrote: post #6

I thought it was well known that GPT-3 is pretty good at producing incoherent bullshit. No surprise here. Take this for example: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Crem…

Sounds like the way dreams are usually structured. Seemingly coherent overarching structure with completely arbitrary zig-zags at the micro level.

Re: GPT-3 has no idea what it’s talking about

#26

Post by ohgodplsno »

colesantiago wrote: post #6

I thought it was well known that GPT-3 is pretty good at producing incoherent bullshit. No surprise here. Take this for example: > At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Crem…

I mean, switch a few words for more tech related ones, and you've got the pitch of quite a few YCombinator candidates.

Re: GPT-3 has no idea what it’s talking about

#27

Post by Barrin92 »

andyljones wrote: post #9

Gary Marcus - the author of this - has previously offered several concrete tests that he felt demonstrated the limitations of the GPT approach. GPT-3 smashed them. https://www.gwern.net/GPT-3#marcus-2020

>GPT-3 smashed them.

which isn't surprising because virtually all of the questions are so simple they could literally appear in the training data that GPT-3 was trained on. I'm a little tired of proving how "intelligent" GPT is by asking these superficial questions.

the MIT article gives much better examples that actually require physical, biological or higher-level reasoning and it produces complete nonsense as one would expect.

Re: GPT-3 has no idea what it’s talking about

#28

Post by emteycz »

_y5hn wrote: post #16
emteycz wrote: post #2

I got this as a reply (to an unrelated question) from PhilosopherAI.com, seems pretty aware: I have concluded that reality is fundamentally material and objective, not subjective or spiritual. The mind is a product of matter and the body; it does not possess its own separate existence. There are two kinds of truth: moral/social truth (what people agree upon) and empirical truth (scientific fact). The scientific metho…

How can something non-conscious be aware?

This text shows high awareness - it simply stated what it is, what its goal is, how the goal came to be - and even that it is not conscious, and that from its (a single ran program) point of view time or space does not exist.

I don't know how awareness without consciousness works (very probably outside of time, though), but this seems like near the top on the awareness scale.

Post reply on HN