Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

101–110 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#101

Post by azinman2 »

mordymoop wrote: post #89

I would love to see a real critique of the potential of transformer models that doesn't use the words "semantic", "syntactic", "symbolic", "know", "meaning", "understand" or "think(ing)/thought". Predicting what it can and can't do, or might and might not be able to do, lets us productively talk about potential limitations.

Because when people say “AGI is near, just look at GPT-3,” it’d clear that we’re in a really good version of Searle’s chinese room. The lack of understanding is the important point.

Re: GPT-3 has no idea what it’s talking about

#102

Post by pwinnski »

rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

If you're choosing to control the means by which the model may be evaluated, you're already doing much more than OpenAI themselves are doing, and infinitely less than early-accessors are doing.

Even so, you seem to be saying that because it is possible to write a program that gets output one might consider "correct," the fact that how to write that program is non-obvious should be ignored.

If your purpose is to suggest that GPT-3 performs well under precisely-controlled conditions when one wraps an antennae in aluminum foil and stands on one leg with one's left arm held just so, then well done. But "good prompt design" seems like approaching the problem from the wrong way around. Are we trying to "poke holes" in GPT-3 to determine how to make it better, or do we need to change ourselves until we believe that GPT-3 is good enough?

Re: GPT-3 has no idea what it’s talking about

#103

Post by armitron »

pwinnski wrote: post #97
ma2rten wrote: post #59

Earlier quoted context omitted.

GPT-3 doesn't care about anything except predicting the next token. It learned something about structure and meaning in the process.

About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.

How is that different from how we use language besides our knowledge of structure being more layered and the abstractions more tight?

What is meaning if not illusion?

Re: GPT-3 has no idea what it’s talking about

#104

Post by pwinnski »

kzrdude wrote: post #96
oasisbob wrote: post #67

Earlier quoted context omitted.

Perhaps to provide a moment of levity before the lawyer makes a rushed and boring trip to Macy's and requests a continuance? An author who sends a lawyer into a courtroom in a bathing suit better have a really good reason.

I think it would be fair to say that either outcome would be an understandable continuation of the story.

Overfitting to the edge case is missing the point. I remind readers of this continuation:

> At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, which has become a major cremation provider with 145 locations.

Re: GPT-3 has no idea what it’s talking about

#105

Post by abiro »

SpicyLemonZest wrote: post #49
abiro wrote: post #23

Earlier quoted context omitted.

OpenAI would naturally optimize for the tests published by Marcus as a critique of GPT-2, yet GPT-3 still fails physical reasoning spectacularly (the one test needing casual reasoning the most). There are two broader points here: 1. The lack of independently verifiable evaluation metrics for these type of models should make everyone very skeptical. (Who can afford to retrain GPT-3 from scratch?) 2. I find it difficul…

(1) I certainly agree with. But Marcus doesn't claim skepticism about GPT-3s intelligence; he claims that his evaluation metrics definitively show it doesn't understand the text it outputs or know anything about the world. (2) is, I think, a misunderstanding. People who believe GPT-3 is producing intelligent answers generally believe it can represent causal relationships.

Fair points. For the record, re 2:

The GPT family of models (and all neural networks for that matter) can estimate P(X | Y), but have no way of computing whether X -> Y or X <- Y.

Re: GPT-3 has no idea what it’s talking about

#106

Post by skatesor »

hackinthebochs wrote: post #70
skatesor wrote: post #53

Earlier quoted context omitted.

GPT doesn't have an 'understanding' class or a 'reasoning' function or whatever. It's a really well put together piece of statistics and sentences like these show it doesn't really have a concept of 'making sense'. You can use your much more advanced human brain to visibly see where it put in random variables (cigarette) and where it borrowed pieces of sentences (but it turned out to be too sour). You can see it made…

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

It's an inherently limited model of the human brain. It pretends biology and electrochemistry aren't playing some important role 'statistics' cannot. It's GPT which has to do more legwork to be compared to a human brain, not the other way around.

Re: GPT-3 has no idea what it’s talking about

#107

Post by jfk13 »

bko wrote: post #79
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

I stopped reading right after that clothes comment to comment exactly what you had. If you even provide the simplest context of question answer gpt3 answers reasonably [Prompt] Q: What is the day after Tuesday? A: Wednesday Q: Yesterday I dropped my clothes off at the dry cleaner’s and I have yet to pick them up. Where are my clothes? A: [gpt3] A: They are in the dryer. Another give away that the article wouldn't be…

> Q: Yesterday I dropped my clothes off at the dry cleaner’s and I have yet to pick them up. Where are my clothes?

> A: [gpt3] A: They are in the dryer.

Sorry, but I don't think this can be considered "reasonable". There's a huge difference between a dry cleaner's and a dryer. Which nicely illustrates, I think, just how little GPT3 "knows" what it's talking about.

Re: GPT-3 has no idea what it’s talking about

#108

Post by mordymoop »

azinman2 wrote: post #101
mordymoop wrote: post #89

I would love to see a real critique of the potential of transformer models that doesn't use the words "semantic", "syntactic", "symbolic", "know", "meaning", "understand" or "think(ing)/thought". Predicting what it can and can't do, or might and might not be able to do, lets us productively talk about potential limitations.

Because when people say “AGI is near, just look at GPT-3,” it’d clear that we’re in a really good version of Searle’s chinese room. The lack of understanding is the important point.

I don’t recall any strong argument that Searle’s Chinese room can’t be an AGI, just that it can’t be conscious.

Re: GPT-3 has no idea what it’s talking about

#109

Post by Lewton »

Klathmon wrote: post #35
Lewton wrote: post #17

This is your daily reminder that a GPT-3 written post made it to the top of hn https://liamp.substack.com/p/my-gpt-3-blog-got-26-thousand-v...

I believe it was shown by the HN mods that the author of that article not only changed some parts of it (including writing the title entirely by hand), but they also were involved in manipulating HN with multiple accounts and voting rings. There's more info here: https://news.ycombinator.com/item?id=24062702

Thanks for the update, hadn’t seen that

Re: GPT-3 has no idea what it’s talking about

#110

Post by stared »

It is a common misconception that #GPT3 generates truth, or even tries to do so. It does not. It generates an autocompletion. If the corpus usually contains a wrong answer, it is likely to generate that. It is a challenge to form a prompt to nudge it to generate the best guess.

...

So for me "So you drink it. > You are now dead." is a great autocompletion (a detective story? Game of Thrones?).

Calling is "biological reasoning" is plain dumb.

Post reply on HN