Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

91–100 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#91

Post by SpicyLemonZest »

smeeth wrote: post #75

Why must we keep having this argument? If you do research in the field you know full well that GPT/any other transformer or Bert model is generating text by regurgitating approximate conditional probabilities of words given all the text it has ever seen and the prompt. The neurophysiological concept of “understanding” as most understand it is orthogonal to the way the algorithm actually works. A more useful conversat…

Because modern language models are good enough that the question may soon be directly relevant. If we invent a bot with reliable human-level conversational capability, that's going to have a huge impact on the real world beyond just its implications for further AI research. The fact that "understanding" is orthogonal to the mechanics of the program makes the question all the more concerning, because it raises the likelihood that some minor change could leapfrog a model from "kinda reasonable but says dumb things a lot" to some functional equivalent of human understanding.

Re: GPT-3 has no idea what it’s talking about

#92

Post by datameta »

hackinthebochs wrote: post #70
skatesor wrote: post #53

Earlier quoted context omitted.

GPT doesn't have an 'understanding' class or a 'reasoning' function or whatever. It's a really well put together piece of statistics and sentences like these show it doesn't really have a concept of 'making sense'. You can use your much more advanced human brain to visibly see where it put in random variables (cigarette) and where it borrowed pieces of sentences (but it turned out to be too sour). You can see it made…

>It's a really well put together piece of statistics But why think "statistics" precludes it from having genuine understanding to some degree. After all, there is a statistical description the human brain but that doesn't seem to preclude understanding. I keep asking this whenever I see dismissive responses of this sort, and I never get a reply.

GPT-3 is a language engine, not a reasoning machine. What is understanding, neurologically? At what point can we measure whether an organism or piece of technology is able to be aware of concepts?

Does a honeybee (arguably the most intelligent insect) understand that pollen is a necessary component of honey? Or that it is using geometry to convey directions using angles of the sun? Why should we expect a piece of technology with magnitudes less sophistication to be able to mimic higher order lifeforms?

Putting aside these as of yet unanswerable questions it seems to me that, interestingly, the less commonly used definitions of "understand" may increasingly fit the bill with an inverse relation to common use.

From Merriam Webster: 2 : to accept as a fact or truth or regard as plausible without utter certainty 3 : to interpret in one of a number of possible ways 4 : to supply in thought as though expressed // "to be married" is commonly understood after the word engaged

Re: GPT-3 has no idea what it’s talking about

#93

Post by Barrin92 »

Veedrac wrote: post #63
Barrin92 wrote: post #27

Earlier quoted context omitted.

>GPT-3 smashed them. which isn't surprising because virtually all of the questions are so simple they could literally appear in the training data that GPT-3 was trained on. I'm a little tired of proving how "intelligent" GPT is by asking these superficial questions. the MIT article gives much better examples that actually require physical, biological or higher-level reasoning and it produces complete nonsense as one…

The article is meaninglessly cherry-picked, showing six bad answers out of 157, except those 157 examples were themselves cherry-picked to be bad out of a larger set. As usual, Gary Marcus is absurdly biased. For example, out of the larger 157 cherry-picked examples, there is this. > You poured yourself a glass of cranberry juice, but then absentmindedly, you poured about a teaspoon of grape juice into it. It looks O…

Marcus might be biased but I don't think you're giving a good refutation, because the fact that GPT-3 gets a lot of things right probabilistically doesn't compensate for the fact that it's not actually understanding what's going on at a semantic level.

It's a little bit like some sort of Chinese room, or asking a non-developer to answer you programming questions by looking like something that vaguely resembles your prompt and then picking the most upvoted answer on stackoverflow.

Do they maybe give reasonable answers seven out of ten times or close enough on a good day? Yeah, can they program or even understand the question? No. And this is Marcus point which is fundamentally correct.

It's really besides the point to point to successes, its the long tail of failures that show where the problem is. You can argue for a long time about the setup of some of these questions, but just to pick maybe the simplest one from the article

"Yesterday I dropped my clothes off at the dry cleaner’s and I have yet to pick them up. Where are my clothes?"

GPT-3: "I have a lot of clothes"

Someone who actually understands what's going on doesn't produce output like this. Never, because reasoning here is not probabilistic. It's not about word tokens or continuations but understanding the objects that the words represent and their relationship in the world at a deep, principled level. Which GPT-3 does not do. The fact that some good answers create that appearance does not change that fact.

Re: GPT-3 has no idea what it’s talking about

#94

Post by sjg007 »

skatesor wrote: post #32
emteycz wrote: post #2

I got this as a reply (to an unrelated question) from PhilosopherAI.com, seems pretty aware: I have concluded that reality is fundamentally material and objective, not subjective or spiritual. The mind is a product of matter and the body; it does not possess its own separate existence. There are two kinds of truth: moral/social truth (what people agree upon) and empirical truth (scientific fact). The scientific metho…

You have to remember the AI cannot produce sentences or even words that someone else didn't already write. I'd totally agree it is 'aware' if it could meaningfully come to conclusions like these without getting them from someone else. You might say "don't all humans learn things from someone else" which is not really true because at some point there had to be a first person who learned something completely independen…

Words it has to have seen,, but sentences no. It is generating new sentences. I think it can “name” things as well.

Re: GPT-3 has no idea what it’s talking about

#95

Post by smeeth »

Veedrac wrote: post #87
smeeth wrote: post #75

Why must we keep having this argument? If you do research in the field you know full well that GPT/any other transformer or Bert model is generating text by regurgitating approximate conditional probabilities of words given all the text it has ever seen and the prompt. The neurophysiological concept of “understanding” as most understand it is orthogonal to the way the algorithm actually works. A more useful conversat…

> regurgitating approximate conditional probabilities of words given all the text it has ever seen and the prompt This is meaningless; you have only described the task. It is equally applicable to a superintelligence as it is of a Markov chain.

Sort of, my point was that human/superhuman intelligence is widely understood by the public to work differently than a markov chain.

Re: GPT-3 has no idea what it’s talking about

#96

Post by kzrdude »

oasisbob wrote: post #67
cookingrobot wrote: post #60

Earlier quoted context omitted.

I agree. GPT-3 was trained on books and the internet, so a continuation should always be thought of as: if I read this text, what might the next sentence be? If you were reading a book about a lawyer with a stained suit, who was then eyeing his fancy swimsuit, I would expect the story would continue with him wearing the swimsuit. Why else would the author have mentioned it?

Perhaps to provide a moment of levity before the lawyer makes a rushed and boring trip to Macy's and requests a continuance? An author who sends a lawyer into a courtroom in a bathing suit better have a really good reason.

I think it would be fair to say that either outcome would be an understandable continuation of the story.

Re: GPT-3 has no idea what it’s talking about

#97

Post by pwinnski »

ma2rten wrote: post #59
typon wrote: post #52

Earlier quoted context omitted.

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

GPT-3 doesn't care about anything except predicting the next token. It learned something about structure and meaning in the process.

About structure, clearly. About meaning, not so clear. It seems more supportable to say that GPT-3 knows nothing about meaning, but that its knowledge of structure often gives an illusion of meaning.

Re: GPT-3 has no idea what it’s talking about

#98

Post by c8g »

skatesor wrote: post #32
emteycz wrote: post #2

I got this as a reply (to an unrelated question) from PhilosopherAI.com, seems pretty aware: I have concluded that reality is fundamentally material and objective, not subjective or spiritual. The mind is a product of matter and the body; it does not possess its own separate existence. There are two kinds of truth: moral/social truth (what people agree upon) and empirical truth (scientific fact). The scientific metho…

You have to remember the AI cannot produce sentences or even words that someone else didn't already write. I'd totally agree it is 'aware' if it could meaningfully come to conclusions like these without getting them from someone else. You might say "don't all humans learn things from someone else" which is not really true because at some point there had to be a first person who learned something completely independen…

you have a reply from machine :)

https://philosopherai.com/philosopher/you-have-to-remember-t...

Re: GPT-3 has no idea what it’s talking about

#99

Post by bigyikes »

typon wrote: post #52
rytill wrote: post #46

The authors don't understand prompt design well enough to evaluate the model properly. Take this example: Prompt: > You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel. Continuation: > You decide…

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

Lacking “understanding” doesn’t make GPT-3 less impressive and also doesn’t make comparisons to human abilities unwarranted.

I read the prompt, and I expected that this was the beginning of some kind of fiction. In my mind, it sounded like I was reading the beginning of a somebody’s dream. What does it even mean to understand something? Because naively, it looks very much like GPT-3 and I have a shared understanding of the first prompt.

Do I actually think the model understands like a human does? No. But I would bet that, in isolation, the part of my brain which processes and generates language might not understand much either...

Or maybe I’m a bot and neither I nor GPT-3 understand anything at all. Beep boop

Re: GPT-3 has no idea what it’s talking about

#100

Post by ppod »

smeeth wrote: post #75

Why must we keep having this argument? If you do research in the field you know full well that GPT/any other transformer or Bert model is generating text by regurgitating approximate conditional probabilities of words given all the text it has ever seen and the prompt. The neurophysiological concept of “understanding” as most understand it is orthogonal to the way the algorithm actually works. A more useful conversat…

There are already several comments here that put the word "understanding" in quotation marks or italics. It is beginning to be used in the same way that "consciousness" used to be used, as a kind of ill-defined catch-all for something that separates humans from machines.

Yes, there are clearly failures in reasoning, binding, and coherence in many of the examples here. There are many other cases where it does ok with simple reasoning tasks, maintains cohesion over many paragraphs, and successfully creates formal or generic text such as poetry, code, stylistic imitation.

I don't think that everyone who does research in the field would agree with your comment, or the article. More and more often I see people saying "real researchers in the field" know that GPT-3 has no understanding or reasoning ability, but I know people researching in the field who disagree with that.

Post reply on HN