Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-2 and the Nature of Intelligence

thegradient.pub

Post reply on HN
Print view View original

31–40 of 58 posts

Re: GPT-2 and the Nature of Intelligence

#31

Post by Rioghasarig »

fernly wrote: post #19
fernly wrote: post #16

I think he could have wrapped his paper up after showing this one example: > (input) I put two trophies on a table, and then add another, the total number is (GPT-2 continuation) five trophies and I'm like, 'Well, I can live with that, right? GPT-2 correctly inferred that the continuation should be a number of trophies, based on bazillions of similar sentences. But it had no understanding that arithmetic was called f…

Before anyone says that example (or any of the other fluent but _completely nonsensical_ continuations in the article) shows some sort of "understanding," please explain what you would define as understanding? I would (and I think anyone would) offer an operational definition: there is some class of questions to which this system could reply with sensible, actionable responses. Obviously the present system is not abl…

> The only question it appears to answer is, "given some words, what are other words that are likely to follow them in a typical blog post?" The fact that the words are syntactically correct is unimportant, when the fluent words convey no information relevant to the input.

You say that like that's a bad things. That's literally all it's been trained to do.

Re: GPT-2 and the Nature of Intelligence

#32

Post by drongoking »

Rioghasarig wrote: post #14
zanek wrote: post #2

I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning. It seems that OpenAi and others are peddling this AI when its simply a glorified Eliza on steroids.

I can't agree with you here. While GPT-2 isn't good at filling in particular details (like the language someone from Boston should speak) it is astonishingly good at recognizing the _kind_ of answer that should be produced. The fact that it usually answers with a language here is a reflection of what it understands. And it behaves similarly for a whole range of different tasks. If you write a sentence that should obv…

> it is astonishingly good at recognizing the _kind_ of answer that should be produced. The fact that it usually answers with a language here is a reflection of what it understands. And it behaves similarly for a whole range of different tasks. If you write a sentence that should obviously end with the name of a person it will give you the name of a person.

I think you're anthropomorphizing a fair amount. GPT-2 has memorized huge amounts of text and can, yes, generalize the characteristics of words that fill certain slots. To say that it recognizes the kind of answer that should be produced is implying agency where there is none.

The original author is right. GPT-2 has no idea what it's talking about. Play with it any amount of time and you'll realize this. It's more than "Eliza on steroids," but the impressiveness of GPT-2 comes from the style of its language, not the substance.

Re: GPT-2 and the Nature of Intelligence

#33

Post by doktorkrek »

fernly wrote: post #16

I think he could have wrapped his paper up after showing this one example: > (input) I put two trophies on a table, and then add another, the total number is (GPT-2 continuation) five trophies and I'm like, 'Well, I can live with that, right? GPT-2 correctly inferred that the continuation should be a number of trophies, based on bazillions of similar sentences. But it had no understanding that arithmetic was called f…

There are people with lessions affecting only very specific regions of their brain, who can proceed in regular conversations normally, but when asked numerical questions, they give seemingly nonsensical answers. Like when asked how much one cup and two cups are, they could very well say five. Yet you wouldn't infer they lack the understanding of the world.

Re: GPT-2 and the Nature of Intelligence

#34

Post by drongoking »

missosoup wrote: post #10
zanek wrote: post #2

I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning. It seems that OpenAi and others are peddling this AI when its simply a glorified Eliza on steroids.

> I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning. There's a pretty strong argument that most humans also frequently do this. My go-to example is high school physics. The majority of students merely learns to associate keywords in problem statements with a table of equations and a mapping of what numbers to substitute fo…

Arguing that physics students don't understand equations very well is a poor way to make a point about GPT-2. GPT-2 fails at a much more basic level, and that's Marcus's point.

Talk to a five year old for a while. The five-year-old's language may be crude but it shows basic concepts of a conversation, continuity, referents, basic causality, etc. GPT-2 has none of these. It regurgitates smooth language fragments because that's what it was trained on, but it exhibits no awareness that it's involved in a communication event with another being. Except possibly at cocktail parties, humans don't simply regurgitate words.

Re: GPT-2 and the Nature of Intelligence

#35

Post by AgentME »

jlebar wrote: post #30

I just came here to say that > Every person in the town of Springfield loves Susan. Peter lives in Springfield. Therefore he obviously has no love for that bitch. is an awesome completion. I would read that short story.

I think some of these examples are interesting because they show that GPT-2 was trained on data (web sites) that were optimized to be interesting, rather than lists of facts or logical inferences.

Hmm, now I wonder if you could take GPT-2, add on a little bit of training on some boring rote lists of logical inferences, and get something useful out of it.

Re: GPT-2 and the Nature of Intelligence

#36

Post by sandoooo »

I'm excited about future applications that strap this onto some relatively simple, logically consistent, non-AI number-crunching program. As a toy example, Scott Alexander trained it to output chess moves in a consistent manner that avoids nonsensical moves, but it can't win against competent human players. If you strap it onto a chessbot during both training and use I'm fairly sure it'll easily beat human grandmasters.

So what you have here is a human compatibility/abstraction layer for programs. What can you do with this strapped to Wolfram Alpha? Or trained with a Github dataset?

Put another way, apparently this does linguistic style /convincingly humanlike writing without being able to reason about cause/effect or basic arithmatics. But we already have programs that does cause/effect and arithmetics, quickly and at 100% accuracy. Now we just need to combine the two.

Re: GPT-2 and the Nature of Intelligence

#37

Post by Animats »

wilg wrote: post #4

I don't agree with the conclusion here. It's all about the input data. GPT-2 is trained on words people actually write on the internet, which is an inherently incomplete dataset. It leaves out all the other information an "intelligence" knows about the world. We know what sources are authoritative, we know the context of words from the visual appearance of the page, and we connect it all with data from our past exper…

GPT-2 is trained on words people actually write on the internet

It sure is. Go to the site [1] and paste in anything from an Internet rant. It does a really good job of autocompleting rants.

At last, high-quality artificial stupidity.

The other extreme is the MIT question-answering system [2]. Or Wolfram Alpha. Just the facts.

[1] https://talktotransformer.com/ [2] http://start.csail.mit.edu/index.php

Re: GPT-2 and the Nature of Intelligence

#38

Post by AgentME »

abrax3141 wrote: post #29
gog-ma-gog wrote: post #6

Earlier quoted context omitted.

To elaborate a bit: people like Marcus tend to overload/move the goal posts with what the word “understand” means. I kinda feel like in a world where we have perfectly conversational chat bots that are capable of AI complete tasks—-that if these bots look like Chinese rooms under the hood, he’ll still be complaining that they don’t “understand” anything. I don’t think it’s unreasonable to say that if you think someth…

Understanding is not hard to understand. To understand is to reason from a model. Reasoning from a model is easy. Discovering the correct model is hard, analogous to the way that algebraic rules are easy, but finding the right equation for a particular problem is hard. Data trained NNs have neither a model, nor do they reason. QED

You could say that a trained neural net contains a model of how language works, and it reasons about sentences based on this model.

I think people are really hung up on that it has trouble reasoning about what its sentences are reasoning about, and skipping how amazing it is at reasoning about sentence structure itself.

Re: GPT-2 and the Nature of Intelligence

#40

Post by laichzeit0 »

zanek wrote: post #2

I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning. It seems that OpenAi and others are peddling this AI when its simply a glorified Eliza on steroids.

Define the word “understanding” in a mathematically rigorous way. Otherwise it’s not clear exactly what you’re even saying in that sentence.
Post reply on HN