Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I disagree with Geoff Hinton regarding "glorified autocomplete"

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

261–270 of 279 posts

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#261

Post by mannykannot »

hackinthebochs wrote: post #140
mannykannot wrote: post #134

Earlier quoted context omitted.

I feel that LLMs raise some very interesting challenges for anyone trying to figure out what it means to understand something and how we do it, but I am not yet ready to agree with Hinton. For example, we are aware that some, but by no means all, of what people say is about an external world that may or may not conform to what the words say. We can also doubt that we have understood things correctly, and take steps t…

>I feel that LLMs raise some very interesting challenges for anyone trying to figure out what it means to understand something and how we do it, but I am not yet ready to agree with Hinton. Agreed. What LLMs say about understanding deserves a lot more attention than it has received. I wrote down some of my thoughts on the matter: https://www.reddit.com/r/naturalism/comments/1236vzf >Do LLMs do these things, or is wha…

I agree with a lot of what you say in the linked article, and I particularly agree that it is not helpful to define understanding in a way that would, a priori, make it a category error to propose that a suitably-programmed computer might understand things. I do, however, have a few words to say about the relationship between modeling and understanding. I can easily accept that an ability to model is necessary in order to understand something, but I feel the idea that it is sufficient would leave something out.

For example, meteorologists understand a lot about the weather in terms of the underlying physics, representing it as a special application of more general laws, but they are not very good at predicting it. Machine learning produces models which are much better predictors, but it does not seem to follow that they have a superior understanding of the weather.

One problem in assessing whether a token predictor has some sort of understanding is that if its training material is consistent with the supposition that, broadly speaking, it was produced by people who do have a reasonable understanding of what they were writing about, then it seems likely that the productions of a good predictor would unavoidably have that feature as well - but maybe that just is how most human understanding works? I am on the fence on this one.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#262

Post by weebull »

xcv123 wrote: post #244
Guvante wrote: post #228

Earlier quoted context omitted.

You understand the concept of true vs false. LLM does not, that isn't how it works. You can say the difference is academic but there is a difference. What is the difference between a real good faker of intelligence and actual intelligence is an open question. But I will say most AI experts agree that LLM are not artificial general intelligence. It isn't just a lack of training data, they just are not of the category…

> You understand the concept of true vs false. > LLM does not, that isn't how it works. GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans. I think what you are trying to say is that LLMs are not conscious. Consciousness has no precise universally agreed formal definition, but we a…

> GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans.

Sigh

GPT produces output which obeys the patterns it has been trained on for definitions of true and false. It does not understand anything. It is a token manipulation machine. It does it well enough that it convinces you, a walking ape, that it understands. It does not.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#263

Post by robbrown451 »

user_named wrote: post #248
robbrown451 wrote: post #229

Earlier quoted context omitted.

It certainly doesn't "look up" text data it has seen before. That shows a fundamental misunderstanding of how this stuff works. That's exactly why I use the example above of Alpha Zero and how it learns to play Go, since that demonstrates very clearly that it's not just looking things up. And I have no idea what you mean by saying that it has no concept of true or false. Even the simplest computer programs have a con…

Yes, you don't understand what I said. The model has no concept of true or false. It only has embeddings. If 'asked' a question it can see if that is consistent with its embeddings and probabilities or not. This is not a representation of the real world, of facts, but simply a product of its training.

"This is not a representation of the real world, of facts, but simply a product of its training."

Tell me how that doesn't apply to the human brain as well.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#264

Post by xcv123 »

weebull wrote: post #262
xcv123 wrote: post #244

Earlier quoted context omitted.

> You understand the concept of true vs false. > LLM does not, that isn't how it works. GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans. I think what you are trying to say is that LLMs are not conscious. Consciousness has no precise universally agreed formal definition, but we a…

> GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans. Sigh GPT produces output which obeys the patterns it has been trained on for definitions of true and false. It does not understand anything. It is a token manipulation machine. It does it well enough that it convinces you, a wal…

A human is an ape that is obeying patterns that it has been trained on. What is school but a bunch of apes being trained to obey patterns? Some of these apes do well enough to convince you that it understands things. Some apes fully "understand" that flat earth theory is true, or they "understand" that the Apollo moon landings were faked.

You have a subjective philosophical disagreement about what constitutes understanding. That is fine. I clearly understand it is not conscious and that programs do not understand things the way that humans do. We are fundamentally different to LLMs. That is obvious. But you are not making a technical argument here unless you can define "understand" in technical terms. This is a matter of semantics.

> It is a token manipulation machine

Deep learning and machine learning in general is more than token manipulation. They are designed for pattern recognition.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#265

Post by hackinthebochs »

mannykannot wrote: post #261
hackinthebochs wrote: post #140

Earlier quoted context omitted.

>I feel that LLMs raise some very interesting challenges for anyone trying to figure out what it means to understand something and how we do it, but I am not yet ready to agree with Hinton. Agreed. What LLMs say about understanding deserves a lot more attention than it has received. I wrote down some of my thoughts on the matter: https://www.reddit.com/r/naturalism/comments/1236vzf >Do LLMs do these things, or is wha…

I agree with a lot of what you say in the linked article, and I particularly agree that it is not helpful to define understanding in a way that would, a priori, make it a category error to propose that a suitably-programmed computer might understand things. I do, however, have a few words to say about the relationship between modeling and understanding. I can easily accept that an ability to model is necessary in ord…

>Machine learning produces models which are much better predictors, but it does not seem to follow that they have a superior understanding of the weather.

Fair points, and I agree. I don't recall if I made this point in the linked piece, but I think the extra function is a model embedded within some dynamic such that the capacity for modelling is in service to some goal. The goal can be simple like answering questions or something more elaborate. But the point is to engage the model as to influence the dynamic in a semantically rich way. The model itself doesn't represent understanding, but a process that understands will have a model that can be queried and manipulated in various ways corresponding to the process' goals.

>then it seems likely that the productions of a good predictor would unavoidably have that feature as well

Yeah, assessment is hard because of the sheer size of the training data. We can't be sure that some seemingly intelligent response isn't just recalling a similar query from training. One of the requirements for understanding is the counterfactual capacity, being able to report accurate information that is derivative of the training data but not explicitly in the training data. The Sparks of AGI paper, assuming it can be believed, demonstrates this capacity IMO. Particularly where GPT-4 draws a graph of a room after having been given navigation instructions. But its hard to make a determination in particular cases.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#266

Post by nighthawk454 »

theGnuMe wrote: post #252
nighthawk454 wrote: post #203

Earlier quoted context omitted.

I think the analogy is something like: if you have a simple distribution over all words, then that's just word frequency. Obviously not a good predictor. The 'information' necessary to predict the correct next word contextually is just not there if you're predicting words in a vacuum. In order to be practically useful and predict the right words _in context_, the model must be conditioning off of more of the sentence…

That's not information theoretic, that's just conditional probability.

it is conditional probability, but that is a fundamental concept used in information theory

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#267

Post by SkiFire13 »

xcv123 wrote: post #181
SkiFire13 wrote: post #121

Earlier quoted context omitted.

> One other thing to take into consideration, is that to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move. It doesn't necessarily have to look ahead. Since Go is a deterministic game there is always a best move (o…

> Since Go is a deterministic game there is always a best move The rules of the game are deterministic, but you may be going a step too far with that claim. Is the game deterministic when your opponent is non-deterministic? Is there an optimal move for any board state given that various opponents have varying strategies? What may be the best move against one opponent may not be the best move against another opponent.

Maybe "deterministic" is not the correct term here. What I meant is that there's no probability or unknown in the game, so you can always know what are the possible moves and the relative new state.

The opponent's moves may be considered non-deterministic, but you can just assume the worst case for you, that is the best case for the opponent, which is the opponent will always play the best move too.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#268

Post by SkiFire13 »

bytefactory wrote: post #138
SkiFire13 wrote: post #121

Earlier quoted context omitted.

> One other thing to take into consideration, is that to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move. It doesn't necessarily have to look ahead. Since Go is a deterministic game there is always a best move (o…

> It doesn't necessarily have to look ahead. Since Go is a deterministic game there is always a best move Is there really a difference between the two? If a certain move shapes the opponent's remaining possible moves into a smaller subset, hasn't AlphaGo "looked ahead"? In other words, when humans strategize and predict what happens in the real world, aren't they doing the same thing? I suppose you could argue that h…

> If a certain move shapes the opponent's remaining possible moves into a smaller subset, hasn't AlphaGo "looked ahead"?

You're confusing the reason why a move is good with how you can find that move. Yeah, a move is good due to how it shapes the opponent remaining moves, and this is also the reasoning we make in order to find that move, but it doesn't mean you can only find that move by doing that reasoning. You could have found that move just by randomly picking one, it's not very probably but it's possible. AIs just try to maximize such probability of picking a good move, meanwhile we try to find a reason a move is good. IMO it doesn't make sense to try to fit the way AI do this into our mental model, since the middle goal is fundamentally different.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#269

Post by Guvante »

xcv123 wrote: post #264
weebull wrote: post #262

Earlier quoted context omitted.

> GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans. Sigh GPT produces output which obeys the patterns it has been trained on for definitions of true and false. It does not understand anything. It is a token manipulation machine. It does it well enough that it convinces you, a wal…

A human is an ape that is obeying patterns that it has been trained on. What is school but a bunch of apes being trained to obey patterns? Some of these apes do well enough to convince you that it understands things. Some apes fully "understand" that flat earth theory is true, or they "understand" that the Apollo moon landings were faked. You have a subjective philosophical disagreement about what constitutes underst…

You acknowledged above that consciousness isn't what LLM is and you likely understand that the poster was referring to that...

The broad strokes you use here are exactly why discussing LLMs are hard. Sure some people dismiss them because it isn't general AI but having supporters dismiss any argument with "passes the Turning test" is equally useless.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#270

Post by xcv123 »

Guvante wrote: post #269
xcv123 wrote: post #264

Earlier quoted context omitted.

A human is an ape that is obeying patterns that it has been trained on. What is school but a bunch of apes being trained to obey patterns? Some of these apes do well enough to convince you that it understands things. Some apes fully "understand" that flat earth theory is true, or they "understand" that the Apollo moon landings were faked. You have a subjective philosophical disagreement about what constitutes underst…

You acknowledged above that consciousness isn't what LLM is and you likely understand that the poster was referring to that... The broad strokes you use here are exactly why discussing LLMs are hard. Sure some people dismiss them because it isn't general AI but having supporters dismiss any argument with "passes the Turning test" is equally useless.

No you have misunderstood. As I wrote above:

"But you are not making a technical argument here unless you can define "understand" in technical terms. This is a matter of semantics."

I said the nature of their argument is not technical, since they are not dealing with technical definitions, but I did not dismiss their argument altogether. I clarified and restated their own argument for them in clearer terms. LLMs are not conscious, but they can still "understand" very well depending on your definition of understand. Understanding is not a synonym for consciousness. Language is evolving and you need to be more precise when discussing AI / machine learning.

One definition of understand is:

"perceive the intended meaning of (words, a language, or a speaker)."

Deep learning models recognize patterns. Mechanical perception of patterns. They understand things mechanically, unconsciously.

Post reply on HN