Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I disagree with Geoff Hinton regarding "glorified autocomplete"

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

241–250 of 279 posts

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#241

Post by edflsafoiewq »

jeffparsons wrote: post #220
user_named wrote: post #214

Earlier quoted context omitted.

Okay, so then tell me how does it decide whether it is true or false that Biden is the POTUS? It's response is not based on facts about the world as it exists, but on the text data it has been trained on. As such, it is not able to determine true or false even if the response in the above example would be correct.

Serious question, in pursuit of understanding where you're coming from: in what way do you think that your own reckoning is fundamentally different to or more "real" than what you're describing above? I know I don't experience the world as it is, but rather through a whole bunch of different signals I get that give me some hints about what the real world might be. For example, text.

In order to affirm something is true, you don't just need to know it, you need to know that you know it. LLMs fundamentally have no self-knowledge.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#243

Post by xcv123 »

edflsafoiewq wrote: post #241
jeffparsons wrote: post #220

Earlier quoted context omitted.

Serious question, in pursuit of understanding where you're coming from: in what way do you think that your own reckoning is fundamentally different to or more "real" than what you're describing above? I know I don't experience the world as it is, but rather through a whole bunch of different signals I get that give me some hints about what the real world might be. For example, text.

In order to affirm something is true, you don't just need to know it, you need to know that you know it. LLMs fundamentally have no self-knowledge.

> LLMs fundamentally have no self-knowledge

ChatGPT can tell me about itself when prompted. It tells me that it is an LLM. It can tell me about capabilities and limitations. It can describe the algorithms that generate itself. It has deep self knowledge, but is not conscious.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#244

Post by xcv123 »

Guvante wrote: post #228
jeffparsons wrote: post #220

Earlier quoted context omitted.

Serious question, in pursuit of understanding where you're coming from: in what way do you think that your own reckoning is fundamentally different to or more "real" than what you're describing above? I know I don't experience the world as it is, but rather through a whole bunch of different signals I get that give me some hints about what the real world might be. For example, text.

You understand the concept of true vs false. LLM does not, that isn't how it works. You can say the difference is academic but there is a difference. What is the difference between a real good faker of intelligence and actual intelligence is an open question. But I will say most AI experts agree that LLM are not artificial general intelligence. It isn't just a lack of training data, they just are not of the category…

> You understand the concept of true vs false.

> LLM does not, that isn't how it works.

GPT-4 can explain the concept when prompted and can evaluate logic problems better than most human beings can. I would say it has a deeper understanding of "true vs false" than most humans.

I think what you are trying to say is that LLMs are not conscious. Consciousness has no precise universally agreed formal definition, but we all know that LLMs are not conscious.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#245

Post by xcv123 »

astrange wrote: post #240
xcv123 wrote: post #185

Earlier quoted context omitted.

Oh well in that case the answer is straightforward. If you close your eyes and get lost after a few seconds, that's because that aspect of your model was not a 100% perfect exact replica of external reality that extended infinitely far in all spatial directions at all resolutions. For example, your internal spatial model is limited to some degree of accuracy and does not include the entire surface of Mars, but that d…

> For example, your internal spatial model is limited to some degree of accuracy and does not include the entire surface of Mars, but that doesn't mean that your model does not exist at all. You're using "your model" as a metaphorical term here, but if you came up with any precise definition of the term here, it'd turn out to be wrong; people have tried this since the 50s and never gotten it correct. (For instance, i…

So basically you agree with what I was saying.

> What principle can you use to decide how precise it should be?

It is not up to me or anyone else to decide. Our subjective definitions and concepts of the model are irrelevant. How the brain works is a result of our genetic structure. We don't have a choice.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#246

Post by srean »

fhe wrote: post #221
juhanima wrote: post #204

The only reason why output from a generative LLM appears intelligent or sentient is that it parrots a random sampling of texts written by intelligent and sentient people. In order to play the game of go effectively one needs to have a model or theory of how the game of go works. That's a very simple model that can be defined by a simple formula. That's why it is fairly easy for a neural network to learn how to play t…

here's my thought experiment: suppose one builds a generative model that predicts the next digit of pi. if a program can do this perfectly, then it's arguable that it understands what the number pi is. the question is, can such a model be trained by feeding it a large amount of known digits of pi? My intuition is that it's not doable with current approach to building generative models. the number pi arose out of cert…

I am (mostly) with you except for this bit...

> the number pi arose out of certain constraints and characteristics of the physical world we live in

Pi arose from the notion of a circle, which is an abstractions and axioms. Pi would still be pi in a completely different world under the same axioms and abstractions.

I qualified my statement with 'mostly' because a circular motion can indeed be defined by a differential equation, or in other words by a rule that dictates the 'next' value based on current value (and recent changes). So learning an approximation of a circle is very much in the realms of a sequence learner and it may learn about pi (and made to store the information to retrieve/recognize it later). However learning pi directly from the sequence of digits of pi, which is what you were talking about, that does seem difficult.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#247

Post by voitvodder »

wbillingsley wrote: post #188
robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

LLMs are very good at uncovering the mathematical relationships between words, many layers deep. Calling that understanding is a claim about what understanding is. But because we know how the LLMs we're talking about at the moment are trained, it seems to have more problems: LLMs do not directly model the world; they train on and model what people write about the world. It is an AI model of a computed gestalt human m…

We could anthropomorphize any textbook too and claim it has human level understanding of the subject. We could then claim the second edition of the textbook understands the subject better than the first. Anyone who claims the LLM "understands" is doing exactly this. What makes the LLM more absurd though is the LLM will actually tell you it doesn't understand anything while a book remains silent but people want to pretend we are living in the Matrix and the LLM is alive.

Most arguments then descend into confusing the human knowledge embedded in a textbook with the human agency to apply the embedded knowledge. Software that extracts the knowledge from all textbooks has nothing to do with the human agency to use that knowledge.

I love chatGPT4 and had signed up in the first few hours it was released but I actually canceled my subscription yesterday. Part because of the bullshit with the company these past few days but also because it had just become a waste of time the past few months for me. I learned so much this year but I hit a wall that to make any progress I need to read the textbooks on the subjects I am interested in just like I had to this time last year before chatGPT.

We also shouldn't forget that children anthropomorphize toys and dolls quite naturally. It is entirely natural to anthropomorphize a LLM and especially when it is designed to pretend it is typing back a response like a human would. It is not bullshitting you though when it pretends to type back a response about how it doesn't actually understand what it is writing.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#248

Post by user_named »

robbrown451 wrote: post #229
user_named wrote: post #191

Earlier quoted context omitted.

It is a very basic fact that LLMs have no concept of true or false, it only has an ability to look up what text data it has seen before. If you do not understand this you are in no position to discuss LLMs.

It certainly doesn't "look up" text data it has seen before. That shows a fundamental misunderstanding of how this stuff works. That's exactly why I use the example above of Alpha Zero and how it learns to play Go, since that demonstrates very clearly that it's not just looking things up. And I have no idea what you mean by saying that it has no concept of true or false. Even the simplest computer programs have a con…

Yes, you don't understand what I said. The model has no concept of true or false. It only has embeddings. If 'asked' a question it can see if that is consistent with its embeddings and probabilities or not. This is not a representation of the real world, of facts, but simply a product of its training.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#249

Post by user_named »

xcv123 wrote: post #223
user_named wrote: post #214

Earlier quoted context omitted.

Okay, so then tell me how does it decide whether it is true or false that Biden is the POTUS? It's response is not based on facts about the world as it exists, but on the text data it has been trained on. As such, it is not able to determine true or false even if the response in the above example would be correct.

> It's response is not based on facts about the world as it exists, but on the text data it has been trained on How did you find out that Biden was elected if not through language by reading or listening to news? Do you have extra sensory perception? Psychic powers? Do you magically perceive "facts" without any sensory input or communication? Ridiculous. By the same argument your knowledge is also not based on "facts…

You didn't answer my question ergo you concede that LLMs don't know true or false.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#250

Post by Someone »

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

> There may be some theoretical limit of a "perfect" Go player, or maybe not, but it will continue to converge towards perfection by continuing to train

I don’t think that’s a given. AlphaZero may have found an extremely high local optimum that isn’t the global optimum.

When playing only against itself, it won’t be able to get out of that local optimum, and when getting closer and closer to it even may ‘forget’ how to play against players that make moves that AplhaGo never would make, and that may be sufficient for a human to beat it (something like that happened with computer chess in the early years, where players would figure out which board positions computers were bad at, and try to get such positions on the board)

I think you have to keep letting it play against other good players (human or computer) that play differently to have it keep improving, and even then, there’s no guarantee it will find a global optimum.

Post reply on HN