Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I disagree with Geoff Hinton regarding "glorified autocomplete"

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

221–230 of 279 posts

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#221

Post by fhe »

juhanima wrote: post #204

The only reason why output from a generative LLM appears intelligent or sentient is that it parrots a random sampling of texts written by intelligent and sentient people. In order to play the game of go effectively one needs to have a model or theory of how the game of go works. That's a very simple model that can be defined by a simple formula. That's why it is fairly easy for a neural network to learn how to play t…

here's my thought experiment: suppose one builds a generative model that predicts the next digit of pi. if a program can do this perfectly, then it's arguable that it understands what the number pi is. the question is, can such a model be trained by feeding it a large amount of known digits of pi?

My intuition is that it's not doable with current approach to building generative models. the number pi arose out of certain constraints and characteristics of the physical world we live in. but if a model ever sees is just an endless stream of digits, without access to the underlying physical model, I don't see a path for it to 'reverse-engineer' and figure out the physical model that gave rise to it.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#222

Post by ChatGTP »

user_named wrote: post #214
xcv123 wrote: post #205

Earlier quoted context omitted.

One of the most ridiculous comments I have read about LLMs here. The ~100 layer deep neural networks infer many levels of features over the text, including the concept of true and false. That is trivial for an LLM. Are you completely unaware these are based on deep neural networks? Convolutional Neural Networks don't operate by "look up" of text data.

Okay, so then tell me how does it decide whether it is true or false that Biden is the POTUS? It's response is not based on facts about the world as it exists, but on the text data it has been trained on. As such, it is not able to determine true or false even if the response in the above example would be correct.

It’s giving the most likely answer as opposed to the factual answer?

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#223

Post by xcv123 »

user_named wrote: post #214
xcv123 wrote: post #205

Earlier quoted context omitted.

One of the most ridiculous comments I have read about LLMs here. The ~100 layer deep neural networks infer many levels of features over the text, including the concept of true and false. That is trivial for an LLM. Are you completely unaware these are based on deep neural networks? Convolutional Neural Networks don't operate by "look up" of text data.

Okay, so then tell me how does it decide whether it is true or false that Biden is the POTUS? It's response is not based on facts about the world as it exists, but on the text data it has been trained on. As such, it is not able to determine true or false even if the response in the above example would be correct.

> It's response is not based on facts about the world as it exists, but on the text data it has been trained on

How did you find out that Biden was elected if not through language by reading or listening to news? Do you have extra sensory perception? Psychic powers? Do you magically perceive "facts" without any sensory input or communication? Ridiculous.

By the same argument your knowledge is also not based on "facts" about the world, since you only learned about it by reading or listening. Absurd nonsense.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#224

Post by MVissers »

juhanima wrote: post #217
saurik wrote: post #209

Earlier quoted context omitted.

I don't think the question of whether an LLM that keeps getting restarted and seems to not remember things is conscious due to that lack is fair, as it feels more like suddenly making three duplicate copies of me or actively attempting to delete my memory of something... which, btw, I might not have stored in the first place: if someone has interograde amnesia, are they inherently not sentient? Even Sydney (the name…

I would argue that total anterograde amnesia would be a serious challenge for sentience, yes.

So when your drunk and you forget your actions the day afterwards you don't consider yourself to have been sentient/conscious?

That's not how we define conscious anywhere.

You can process the world around you, feel and introspect. Even if your judgment is off and you forget your actions, you're conscious in that moment.

From a neuroscience perspective, what you're suggesting is absolutely false btw.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#225

Post by juhanima »

MVissers wrote: post #224
juhanima wrote: post #217

Earlier quoted context omitted.

I would argue that total anterograde amnesia would be a serious challenge for sentience, yes.

So when your drunk and you forget your actions the day afterwards you don't consider yourself to have been sentient/conscious? That's not how we define conscious anywhere. You can process the world around you, feel and introspect. Even if your judgment is off and you forget your actions, you're conscious in that moment. From a neuroscience perspective, what you're suggesting is absolutely false btw.

I suppose there is a concept of sentience from outside and a different concept from internal sentience. The movie "Johny Got His Gun" by Dalton Trumbo discusses a situation where a badly injured soldier in WW1 is considered brain dead by outsiders while he's fully conscious and sentient internally.

I haven't studied neuroscience so I don't know how you define consciousness. I have read Julian Jaynes's "The Origin of Consciousness..." which in my untrained opinion makes a compelling case that consciousness is a hard term to define.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#226

Post by MVissers »

stubybubs wrote: post #125
continuational wrote: post #80

Earlier quoted context omitted.

What is the test for this? I taught it Firefly, which is an undocumented programming language I'm working on, through conversion. I find it's a lot quicker than any human at picking up syntax and semantics, both in real time and in number of messages, and makes pretty good attempts at writing code in it, as much as you could expect from a human programmer. That is, until you run out of context - is this what you mean…

I gave it the three lightbulbs in a closet riddle. https://puzzles.nigelcoldwell.co.uk/seven.htm The key complication is "once you've opened the door, you may no longer touch a switch." It gets this. There are many examples of it written out on the web. When I give it a variation and say "you can open the door to look at the bulbs and use the switches all you want" and it is absolutely unable to understand this. To a…

GPT-4 on platform.openai.com says this on the first try:

Switch on the first switch and leave it on for a few minutes. Then, switch it off and switch on the second switch. Leave the third switch off. Now, walk into the room.

The bulb that is on corresponds to the second switch. The bulb that is off and still warm corresponds to the first switch because it had time to heat up. The bulb that is off and cool corresponds to the third switch, the one you never turned on.

GPT-4-0314: 1. Turn on the first switch and leave it on for about 5 minutes. 2. After 5 minutes, turn off the first switch and turn on the second switch. 3. Open the door and enter the room.

Now observe the lights: - The bulb that is on is connected to the second switch (which is currently on). - The bulb that is off but warm to the touch is connected to the first switch (it was on long enough to heat up the bulb). - The bulb that is off and cool to the touch is connected to the third switch (it was never turned on).

----

But– It's also trained on the internet. GPT-4 paper 'sparks of AGI' had a logical puzzle it most likely never encountered in the training data that it could solve.

Also– I encourage you to ask these types of logical puzzles on the street to rando's. They're not easy to solve.

My question to you would be: What would convince you that it actually can 'think' logically?

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#227

Post by icy_deadposts »

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

> there isn't really a limit to how good it can get.

> it will continue to converge towards perfection

Then someone discovered a flaw that made it repeatably beatable by relative amateurs in a way that no human player would be

https://www.vice.com/en/article/v7v5xb/a-human-amateur-beat-...

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#228

Post by Guvante »

jeffparsons wrote: post #220
user_named wrote: post #214

Earlier quoted context omitted.

Okay, so then tell me how does it decide whether it is true or false that Biden is the POTUS? It's response is not based on facts about the world as it exists, but on the text data it has been trained on. As such, it is not able to determine true or false even if the response in the above example would be correct.

Serious question, in pursuit of understanding where you're coming from: in what way do you think that your own reckoning is fundamentally different to or more "real" than what you're describing above? I know I don't experience the world as it is, but rather through a whole bunch of different signals I get that give me some hints about what the real world might be. For example, text.

You understand the concept of true vs false.

LLM does not, that isn't how it works.

You can say the difference is academic but there is a difference.

What is the difference between a real good faker of intelligence and actual intelligence is an open question.

But I will say most AI experts agree that LLM are not artificial general intelligence. It isn't just a lack of training data, they just are not of the category that we mean by that.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#229

Post by robbrown451 »

user_named wrote: post #191
famouswaffles wrote: post #133

Earlier quoted context omitted.

>I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model and the world disagree, the world wins. Yeah this isn't really true. There's not how humans work. For a variety of reasons, Plenty stick with their incorrect model despite the world indicating otherwise. In fact, this seems to be normal enough human behaviour. Everyone does it, for somet…

It is a very basic fact that LLMs have no concept of true or false, it only has an ability to look up what text data it has seen before. If you do not understand this you are in no position to discuss LLMs.

It certainly doesn't "look up" text data it has seen before. That shows a fundamental misunderstanding of how this stuff works. That's exactly why I use the example above of Alpha Zero and how it learns to play Go, since that demonstrates very clearly that it's not just looking things up.

And I have no idea what you mean by saying that it has no concept of true or false. Even the simplest computer programs have a concept of true or false, that's kind of the simplest data type, a boolean. Large language models have a much more sophisticated concept of true and false that has a lot more nuance. That's really a pretty ridiculous thing to say.

Post reply on HN