Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I disagree with Geoff Hinton regarding "glorified autocomplete"

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

121–130 of 279 posts

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#121

Post by SkiFire13 »

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

> One other thing to take into consideration, is that to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move.

It doesn't necessarily have to look ahead. Since Go is a deterministic game there is always a best move (or moves that are better than others) and hence a function that goes from the state of the game to the best move. We just don't have a way to compute this function, but it exists. And that function doesn't need the concept of lookahead, that's just an intuitive way of how could find some of its values. Likewise ML algorithms don't necessarily need lookahead, they can just try to approximate that function with enough precision by exploiting patterns in it. And that's why we can still craft puzzles that some AIs can't solve but humans can, by exploiting edge cases in that function that the ML algorithm didn't notice but are solvable with understanding of the game.

The thing is though, does this really matter if eventually we won't be able to notice the difference?

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#122

Post by famouswaffles »

lsy wrote: post #116
robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

The difference is that "the world" is not exhaustible in the same way as Go is. While it's surely true that the number of possible overall Go game states is extremely large, the game itself is trivially representable as a set of legal moves and rules. The "world model" of the Go board is actually just already exhaustive and finite, and the computer's work in playing against itself is to generate more varied data with…

>The sum of all human artifacts ever made (or yet to be made) doesn't exhaust the description of a rock in your front yard, let alone the world in all its varied possibility.

No human or creature we know of has a "true" world model so this is irrelevant. You don't experience the "real world". You experience a tiny slice of it, a few senses that is further slimmed down and even fabricated at parts.

To the bird who can intuitively sense and use electromagnetic waves for motion and guidance, your model of the world is fundamentally incomplete.

There is a projection of the world in text. Moreover training on additional modalities is trivial for a transformer. That's all that matters.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#123

Post by eviks »

> Suppose you want to be really good at predicting the next word. If you want to be really good, you have to understand what’s being said. That’s the only way. So by training something to be really good at predicting the next word, you’re actually forcing it to understand.

Ok, so in this framing you'll just get stuck below the "really good" level because the training isn't based on understanding

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#124

Post by metanonsense »

merizian wrote: post #82

The fallacy being made in this argument is that computers need to perform tasks the same way as humans to achieve equal or better performance on them. While having better "system 2" abilities may improve performance, it's plausible that scaled-up next-token prediction along with a bit of scaffolding and finetuning could match human performance on the same diversity of tasks while doing them a completely different way…

In addition to that, the "system 2" abilities might already be there with "epi" strategies like chain-of-thought prompting. Talking / writing to yourself might not be the most efficient way to think but at least I do it often enough when pondering a problem.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#125

Post by stubybubs »

continuational wrote: post #80
skepticATX wrote: post #72

Earlier quoted context omitted.

How about this: understanding is the ability to generalize knowledge and apply it to novel scenarios. This definition is something that humans, and animals for that matter, do every day - both in small and large ways. And this is something that current language models aren't very good at.

What is the test for this? I taught it Firefly, which is an undocumented programming language I'm working on, through conversion. I find it's a lot quicker than any human at picking up syntax and semantics, both in real time and in number of messages, and makes pretty good attempts at writing code in it, as much as you could expect from a human programmer. That is, until you run out of context - is this what you mean…

I gave it the three lightbulbs in a closet riddle.

https://puzzles.nigelcoldwell.co.uk/seven.htm

The key complication is "once you've opened the door, you may no longer touch a switch." It gets this. There are many examples of it written out on the web. When I give it a variation and say "you can open the door to look at the bulbs and use the switches all you want" and it is absolutely unable to understand this. To a human it's simple: look at the bulbs and flick the switches. It kept giving me answers about using a special lens to examine the bulbs, using something to detect heat. I explained it in many ways and tried several times. I was paying for GPT-4 at the time as well.

I would not consider this thinking. It's unable to make this simple abstraction from its training data. I think 4 looks better than 3 simply because it's got more data, but we're reaching diminishing returns on that, as has been stated.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#126

Post by lsy »

famouswaffles wrote: post #122
lsy wrote: post #116

Earlier quoted context omitted.

The difference is that "the world" is not exhaustible in the same way as Go is. While it's surely true that the number of possible overall Go game states is extremely large, the game itself is trivially representable as a set of legal moves and rules. The "world model" of the Go board is actually just already exhaustive and finite, and the computer's work in playing against itself is to generate more varied data with…

>The sum of all human artifacts ever made (or yet to be made) doesn't exhaust the description of a rock in your front yard, let alone the world in all its varied possibility. No human or creature we know of has a "true" world model so this is irrelevant. You don't experience the "real world". You experience a tiny slice of it, a few senses that is further slimmed down and even fabricated at parts. To the bird who can…

That's the difference though. I know my world model is fundamentally incomplete. Even more foundationally, I know that there is a world, and when my world model and the world disagree, the world wins. To a neural network there is no distinction. The closest the entire dynamic comes is the very basic annotation of RLHF which itself is done by an external human who is providing the value judgment, but even that is absent once training is over.

Despite not having the bird's sense for electromagnetic waves, I have an understanding that they are there, because humans saw behavior they couldn't describe and investigated, in a back-and-forth with a world that has some capacity to disprove hypotheses.

Additional modalities are really just reducible to more kinds of text. That still doesn't exhaust the world, and unless a machine has some ability to integrate new data in real time alongside a meaningful commitment and accountability to the world as a world, it won't be able to cope with the real world in a way that would constitute genuine intelligence.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#127

Post by marginalia_nu »

firebaze wrote: post #48
marginalia_nu wrote: post #20

Earlier quoted context omitted.

What are you even asking when you're asking why it works. Ultimately, it works the way it does because that's the way things are. I'm not sure what sort of answers you expect when asking why physics acts in a particular fashion. Also here's Feynman ranting about why questions: https://www.youtube.com/watch?v=36GT2zI8lVA

Asking "why" may lead to really new physics, though. I assume the reason for discovering the theory of relativity was asking "Why is the speed of light constant in all directions".

My point is, why isn't really a question that makes sense.

There isn't a "why" the speed of light is constant and isotropic. The answers you typically see for that question is just different ways of saying the same thing; there is no causal relationship.

In the end, it just is. There are things that must be true if this is the case, and things that can't be true if this is the case.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#129

Post by xcv123 »

overactor wrote: post #22
zeroonetwothree wrote: post #9

We have quantum mechanics, a model of the universe that makes predictions with absurd levels of accuracy. Yet we don’t really understand “why” it works—-I don’t think we can claim that we really “understand” the universe. This disproves the claim that making accurate predictions implies understanding. Or perhaps a better way to put it is that the model has some understanding but a model doesn’t necessarily reflect un…

Making accurate predictions in a specific domain doesn't imply understanding, sure. Universal autocomplete can make accurate predictions in all domains. Apples to oranges.

By "understanding" he simply means it builds a model of higher order abstractions over the text in order to model the meaning and context of each token in relation to other tokens. This is more sophisticated than a Markov process.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#130

Post by xcv123 »

mbrumlow wrote: post #5

> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…

By "understanding" he means it builds a model of higher order abstractions over the text in order to model the meaning and context of each token in relation to other tokens. This is more sophisticated than a Markov process.
Post reply on HN