I disagree with Geoff Hinton regarding "glorified autocomplete" • Hacker News Board

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#111

Post by usrbinbash » Sat, Nov 18, 2023, 6:45 PM UTC

jameshart wrote: post #91

usrbinbash wrote: post #78

Earlier quoted context omitted.

> You would realize that all we are doing is using statistics to predict what the next item might be. Two questions: 1. How can humans come up with novel ideas then? Statistically predicting the next item means I am limited to the training set. How do humans formulate new concepts in this scenario? 2. Why can humans learn from their own output then? If all we do is statistically predicting the next token, then, by ne…

> Statistically predicting the next item means I am limited to the training set Not at all. Extrapolation is perfectly possible in a purely predictive model. It’s one of the things GPTs are best at. In the stream of tokens output by an LLM it’s completely possible for new concepts to emerge, and for it then to continue to use and build on them in the remainder of the stream. You see this simply executed in programmin…

> with LLMs where it is able to declare a novel function and then use it.

Novel as in "implements a new algorithm that has never seen in any form and is actually an improvement over existing methodology"?

Here is a little thought experiment: If all the training data in the set says that manned, powered flight is impossible, is a statistical prediction engine trained on that data capable of developing an airplane?

In other words, could a very powerful stochastic parrot do what the Wright Brothers did?

My best guess is that no, it cannot.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#112

Post by jon_richards » Sat, Nov 18, 2023, 6:50 PM UTC

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

> to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move.

While I imagine alpha go does some brute force and some tree exploration, I think the main "intelligent" component of alpha go is the ability to recognize a "good" game state from a "bad" game state based on that moment in time, not any future plans or possibilities. That pattern recognition is all it has once its planning algorithm has reached the leaves of the trees. Correct me if I'm wrong, but I doubt alpha go has a neural net evaluating an entire tree of moves all at once to discover meta strategies like "the opponent focusing on this area" or "the opponent feeling on the back foot."

You can therefore imagine a pattern recognition algorithm so good that it is able to pick a move by only looking 1 move into the future, based solely on local stone densities and structures. Just play wherever improves the board state the most. It does not even need to "understand" that a game is being played.

> while on the surface, they may be just predicting the next word one after another, to do so effectively they have to be planning ahead.

So I don't think this statement is necessarily true. "Understanding" is a major achievement, but I don't think it requires planning. A computer can understand that 2+2=4 or where to play in tic-tac-toe without any "planning".

That said, there's probably not much special about the concept of planning either. If it's just simulating a tree of future possibilities and pruning it based on evaluation, then many algorithms have already achieved that.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#113

Post by Jensson » Sat, Nov 18, 2023, 6:50 PM UTC

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

A big difference between a game like Go and writing text is that text is single player. I can write out the entire text, look at it and see where I made mistakes on the whole and edit those. I can't go back in a game of Go and change one of my moves that turned out to be a mistake.

So trying to make an AI that solves the entire problem before writing the first letter will likely not result in a good solution while also making it compute way too much since it solves the entire problem for every token generated. That is the kind of AI we know how to train so for now that is what we have to live with, but it isn't the kind of AI that would be efficient or smart.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#114

Post by Probiotic6081 » Sat, Nov 18, 2023, 6:50 PM UTC

breadwinner wrote: post #106

There is evidence that the human brain is also doing "autocomplete" (prediction). The human brain uses predictive mechanisms when processing language, and these mechanisms play an important role in forming thoughts. When we hear or read a word, our brain quickly generates a set of predictions about what word might come next, based on the context of the sentence and our past experiences with language. These prediction…

And curiously, those predictions that are made during language comprehension are made by the language production system itself!

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#115

Post by klodolph » Sat, Nov 18, 2023, 6:54 PM UTC

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

> As Hinton says, there is no real limit to how sophisticated they can get.

There’s no limit to how sophisticated a model can get, but,

1. That’s a property shared with many architectures, and not really that interesting,

2. There are limits to the specific ways that we train models,

3. We care about the relative improvement that these models deliver, for a given investment of time and money.

From a mathematical perspective, you can just kind of keep multiplying the size of your model, and you can prove that it can represent arbitrary complicated structures (like, internal mental models of the world). That doesn’t mean that your training methods will produce those complicated structures.

With Go, I can see how the model itself can be used to generate new, useful training data. How such a technique could be applied to LLMs is less clear, and its benefits are more dubious.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#116

Post by lsy » Sat, Nov 18, 2023, 6:56 PM UTC

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

The difference is that "the world" is not exhaustible in the same way as Go is. While it's surely true that the number of possible overall Go game states is extremely large, the game itself is trivially representable as a set of legal moves and rules. The "world model" of the Go board is actually just already exhaustive and finite, and the computer's work in playing against itself is to generate more varied data within that model rather than to develop that model itself. We know that when Alpha Zero plays a game against itself it is valuable data because it is a legitimate game which most likely represents a new situation it hasn't seen before and thus expands its capacity.

For an LLM, this is not even close to being the case. The sum of all human artifacts ever made (or yet to be made) doesn't exhaust the description of a rock in your front yard, let alone the world in all its varied possibility. And we certainly haven't figured out a "model" which would let a computer generate new and valid data that expands its understanding of the world beyond its inputs, so self-training is a non-starter for LLMs. What the LLM is "understanding", and what it is reinforced to "understand" is not the world but the format of texts, and while it may get very good at understanding the format of texts, that isn't equivalent to an understanding of the world.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#117

Post by lsy » Sat, Nov 18, 2023, 7:04 PM UTC

breadwinner wrote: post #106

There is evidence that the human brain is also doing "autocomplete" (prediction). The human brain uses predictive mechanisms when processing language, and these mechanisms play an important role in forming thoughts. When we hear or read a word, our brain quickly generates a set of predictions about what word might come next, based on the context of the sentence and our past experiences with language. These prediction…

This is of course sometimes true, we take shortcuts to minimize cognitive effort. However, when the situation warrants it we think more carefully about which words to use to achieve goals, or to reach correspondence with the situation at hand. Or we move more precisely and carefully to do something we haven't done before. I've no doubt that an LLM can approximate whatever model of language a person has from their life experience, but I don't think this type of model is capable of active coping, making judgments, or of having accountability to the world it's meant to operate in.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#118

Post by [deleted] » Sat, Nov 18, 2023, 7:11 PM UTC

lsy wrote: post #116

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

The difference is that "the world" is not exhaustible in the same way as Go is. While it's surely true that the number of possible overall Go game states is extremely large, the game itself is trivially representable as a set of legal moves and rules. The "world model" of the Go board is actually just already exhaustive and finite, and the computer's work in playing against itself is to generate more varied data with…

[deleted]

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#119

Post by huytersd » Sat, Nov 18, 2023, 7:13 PM UTC

robbrown451 wrote: post #99

I agree with Hinton, although a lot hinges on your definition of "understand." I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go. Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they f…

Next word generation is one way to put it. The key point here is we have no idea what’s happening in the black box that is the neural network. It could be forming very strong connections between concepts in there with multi tiered abstractions.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#120

Post by naveen99 » Sat, Nov 18, 2023, 7:18 PM UTC

IshKebab wrote: post #108

civilized wrote: post #52

Earlier quoted context omitted.

It is very easy to separate humans from LLMs. Humans created math without being given all the answers beforehand. LLMs can't do that yet. When an LLM can create math to solve a problem, we will be much closer to AGI.

Some humans created maths. And it took thousands of years of thinking and interaction with the real world. Seems like goalpost moving to me. I think the real things that separate LLMs from humans at the moment are: * Humans can do online learning. They have long term memory. I guess you could equate evolution to the training phase of AI but it still seems like they don't have quite the same on-line learning capabilit…

Interestingly some humans will admit to not knowing but are allergic to admitting being wrong (and can get fairly vindictive if forced to admit being wrong).

LLM’s actually admit to being wrong easily, but aren’t great at introspection and confabulate too often. also their Meta cognition is poor still.