Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I disagree with Geoff Hinton regarding "glorified autocomplete"

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

91–100 of 279 posts

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#91

Post by jameshart »

usrbinbash wrote: post #78
mbrumlow wrote: post #5

> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…

> You would realize that all we are doing is using statistics to predict what the next item might be. Two questions: 1. How can humans come up with novel ideas then? Statistically predicting the next item means I am limited to the training set. How do humans formulate new concepts in this scenario? 2. Why can humans learn from their own output then? If all we do is statistically predicting the next token, then, by ne…

> Statistically predicting the next item means I am limited to the training set

Not at all. Extrapolation is perfectly possible in a purely predictive model. It’s one of the things GPTs are best at.

In the stream of tokens output by an LLM it’s completely possible for new concepts to emerge, and for it then to continue to use and build on them in the remainder of the stream. You see this simply executed in programming contexts with LLMs where it is able to declare a novel function and then use it.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#92

Post by ijidak »

famouswaffles wrote: post #46
cmdli wrote: post #21

The argument “a sufficiently capable autocomplete must contain a level of general intelligence” is correct but also not very useful. It is a lot like saying “a sufficiently fast horse can fly”. It is technically correct that when you take things to the extreme you can accomplish great things, but we may not reach those levels. We may require completely different technology to reach those levels of autocomplete, and w…

The argument is simpler than that. Prediction requires a model, completely accurate or not. There's a projection of the world in text. A model of the text data we feed it is a model of the world as humans see it. The trend of loss is more and more accurate models of the dataset. So it won't stop at any arbitrary competency level. Indeed, there are already a few abilities GPT possess that are deemed Super Human. It's…

I've noticed that when I speak I really don't control each word.

I have an idea that I want to convey, but how each word comes to my mind as I form a sentence has always felt like it's controlled by an unconscious algorithm.

So I don't understand why people find this prediction mechanism so alien.

It isn't clear to me how much of communication is really in our control.

With the current tools, it feels like we still provide the ideas we want the AI to convey, and it may be using a nearly identical mechanism to us to form the words.

Consciousness would be the computer being able to come up with the ideas.

So, it seems to me we've gotten close enough on the communication side of intelligence.

But the machine is not conscious. When it is, it seems like it will generate its own ideas.

Are people debating whether the machine is conscious?

Otherwise, it feels very straightforward to grasp what we've made up to now.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#93

Post by TerrifiedMouse »

brookst wrote: post #63
TerrifiedMouse wrote: post #55

Earlier quoted context omitted.

> when I speak am I just spitting out the next word that makes sense or is there more to it. There is more to it. Specifically you are doing so to advance towards a specific goal. LLMs don't have goals. They just pick from a list of likely tokens - based on their training data - at random to generate the next token.

So the externally provided metaprompt doesn’t establish a goal that generation works toward?

I don’t think LLMs work towards anything. It just picks from a list of likely “next tokens” at random.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#94

Post by skepticATX »

continuational wrote: post #80
skepticATX wrote: post #72

Earlier quoted context omitted.

How about this: understanding is the ability to generalize knowledge and apply it to novel scenarios. This definition is something that humans, and animals for that matter, do every day - both in small and large ways. And this is something that current language models aren't very good at.

What is the test for this? I taught it Firefly, which is an undocumented programming language I'm working on, through conversion. I find it's a lot quicker than any human at picking up syntax and semantics, both in real time and in number of messages, and makes pretty good attempts at writing code in it, as much as you could expect from a human programmer. That is, until you run out of context - is this what you mean…

There are plenty of results supporting my assertion; but the tests must be carefully designed. Of course, LLMs are not databases that store exact answers - so it's not enough to ask it something that it hasn't seen, if it's seen something similar (as is likely the case with your programming language).

One benchmark that I track closely is ConceptARC, which aims to test generalization and abstraction capabilities.

Here is a very recent result that uses the benchmark: https://arxiv.org/abs/2311.09247. Humans correctly solved 91% of the problems, GPT-4 solved 33%, and GPT-4V did much worse than GPT-4.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#96

Post by ffwd »

I think there is another aspect to human thinking other than system 1/system 2 though, which is the abstract world model humans have. system 1 / 2 is more like the process, while the world model is the actual data being 'processed'.

And I think basically, humans have a much simplified, 'low dimensional' world model that consists of a set of objects (let's call them patterns), and then a "list" of essential properties that those objects have, that leads to a constraint on how each object can behave in the world model as a whole.

And this is sort of hierarchical or at least, we can zoom in and out in detail depending on the level of knowledge we have about a particular pattern.

So problem 1 is: It's not clear to me that text or any sort of data would contain all the necessary constraints so that any particular prompt would result in a world model that exactly takes into account the constrains of each object and 2) Even if was, I'm not sure the process of step by step thinking (system1/2) about each object and computing world states could occur in current architectures. This is especially important for computing a set of objects, then abstracting the result, then doing another round of computing with that result, or something like this.

I'm not hard set on this but this is my current thinking.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#97

Post by brookst »

TerrifiedMouse wrote: post #93
brookst wrote: post #63

Earlier quoted context omitted.

So the externally provided metaprompt doesn’t establish a goal that generation works toward?

I don’t think LLMs work towards anything. It just picks from a list of likely “next tokens” at random.

“Likely” as defined by…?

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#98

Post by abm53 »

civilized wrote: post #52
lordnacho wrote: post #39

Earlier quoted context omitted.

Ok, so now we need an example that separates humans from LLMs? I struggle to think of one, maybe someone on HN has a good example. Eg if I'm in middle school and learning quadratic equations, am I imitating solving the problem by plugging in the coefficients? Or am I understanding it? Most of what I see coming out of chatGPT and copilot could be said to be either. If you're generous, it's understanding. If not, it's…

It is very easy to separate humans from LLMs. Humans created math without being given all the answers beforehand. LLMs can't do that yet. When an LLM can create math to solve a problem, we will be much closer to AGI.

You can ask ChatGPT to solve maths problems which are not in its training data, and it will answer an astonishing amount of them correctly.

The fact that we have trained it on examples of human-produced maths texts (rather than through interacting with the world over several millennia) seems like more of an implementation detail and not piece of evidence about whether it has “understood” or not.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#99

Post by robbrown451 »

I agree with Hinton, although a lot hinges on your definition of "understand."

I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go.

Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they found is that there isn't really a limit to how good it can get. There may be some theoretical limit of a "perfect" Go player, or maybe not, but it will continue to converge towards perfection by continuing to train. And it can go far beyond what the best human Go player can ever do. Even though very smart humans have spent their lifetimes deeply studying the game, and Alpha Zero had to learn everything from scratch.

One other thing to take into consideration, is that to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move. And that model is obviously very sophisticated, and if you are going to say that it doesn't understand the game of Go, I would argue that you have a very, oddly restricted definition of the word, understand, and one that isn't particularly useful.

Likewise, with large language models, while on the surface, they may be just predicting the next word one after another, to do so effectively they have to be planning ahead. As Hinton says, there is no real limit to how sophisticated they can get. When training, it is never going to be 100% accurate in predicting text it hasn't trained on, but it can continue to get closer and closer to 100% the more it trains. And the closer it gets, the more sophisticated model it needs. In the sense that Alpha Zero needs to "understand" the game of Go to play effectively, the large language model needs to understand "the world" to get better at predicting.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#100

Post by genman »

civilized wrote: post #29
satuke wrote: post #12

Earlier quoted context omitted.

Isn't that exactly how humans learn to respond to stimuli? Don't we just try to predict the best next response to everything? Yes, It's statistics but the fun part is nobody is writing this statistical function by hand.

LLMs don't have a concept of "best". Only most likely in what they've been trained on. I think LLMs ultimately just take imitation to a creative and sophisticated extreme. And imitation simply doesn't comprise the whole of human intelligence at all, no matter how much it is scaled up. The sophistication of the imitation has some people confused and questioning whether everything can be reduced to imitation. It can't.…

I don't think that also humans in general have this concept of "best".

But humans are able to build certain routines within their own system to help them to rationalize.

Post reply on HN