I disagree with Geoff Hinton regarding "glorified autocomplete"
statmodeling.stat.columbia.edu
I disagree with Geoff Hinton regarding "glorified autocomplete"
1–10 of 279 posts
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#2[deleted]
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#3I still want to read a ChatGPT text that doesn't look stupidly verbose
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#4I still want to read a ChatGPT text that doesn't look stupidly verbose
Why don't you ask it not to be?
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#5> If you want to be really good, you have to understand what’s being said. That’s the only way.
This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or requirement for the system doing the prediction to understand.
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#6This kind of was on my mind recently, I was wondering, when I speak am I just spitting out the next word that makes sense or is there more to it.
I think there is for people, I can think before I speak, I can plan out my thought entirely before turning it into words. Each invocation of the chat bot/llm is a new set of probabilities. I can plan what my 2nd token output will be and stick to it. Llm models dont have the ability to think past the first.
To me that is auto completing, just done better, unless models really do have a sense of planning ahead. But I don't think the really complex game playing ai do that either.
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#7I still want to read a ChatGPT text that doesn't look stupidly verbose
Why don't you ask it not to be?
The default style really annoys me with its verbosity and overuse of numbered lists. Describing a tone and length of responses and anything else you want - think step by step, first principals, show don’t tell - all work pretty well.
You can even ask it to only reply with a Boolean true or false - which certainly doesn’t play to its strengths
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#8I think this misses the point made by Hinton. The point is that, if a model is to reach a certain level of autocomplete, it needs to understand what it is talking about. Imagine the result of what required deep thought from you is included in the training data, along with billions of other such exchanges. At some point the only way to improve training score is to develop the ability to "understand" things. (That is, if the model isn't big enough to memorize the training data.)
Universal autocomplete implies understanding and the only ending question is if our current architectures and model sizes are able to achieve any level of it.
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#9We have quantum mechanics, a model of the universe that makes predictions with absurd levels of accuracy. Yet we don’t really understand “why” it works—-I don’t think we can claim that we really “understand” the universe.
This disproves the claim that making accurate predictions implies understanding. Or perhaps a better way to put it is that the model has some understanding but a model doesn’t necessarily reflect underlying truth.
Re: I disagree with Geoff Hinton regarding "glorified autocomplete"
#10> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…
> You would realize that all we are doing is using statistics to predict what the next item might be.
So what does "understanding" really means then?
"Understanding" is not really well defined. Either we (humans) do it, but then LLMs might just do it as well, depending on the definition of "understanding", or we both don't do it. But if no-one is really "understanding" anything, then this definition of "understanding" is maybe not useful.