21–30 of 279 posts
It is technically correct that when you take things to the extreme you can accomplish great things, but we may not reach those levels. We may require completely different technology to reach those levels of autocomplete, and we have simply reached a new plateau at this point in time.
We have quantum mechanics, a model of the universe that makes predictions with absurd levels of accuracy. Yet we don’t really understand “why” it works—-I don’t think we can claim that we really “understand” the universe. This disproves the claim that making accurate predictions implies understanding. Or perhaps a better way to put it is that the model has some understanding but a model doesn’t necessarily reflect un…
This kind of was on my mind recently, I was wondering, when I speak am I just spitting out the next word that makes sense or is there more to it. I think there is for people, I can think before I speak, I can plan out my thought entirely before turning it into words. Each invocation of the chat bot/llm is a new set of probabilities. I can plan what my 2nd token output will be and stick to it. Llm models dont have the…
In theory models can/could do the same thing. Think of the current text output of a model being those thoughts inside your head, you have an internal scratch space you work on ideas, then after you perform 'chain of thought' on them, you output to the world. What you're outputting to the world isn't your directly thoughts, but more of a calculation of what you wanted to achieve.
Really for LLMs you just need to have the model put it's output to an internal buffer, read that buffer and make sure it makes sense, then output that to the end user.
It's probably not fast or compute efficient to do that at this time, but at some point it likely will be.
I still want to read a ChatGPT text that doesn't look stupidly verbose
> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…
> You would realize that all we are doing is using statistics to predict what the next item might be. So what does "understanding" really means then? "Understanding" is not really well defined. Either we (humans) do it, but then LLMs might just do it as well, depending on the definition of "understanding", or we both don't do it. But if no-one is really "understanding" anything, then this definition of "understanding…
> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…
> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…
> You would realize that all we are doing is using statistics to predict what the next item might be. So what does "understanding" really means then? "Understanding" is not really well defined. Either we (humans) do it, but then LLMs might just do it as well, depending on the definition of "understanding", or we both don't do it. But if no-one is really "understanding" anything, then this definition of "understanding…
Everybody opining about it is doing just that: offering an opinion. Geoff Hinton’s opinion is worth more than someone else’s, but it is still an opinion.
> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…
ChatGPT routinely does both.
> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…
Isn't that exactly how humans learn to respond to stimuli? Don't we just try to predict the best next response to everything? Yes, It's statistics but the fun part is nobody is writing this statistical function by hand.
I think LLMs ultimately just take imitation to a creative and sophisticated extreme. And imitation simply doesn't comprise the whole of human intelligence at all, no matter how much it is scaled up.
The sophistication of the imitation has some people confused and questioning whether everything can be reduced to imitation. It can't.
The ability to imitate seeking a goal isn't identical to the ability to seek a goal.
The ability to imitate solving a problem isn't identical to the ability to solve a problem.
Imitation is very useful, and the reduction of everything to imitation is an intriguing possibility to consider, but it's ultimately just wrong.
The argument “a sufficiently capable autocomplete must contain a level of general intelligence” is correct but also not very useful. It is a lot like saying “a sufficiently fast horse can fly”. It is technically correct that when you take things to the extreme you can accomplish great things, but we may not reach those levels. We may require completely different technology to reach those levels of autocomplete, and w…