351–360 of 523 posts
I think it depends on the definition of intelligence we are using. I feel that until A.I. is making better A.I. I wouldn’t really call it intelligent.
Earlier quoted context omitted.
In order to predict the next token it’s doing something more like simulating the writer of the words and the context they were likely to be in while writing the words. You cannot make accurate predictions without understanding the world that gave rise to these words. Consider a detective story with all the clues laid out and then at the end the detective says: “I know who it is. It is: …” Correctly predicting the nex…
“You cannot make accurate predictions without understanding the world that gave rise to these words.” This depends on the definition of understanding. There are an infinite number of equations that could describe the trajectory of a ball being thrown, and none of them are exactly correct depending on how deep down the understanding hole one travels.
The author's whole approach to AI is focusing on the supervised learning aspects of AI but ignoring the likelihood that future advancements in AI will come from reinforcement learning - much as how AlphaGo was first bootstrapped on human play but then improved through self play. On another note, this was amusing: "Many of my friends and colleagues are deeply impressed by their experiences with the latest big models,…
> Is this supposed to be reassuring? I have no idea why people think AIs will be better at general intelligence than they are at driving cars. At least why they think GI will improve any faster than driving.
What the hell, we are looking at essentially millions of matrix multiplications done on powerful GPUs. And we're told this is what the brain does. Which is supremely stupid, since as Bernardo Kastrup has emphasized many times, our brains are completely UNLIKE those systems. The hubris in Big Tech right now is truly pathetic and imho a sign of a real disconnect with a sense of meaning and purpose to human life. In fac…
> What the hell, we are looking at essentially millions of matrix multiplications done on powerful GPUs. And we're told this is what the brain does. Which is supremely stupid, since as Bernardo Kastrup has emphasized many times, our brains are completely UNLIKE those systems. That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a…
That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain.
Linux is written in C and that‘s Turing complete. And? Therefore Linux is like the brain?
All of these arguments seem to think that the brain isn't also generating a statistical ordering of semantic words and actions. Obviously the brain has a way more complex network of subsystems with various specialties than the artificial models we have today and is able to generalize language, math, and positional reasoning and mix it with the older parts of the brain for reward and training mechanisms and it can do…
Since LLMs can't do arithmetic, one has to think there's more going on, at least in the arithmetic part of the brain, than is going on in LLMs.
What you have that LLMs lack is a visual part of your brain - one which can instantly count quantities of objects up to about 7. That gives you tools that can be trained to do basic arithmetic operations. Although you have to be taught how to use that natural capability in your brain to solve arithmetic problems.
And of course for more complex things than simple arithmetic, you fall back on verbalized reasoning and association of facts (like multiplication tables) - which an LLM is capable of doing too.
Poor GPT though has only a one dimensional perceptual space - tokens and their embedding from start to end of its attention window - although who’s to say it doesn’t have some sense for ‘quantity’ of repeated patterns in that space too?
> If the new tech isn’t true artificial intelligence, then what is it? In my view, the most accurate way to understand what we are building today is as an innovative form of social collaboration. What an utterly vapid and ignorant view. In the rest of the article Lanier goes on to argue that it's all just "simple mathematics". Yeah, yeah, and computers are just simple NAND gates, amirite? I've waffled over the years…
Earlier quoted context omitted.
> What the hell, we are looking at essentially millions of matrix multiplications done on powerful GPUs. And we're told this is what the brain does. Which is supremely stupid, since as Bernardo Kastrup has emphasized many times, our brains are completely UNLIKE those systems. That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a…
> That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a transformer. That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain. Linux is written in C and that‘s Turing complete. And? Therefore Linux is like the brain…
It's not irrelevant, it was merely the premise for the following point, which was that transformers are trained to learn a function. Since they're Turing complete, they can thus hypothetically implement any function, including the function governing human speech, as I described.
If transformers were not Turing complete, then they arguably would not be computationally powerful enough to model the brain which we know is Turing complete.
What is irrelevant is trying to dismiss transformers as intelligent, or like the brain, simply because they're "merely matrix multiplications".
Earlier quoted context omitted.
This is what I got on ChatGPT today. I assume it used GPT4: Prompt ChatGPT Actual Match 397,356 * 930,547 369,685,207,932 369,758,433,732 FALSE 36,330 * 26,951 979,458,630 979,129,830 FALSE 8,681 * 9,330 80,911,430 80,993,730 FALSE 278 * 903 250,734 251,034 FALSE 82 * 77 6,314 6,314 TRUE Edit: # of correct digits (counting from leftmost) only exceeds 3 on the smallest pair. It drops to two, as well, on the 3x3 set.
You probably used GPT-3.5. That said, I didn't manage to get GPT-4 to calculate 36,330 * 26,951 correctly. I suggested casting out 9s, casting out 11s, doing long multiplication, reversing the digits - nothing. I have a theory that it does arithmetic badly because the logic goes right-to-left, when LLMs write left-to-right. If the digits were to be reversed, it might not make as many mistakes. I ran out of attempts b…
GPT’s attention window is not equivalent to it being able to ‘write stuff down’ - it’s its equivalent of being able to remember the foregoing few minutes of conversation.
All of these arguments seem to think that the brain isn't also generating a statistical ordering of semantic words and actions. Obviously the brain has a way more complex network of subsystems with various specialties than the artificial models we have today and is able to generalize language, math, and positional reasoning and mix it with the older parts of the brain for reward and training mechanisms and it can do…