AGI Overhyped?

> Firstly, what is AGI?

AGI is the biggest succesful scam in human history Sam Altman came up with to get the insane investment and hype they are getting. They are intentionally not defined what it is and when will be achieved, making it a never-reachable goal to keep the flow of money going. "we will be there in a couple of years", "this feels like AGI" was told every fucking GPT release.

It's the best interest for every AI lab to keep this lie going. They are not stupid, they know it can't be reached with the current state-of-the-art techniques, transformers, and even with the recent groundbreaking techniques like reasoning, and I think we are not even close.

> To make AI more general, you just make the input tokenization more granular, change the training method, and perhaps add some kind of iterative framework on top to make it do more stuff.

It's probably a bigger step than that. For example humans learn from experience, current AIs are trained offline and then frozen and can only "learn" by stuffing their short term memory with notes, like someone with anterograde amnesia.

AI technology is changing each year, and AGI will probably be more different from today's transformer-based systems than they were from convolutional nets.

But it will probably incorporate ideas and components from today's LLMs, and just as importantly, it's development will probably be paid for from the pockets of investors who have been tantalized by today's AI and think AGI must soon follow.

I personally hope it's a long way off but it might only be a few key insights away at this point.

> Unlike crypto there is obviously some value

To be fair, there is obviously some economic value in the fungibility of crypto-currency. The political and technical aspects are dubious.

> extraordinary claims require extraordinary evidence

Agreed, the only extraordinary achievement for this magic act so far is market capitalisation.

> Firstly, what is AGI? I've never heard a decent definition. Some say it's an AI that is as smart or as general as humans; some say it's an AI that's conscious.

Correct, but many have noticed this including Sam Altman in some interviews.

Everyone disagrees about all three initials of the initialism, plus if the whole even means something implied by those initials at all, plus treating each as a boolean rather than a number.

Also some people loudly reject this observation.

The definition I was using for "AGI" befor ChatGPT came out was met by ChatGPT-3.5: it's an AI that's general-purpose, it doesn't just e.g. play chess. But there's people who reject that it even counts as "AI" at all, despite, well, the Turing Test.

Anyway.

> I don't see how it could be a "step" by these definitions because nothing actually changes between a regular AI and an AGI. To make AI more general, you just make the input tokenization more granular, change the training method, and perhaps add some kind of iterative framework on top to make it do more stuff.

That's a lot of things.

> Also, AI is already more capable than humans in almost every way except scale.

No, not really. There's specific metrics where some AI can beat a lot of humans, like playing chess, or how many languages it speaks to the level of a competent adult learner, and they can pay attention to a lot more than we can, and they can process data faster than we can, but…

…but LLMs* are only giving the illusion of intelligence by using superhuman speed and superhuman attention to make up for having a mind with the complexity of a squirrel's that's had half a million years of experience reading everything it can lay its metaphorical hands on.

* not VLMs, they're not fast enough or smart-looking enough yet

> AI already blows most people out of an IQ test at a fraction of the computational power of a brain, but that's because IQ tests compare competence to get relative intelligence; they don't test computation.

IQ tests are indeed bad, can be learned. This is demonstrable because the graph you may have seen for AI IQ ratings has two variants from the same people, one with a public IQ test and one with a private IQ test, and most of the AI do much much worse with the private test: https://trackingai.org/home

The problem you're pointing at is also why the ARC-AGI test exists, and indeed this shows that current AI aren't anything close to human performance.

> With that assumption, if AI can't be computationally stronger than humans, it's safe to say we won't have conscious computers for a while, but instead computers that act consciously instead.

If you think "AGI" is poorly defined, you'll be horrified to learn that "consciousness" has something like 40 definitions. Nobody knows where it comes from in our brain structures or how much structure we need to have it. Is a mouse conscious? If so, there's enough compute going on for many AI models to be also. But the quantity of compute is likely a red herring, and the compute exists even if it's not running an AI, just as our brain chemistry still goes on while we're sleeping.

> Does that mean that AI from hear on out is a waste that does nothing but take control from people while benefiting us the same?

Yes, but such people will take any opportunity or tech to do so, we've seen that since "automation" meant "steam engine".

> What is ChatGPT going to look like in 5 years? Am I just going to type in "do my taxes," and it's just going to do whatever it wants on my pc until my taxes are done?

If you're lucky. I think probably not, but 5 years is too long to rule out an architectural breakthrough that makes them much less error-prone; the Transformer models in 2020 were unimpressive for anything except translation.

> Why would I ever want that over a system designed to do my taxes correctly EVERY TIME by accountants?

$$$; plus accountants aren't perfect, they're only human.

> One thing I know about AI is it is slow as a mf, AI is great but you really have to think, we really just built a giant dictionary guy who's going to have the same problems human employees have.

If we're lucky.

Most likely it will continue to have new and exciting problems.

Spot on. We struggle to define what we don't understand. And with AGI, that would include understanding ourselves. And when people start dragging in philosophy, religion, etc. you kind of know it's one of those things that definitely don't have much consensus.

That doesn't mean it's all nonsense. I think objectively we are getting quite a bit of emergent definitions that emerge from established fact and technology that kind of narrow this down. LLMs seem part of the solution on a path to some form of artificial intelligence that can keep up with us and that we might struggle to keep up with. But LLMs are not the whole solution. Though what they can't do keeps shifting. From glorified autocomplete to solving mathematical problems that were previously not solved. In the space of less than 3 years since the launch of chat GPT in November 2022. If the math wasn't solved before, there has to be a bit more to it than a glorified autocomplete.

Of course there are also counter points to that. But it at least challenges the notion that LLMs can't come up with new stuff. Even the notion that they can is deeply upsetting to some people because it challenges their world views. This debate is as much about that as it is about coming up with workable definitions.

LLMs are obviously lacking in a lot of ways. Perhaps the most obvious thing is that after their training is completed, they stop learning and adapting. They have no memory beyond their prompts and what they learned during training. A chat gpt conversation is just a large prompt with the entire history of the conversation and some clever engineering to load that into GPU memory relatively quickly. There are a lot of party tricks that involve elaborate system prompts and other workarounds.

The ability to remember and learn seems pretty fundamental to AGI, whatever that is. It's also a thing that doesn't sound like it's unsolvable. Some kind of indexed DB external to the AI for RAG kind of works but it does not solve the learning problem. And it shifts the problem to how to query and filter the data, which might not be optimal. And it's more similar to us using Google to find something than it is to us remembering information.

Also, it's not like we're particularly good at cramming large amounts of information down or learning. Learning is a slow process. And we kind of get set in our ways as we age. Meaning we're reluctant to learn more stuff and less capable of doing so. That suggests that even modest improvements might have dramatic results. LLMs + some short term memory and ability to learn might end up being a lot more useful.

I like duck typing in programming and that's also my mental model for AGIs. The Turing test is kind of obsolete. But if it quacks like duck and walks like a duck, it probably is a duck. Turing was onto something. Once we have a hard time telling apart the average person in society from an AI over days/weeks/years of interaction, we'll have AGIs. I think that's doable. But I'm not an ML engineer. A lot of those seem to believe it's doable too though.

The rest of society in the form of self appointed arm chair professors, religious leaders, eminent philosophers, and (too put this politely) lesser qualified individuals with strong opinions makes a lot of confused noises. But it does not generate a lot of definitions that have broad consensus.