AGI Overhyped?

I keep hearing about how AGI is going to change the game and how it's going to immediately lead to a dangerous ASI. I even hear it from people I know who know almost nothing about how AI works. I am no pro myself, but I think having a fundamental understanding of how AI computes and stores information changes the perspective greatly.

Firstly, what is AGI? I've never heard a decent definition. Some say it's an AI that is as smart or as general as humans; some say it's an AI that's conscious. I don't see how it could be a "step" by these definitions because nothing actually changes between a regular AI and an AGI. To make AI more general, you just make the input tokenization more granular, change the training method, and perhaps add some kind of iterative framework on top to make it do more stuff. Also, AI is already more capable than humans in almost every way except scale.

I personally think that it's just a hype word that Altman spammed to get more funding and interest in OpenAI. Even if you snapped your fingers and had the right training mechanism and the right networks etc for an AGI/ASI, I get the feeling it wouldn't even be smarter than people in the technical sense. AI already blows most people out of an IQ test at a fraction of the computational power of a brain, but that's because IQ tests compare competence to get relative intelligence; they don't test computation.

With that assumption, if AI can't be computationally stronger than humans, it's safe to say we won't have conscious computers for a while, but instead computers that act consciously instead. Does that mean that AI from hear on out is a waste that does nothing but take control from people while benefiting us the same? What is ChatGPT going to look like in 5 years? Am I just going to type in "do my taxes," and it's just going to do whatever it wants on my pc until my taxes are done? Why would I ever want that over a system designed to do my taxes correctly EVERY TIME by accountants? One thing I know about AI is it is slow as a mf, AI is great but you really have to think, we really just built a giant dictionary guy who's going to have the same problems human employees have.

I don't know just kind of spewing thoughts, I'd love to hear from people who are actual experts in designing these things.

You're absolutely right that AGI and ASI are not well-defined, but we also need to recognize that they definitely are actual things even though we lack a definition.

For any AI concept, I think it might be instructive to consider the human equivalent.

For example, what is the definition of a genius? We don't have one, but genius certainly is a thing that exists. We might argue about who is/isn't a genius because of the missing definition, but we probably agree that exceptional people are a thing.

> I've never heard a decent definition

I would say AGI can do every intellectual task a human can do. Maybe the raw cognitive power is already there but scale matters. If it can't work on a task for months on end, it's not AGI. Because humans can do it.

> AI already blows most people out of an IQ test at a fraction of the computational power of a brain

Does it? I thought the brain is much more energy efficient.

I think the more obvious problem is AI, given access to actuators, still couldn't untangle a ball of string successfully.

The systems appear smart because they're language models trained on quality tested text.

> AI already blows most people out of an IQ test at a fraction of the computational power of a brain

AFAIK, IQ tests used in psychological evaluations do not contain any randomness so exact answers are almost always in distribution. I haven't seen someone compare AI to an IQ test that is not in distribution.

On ARC-AGI, which is mildly similar to a randomly generated IQ test, humans still are much better than LLMs. https://arcprize.org/ (scroll down for chart)