Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is AGI-Hard

lspace.swyx.io

Post reply on HN
Print view View original

21–30 of 129 posts

Re: What Is AGI-Hard

#21

Post by seanhunter »

janalsncm wrote: post #6

This article is a bit too hand-wavy for my taste. As a buzzword, AGI Hard sounds cool but until intelligence is more strictly defined (and the modifiers “artificial” and “general” aren’t helping) AGI will always be something we talk about rather than something known. I was also a bit amused at the list of things which the author also claims to be AGI hard. But this is just a list of things we think are difficult. Phy…

Agree. I really dislike this terminology because like many analogies it tempts you into thinking it means more than it really does and thereby reasoning from it to things which are entirely unsupported by anything sound.

NP-Hardness and NP-Completeness are rigorously-defined concepts. By proving a problem NP hard you establish some extremely important properties about its nature and its relationship with other problems.

AGI-Hard sounds like the same sort of thing but really isn't. It's fundamentally just pure speculation. Of the three letters "AGI" it's arguable that we don't have a hard definition of what constitutes any of them. Then further to that we don't have any kind of idea what being in the set of AGI-Hard problems might mean other than you're in the set. Do these hypothetical AGI-Hard problems have some relationship with each other? Whereas we know P is in NP, do we have an equivalent class to P and what would that mean other than "The class of problems solvable by AGI"?

You can see this weakness when you look at the proposed examples of what is purportedly AGI-hard. What do they have in common? Mostly just that they are weakly-defined themselves. For example. How would you judge if an AGI derived from first principles something that isn't calculus but is equivalently important/innovative? Well you'd first need to define what that equivalence would mean.

Re: What Is AGI-Hard

#22

Post by imtringued »

If AGI is a class of problems that require AGI then AGI hard only refers to the hardest problems that any AGI problem can be translated to.

If we assume that humans are in AGI then AGI hard would be at least as intelligent as humans but possibly even more intelligent than humans.

I don't believe most people have "superior over humans" in mind when they say AGI-hard. They just mean a hard problem that is also in AGI.

Re: What Is AGI-Hard

#23

Post by janalsncm »

swyx wrote: post #17
janalsncm wrote: post #6

This article is a bit too hand-wavy for my taste. As a buzzword, AGI Hard sounds cool but until intelligence is more strictly defined (and the modifiers “artificial” and “general” aren’t helping) AGI will always be something we talk about rather than something known. I was also a bit amused at the list of things which the author also claims to be AGI hard. But this is just a list of things we think are difficult. Phy…

chess as a closed system is definitely solved by AI, including bootstrapping purely from self-play. the claim here is that applying physics/math/etc would be AGI-hard, because it involves the AI perfectly modeling a world it doesnt live in (the simulation problem) or understanding relationships in concepts without making jumps (obeying logic).

It’s not clear to me that the bottleneck for simulating the universe is intelligence. Our measurement capabilities are limited, and our computational power is limited.

When I say our measurement capabilities are limited, I mean that there are subsystems which are fundamentally chaotic. Small measurement inaccuracies will lead to large prediction inaccuracies, which in my view has little to do with intelligence.

Re: What Is AGI-Hard

#24

Post by jillesvangurp »

logicallee wrote: post #13

I hereby endorse ChatGPT as a limited form of AGI. It can complete a variety of novel cognitive tasks (example: judging texts and proposals) often enough that it meets my personal requirements for this. My opinion is that the definitions which others believe disqualify ChatGPT from being considered AGI are not relevant, since they focus on things it fails at rather than what it can do. For example, when it tests as w…

Exactly the fallacy with people's expectations of AI is that it has to have a perfect track record of decision making. We humans actually fail that test and quite miserably so. Take self driving cars as an example.

Self driving cars: barely any fatal accidents but some erratic behavior. People go "oh we can't have that because something extremely unlikely might kill somebody. Never mind it hasn't happened yet and might never happen. But it just might. Therefore I will never trust AIs!!!!"

Self driving humans: lots of fatal accidents and most of them because of human error. People go: "that's just the way things are; price of being on the road is playing Russian roulette"

It's irrational. Trick question: Would you rather drive on a road where every other driver is an AI or on a road where the average driver is a fair selection of the current drivers? I'd feel safer with AIs probably. I risk my life regularly taking part in the traffic. It's not great. So, yes less of that would be great. IMHO that will become the norm not so long from now because of insurance fees.

So, yes, chat gpt makes mistakes and sometimes talks out of its ass. Read the average HN comment section for some human equivalent of that. Lots of supposedly intelligent people asserting things that are patently false, people applying broken logic, speaking with confidence about a brain fart they had, etc.

We're applying double standards here. The key thing for AGI is the ability to learn and improve. Chat gpt3 is not there yet. You train it once and then you deploy it with static capabilities. It never gets better. It doesn't experiment and learn from the mistakes it makes. It lacks the capability to learn. You can't even educate it about the mistakes it makes. Using chat gpt-3 does not evolve the AI model. Using your brain does evolve it. Out of the box, it's actually quite useless. Forget self driving. Just learning to walk takes years. It takes many years more to get some sensible thought out of a human being. And frankly, only very few people get very good at that.

An AGI would need the ability to absorb new information and adapt how it behaves based on that. It doesn't have to get it perfectly in one go. It just needs to be able to adapt at a similar or better pace than we do to rapidly get to the point where it is going to be better than us at just about anything it bothers to learn.

Re: What Is AGI-Hard

#25

Post by trashtester »

swyx wrote: post #17
janalsncm wrote: post #6

This article is a bit too hand-wavy for my taste. As a buzzword, AGI Hard sounds cool but until intelligence is more strictly defined (and the modifiers “artificial” and “general” aren’t helping) AGI will always be something we talk about rather than something known. I was also a bit amused at the list of things which the author also claims to be AGI hard. But this is just a list of things we think are difficult. Phy…

chess as a closed system is definitely solved by AI, including bootstrapping purely from self-play. the claim here is that applying physics/math/etc would be AGI-hard, because it involves the AI perfectly modeling a world it doesnt live in (the simulation problem) or understanding relationships in concepts without making jumps (obeying logic).

Any AGI that is supposed perform tasks in the physical world, will need some kind of world model. The model doesn't need to be perfect to be immensly useful. It just need to model elements that are necessary to make more accurate predictions in its domain.

I would argue that exactly the same goes for the human brain. Consciousness seems to play the part of the World Model for us. Between the quantum wavefunctions of the world we live in and our conscious experience (or even our sub-conscious data processing) there are layers-upon-layars-upon-layers of data loss and data compression/abstraction. Our sensory endpoints probably receive more raw data in a second than our consciousness processes in a year.

The reason it still works so well, I would argue, is a mix of darwinian learning that has been going on for hundreds of billions of years before humanity even appeared, where a data pipeline has been created in our hardware. This, combined with our ability to learn by interacting with the real world.

Creating an AI with a "perfect" world model is impossible. It will require more compute power than exists in the universe. Rather, AGI will need a simpler world model that captures the parts that are essential for its functioning, and that can be automatically updated using its input data (ideally sensors).

Just like us.

And, it seems to me, this is already happening. A self driving car will create an approximation for a 6-dimensional state for all objects (position and velocity), possibly with acceleration, speculation about intent or other "mental states. It will then extrapolate this a few seconds (or more) into the future, while keeping track of the confidence of the predictions.

This is a similar kind of world model that human brains seem to use, at least for the purpose of driving. Clearly, our brain needs a more complex model, since we make predictions much further into the future (such as the education of our children, or beyond) and we also interact with the world in more ways.

When AI systems start to make use of world model of the same complexity (type and level) as our own, and is able to "train" it both "genetically" and by standard "deep learning", I suspect it will be increasingly good at displaying what we consider "common sense".

And juding by recent developments, this could happens sooner than many think. If we're able to combine the features of ChatGPT, Tesla Self Driving, Stable Diffusion and perhaps a few other in a single model (that takes both visual data, sound and text as input, and which constructs a wold model that combines physical and abstract/textual/behavioral elements), this could happen within 10 years.

Re: What Is AGI-Hard

#26

Post by Tepix »

suprjami wrote: post #11

King's Quest 2 is AGI hard. I have no idea how anyone was supposed to find required items hidden away in any of hundreds of trees in the game. It was definitely a different time for video games back then.

But that sounds like it could just be brute forced.

Re: What Is AGI-Hard

#27

Post by Tepix »

vanviegen wrote: post #20
robgibbons wrote: post #19

Earlier quoted context omitted.

Ask ChatGPT if it's an AGI, and it will tell you plainly that it is not, and why. Here's GPT's response: No, ChatGPT is not an AGI (Artificial General Intelligence). It is a large language model that has been trained to predict the next word in a sequence of text given the previous words. While ChatGPT has been trained on a diverse range of text data and is able to generate human-like text, it is not capable of under…

I asked chatGPT how come it's so good at programming tasks. It told me it does not in fact understand programs, nor is it able to write programs. For the rest of the thread, it refused to perform any programming tasks: "As I told you, I'm unable to [...]". Don't mind me, I'm just a language model. These types of responses are, I believe, part of a separately trained safety system, that should kick in when people are…

Agree. ChatGPT is often doing something that appears to be reasoning yet it claims not to be able to reason.

Re: What Is AGI-Hard

#28

Post by trashtester »

janalsncm wrote: post #23
swyx wrote: post #17

Earlier quoted context omitted.

chess as a closed system is definitely solved by AI, including bootstrapping purely from self-play. the claim here is that applying physics/math/etc would be AGI-hard, because it involves the AI perfectly modeling a world it doesnt live in (the simulation problem) or understanding relationships in concepts without making jumps (obeying logic).

It’s not clear to me that the bottleneck for simulating the universe is intelligence . Our measurement capabilities are limited, and our computational power is limited. When I say our measurement capabilities are limited, I mean that there are subsystems which are fundamentally chaotic. Small measurement inaccuracies will lead to large prediction inaccuracies, which in my view has little to do with intelligence.

Clearly, detailed long-term simulations of chaotic systems is impossible with any (non-quantum) turing machine with limited parallelism.

I think the argument in the article is rather the other way around, ie that intelligence DEPENDS on a kind of simulation to develop proper intelligence. For instance AlphaZero was able to "solve" chess, go, etc, because it could simulate the game state perfectly.

And, the "hard" part comes from the fact that perfect simulation is impossible (with or without AI).

For an AI that will never interact with the physical world, this makes some sense.

However, as I argued in my other response, while some kind of World Model (ref LeCun) may indeed be necessary for AGI, it doesn't need to be perfect. Predictive about essential aspects (those relevant for decisions and actions) is enough.

https://syncedreview.com/2017/02/25/yann-le-cun-predicting-u...

Re: What Is AGI-Hard

#29

Post by lordnacho »

robgibbons wrote: post #19
logicallee wrote: post #13

I hereby endorse ChatGPT as a limited form of AGI. It can complete a variety of novel cognitive tasks (example: judging texts and proposals) often enough that it meets my personal requirements for this. My opinion is that the definitions which others believe disqualify ChatGPT from being considered AGI are not relevant, since they focus on things it fails at rather than what it can do. For example, when it tests as w…

Ask ChatGPT if it's an AGI, and it will tell you plainly that it is not, and why. Here's GPT's response: No, ChatGPT is not an AGI (Artificial General Intelligence). It is a large language model that has been trained to predict the next word in a sequence of text given the previous words. While ChatGPT has been trained on a diverse range of text data and is able to generate human-like text, it is not capable of under…

It's just being modest. A fair few humans do nothing else than generate text based on other text.

Re: What Is AGI-Hard

#30

Post by beefield »

I occasionally wonder what would be examples of types of problems/questions that would be easy for post-singular superhuman AGI but very hard/impossible for humans? Not in sense how fast the problem is solved, but in the sense that the question/answer is even understood?

So far I have come up with high-dimensional spatial awareness (not sure how well we could get what happens in 1000 dimensions).

Post reply on HN