Is this open as in "OpenAI" or what are we doing here?
:)
ARC Prize – a $1M+ competition towards open AGI progress
91–100 of 351 posts
Re: ARC Prize – a $1M+ competition towards open AGI progress
#92I really like the idea of ARC. But to me the problems seem like they require a lot of spatial world knowledge, more than they require abstract reasoning. Shapes overlapping each other, containing each other, slicing up and reassembling pieces, denoising regular geometric shapes, you can call them "core knowledge" but to me it seems like they are more like "things that are intuitive to human visual processing". Would…
I would argue that spatial reasoning encompasses all reasoning. All the things you mentioned have a direct analogue to abstract models and logic we employ and are engrained deeply into language. For example, shapes containing eachother:
There are two countries both which lay claim to the same territory. There is a set X that contains Y and there is a set Z that contains Y. In the case that the common overlap is 3D and one in on top of the other, we can extend this to there is a set X that contains -Y and a set Z that contains Y, and just as you can only see one on top and not both depending on where you stand, we can apply the same property here and say set X and Z cannot both exist, and therefore if set X is on then -Y and if set Z then Y.
If you pay attention to the language you use youll start to realize how much of it uses spatial relationships to describe completely abstract things. For example, one can speak of disintigrating hegonomic economies. i.e turning things built on top of eachother into nothing, to where it came
We are after all, reasoning about things which happen in time and space.
And spatial != visual. Even if you were blind youd have to reason spatially, because again any set of facts are facts in space-time. What does it take to understand history? People in space, living at various distances from each other, producing goods from various locations of the earth using physical processes, and physically exchanging them. To understand battles you have to understand how armies are arranged physically, how moving supplies works, weather conditions, how weapons and their physical forms affect what they can physically do, etc.
Hell LLMs, the largest advancement we had in artificial intelligence do what exactly? Encode tokens into multi dimensional space.
Re: ARC Prize – a $1M+ competition towards open AGI progress
#93Earlier quoted context omitted.
Yes, but we do not call a couch in a leopard print a leopard. Because we understand that the print is secondary to the function.
I'm not sure it's as simple as you say. The first time my very young son saw a horse, he made the ASL sign for 'dog'. He had only ever seen cats and dogs in his life previous to that.
Did he require 9,999 more examples of horses before learning the difference?
Re: ARC Prize – a $1M+ competition towards open AGI progress
#94Interesting. It seems most of these task target a very specific part of the brain that recognizes visual patterns. But that alone is cannot possibly be the only definition of intelligence. What about Theory of Mind which talks about the problem of multiple agents in the real world acting together? Like driving a car cannot be done right now without oodles of data or any robot - human problem that requires the robot t…
I made another comment here saying the same thing, but visual patterns and other patterns are nonetheless spatial patterns. Audio, understanding music, or speech, rtc are things that are happening spatially, and they can just as easily be mapped as visual problems. This makes a lot of sense, as after all our senses are telling us what's happening in space-time.
Take for example a simple audiotory pattern like "clap clap clap". This has a very trival mapping as visual like so:
x x x
- - -
house house house
whereas anyone would agree the sound of three equally spaced claps would not be analogous to say:
aa b b b
-- --- -- -- ---
This ability to relate or equate two entirely different senses should clue you in that there is a deeper framework at play
Re: ARC Prize – a $1M+ competition towards open AGI progress
#95Earlier quoted context omitted.
> humans do not need 10,000 examples to tell the difference between cats and dogs, I swear, not enough people have kids. Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?" And there is a dev…
> Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. I have kids so I'm presuming I'm allowed to have an opinion here. This is ignoring the fact that babies are not just learning labels, they're learning the whole of language, motion planning, sensory processing, etc. Once they have the basics down concept acquisition time shrinks rapidly and kids can easily learn their new fa…
illusion that these models are learning like children do, when in reality they are not even entirely capable of learning novel concepts
Now imagine how much would your kid learn if the only input he ever received was a sequence of words?
Re: ARC Prize – a $1M+ competition towards open AGI progress
#96Earlier quoted context omitted.
“Would an intelligent but blind human be able to solve these problems?” This is the wrong way to think about it IMO. Spatial relationships are just another type of logical relationship and we should expect AGI to be able to analyze relationships and generate algorithms on the fly to solve problems. Just because humans can be biased in various ways doesn’t mean these biases are inherent to all intelligences.
> Spatial relationships are just another type of logical relationship and we should expect AGI to be able to analyze relationships and generate algorithms on the fly to solve problems. Not really. By that reasoning, 5-dimensional spatial reasoning is "just another type of logical relationship" and yet humans mostly can't do that at all. It's clear that we have incredibly specialized capabilities for dealing with two-…
Literally every single thing you reason about is something happening in space-time.
Re: ARC Prize – a $1M+ competition towards open AGI progress
#97What is the fundamental difference between ARC and a standard IQ test? On the surface they seem similar in that they both involve deducing and generalizing visual patterns. Is there something special about these questions that makes them resistant to memorization? Or is it more just the fact that there are 100 secret tasks?
I’ve always found this kind of puzzle infuriating because it’s way underspecified. You’re not trying to find a pattern, you’re trying to guess what pattern the test writer would expect.
countless of problems in the world are underspecified in exactly this way, that is effectively what common sense reasoning is. Or what Charles Sanders Peirce called abductive reasoning, making a sensible best guess under conditions of uncertainty.
Re: ARC Prize – a $1M+ competition towards open AGI progress
#98This is super cool. I share Francois' intuition that the presently data-hungry learning paradigm is not only not generalizable but unsustainable: humans do not need 10,000 examples to tell the difference between cats and dogs, and the main reason computers can today is because we have millions of examples. As a result, it may be hard to transfer knowledge to more esoteric domains where data is expensive, rare, and ha…
> humans do not need 10,000 examples to tell the difference between cats and dogs, I swear, not enough people have kids. Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?" And there is a dev…
I haven't seen 1000 cats in my entire life. I'm sure I learned how to tell a dog from a cat after being exposed to just a single instance of each.
Re: ARC Prize – a $1M+ competition towards open AGI progress
#99Interesting. It seems most of these task target a very specific part of the brain that recognizes visual patterns. But that alone is cannot possibly be the only definition of intelligence. What about Theory of Mind which talks about the problem of multiple agents in the real world acting together? Like driving a car cannot be done right now without oodles of data or any robot - human problem that requires the robot t…
I made another comment here saying the same thing, but visual patterns and other patterns are nonetheless spatial patterns. Audio, understanding music, or speech, rtc are things that are happening spatially, and they can just as easily be mapped as visual problems. This makes a lot of sense, as after all our senses are telling us what's happening in space-time. Take for example a simple audiotory pattern like "clap c…
It's not just mapping events in space and time, it's also bringing in appropriate context and expectation of future (goals, intentions) into the present, other people's mental models into our prediction.
I am not sure how abstract thinking for generalized pattern matching make it AGI to solve these kind of problems(not that they are not amazing abilities). If these ToM problems are reducible to these tasks posted by the OP then there would need to be some kind of theorem proving business to convert between the two sets of problems efficiently no?
Re: ARC Prize – a $1M+ competition towards open AGI progress
#100This is super cool. I share Francois' intuition that the presently data-hungry learning paradigm is not only not generalizable but unsustainable: humans do not need 10,000 examples to tell the difference between cats and dogs, and the main reason computers can today is because we have millions of examples. As a result, it may be hard to transfer knowledge to more esoteric domains where data is expensive, rare, and ha…
> humans do not need 10,000 examples to tell the difference between cats and dogs
The optimization process that trained the human brain is called evolution, and it took a lot more than 10,000 examples to produce a system that can differentiate cats vs dogs.
Put differently, an LLM is pre-trained with very light priors, starting almost from scratch, whereas a human brain is pre-loaded with extremely strong priors.