Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ARC Prize – a $1M+ competition towards open AGI progress

arcprize.org

Post reply on HN
Print view View original

191–200 of 351 posts

Re: ARC Prize – a $1M+ competition towards open AGI progress

#191

Post by dimask »

lacker wrote: post #6

I really like the idea of ARC. But to me the problems seem like they require a lot of spatial world knowledge, more than they require abstract reasoning. Shapes overlapping each other, containing each other, slicing up and reassembling pieces, denoising regular geometric shapes, you can call them "core knowledge" but to me it seems like they are more like "things that are intuitive to human visual processing". Would…

> Would an intelligent but blind human be able to solve these problems?

Blind people can have spatial reasoning just fine. Visual =/= spatial [0]. Now, one would have to adapt the colour-based tasks to something that would be more meaningful for a blind person, I guess.

[0] https://hal.science/hal-03373840/document

Re: ARC Prize – a $1M+ competition towards open AGI progress

#192

Post by dimask »

parentheses wrote: post #130
andoando wrote: post #92

Earlier quoted context omitted.

I would argue that spatial reasoning encompasses all reasoning. All the things you mentioned have a direct analogue to abstract models and logic we employ and are engrained deeply into language. For example, shapes containing eachother: There are two countries both which lay claim to the same territory. There is a set X that contains Y and there is a set Z that contains Y. In the case that the common overlap is 3D an…

Spatial reasoning is easily isomorphic to many kinds of reasoning - just not all of them. Spatial reasoning in this case also limits the AI to 2 dimensions. I concede that with more dimensions, there will be more isomorphisms. Is there a number of dimensions that captures all reasoning? I don't know..

Claims of isomorphisms are really strong claims to not be backed up with some kind of evidence.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#193

Post by jpc0 »

clipsy wrote: post #121
VirusNewbie wrote: post #107

Earlier quoted context omitted.

In another comment I replied that 3D high fidelity images do end up being thousands of training samples, so the answer is yes.

I'm deeply skeptical that training AI on (effectively) thousands of images of one horse will perform very well at training to recognize horses in general.

I'll double down with you on this.

Then train the AI using a binaural video of a thoroughbred and see if it can distinguish a draft horse and a quarter horse as horse...

Re: ARC Prize – a $1M+ competition towards open AGI progress

#194

Post by dimask »

PheonixPharts wrote: post #77
com2kid wrote: post #53

Earlier quoted context omitted.

> humans do not need 10,000 examples to tell the difference between cats and dogs, I swear, not enough people have kids. Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?" And there is a dev…

> Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. I have kids so I'm presuming I'm allowed to have an opinion here. This is ignoring the fact that babies are not just learning labels, they're learning the whole of language, motion planning, sensory processing, etc. Once they have the basics down concept acquisition time shrinks rapidly and kids can easily learn their new fa…

> How many homework questions did your entire calc 1 class have? I'm guessing less than 100 and (hopefully) you successfully learned differential calculus.

Not just that: people learn mathematics mainly by _thinking over and solving problems_, not by memorising solutions to problems. During my mathematics education I had to practice solving a lot of problems dissimilar what I had seen before. Even in the theory part, a lot of it was actually about filling in details in proofs and arguments, and reformulating challenging steps (by words or drawings). My notes on top of a mathematical textbook are much more than the text itself.

People think that knowledge lies in the texts themselves; it does not, it lies in what these texts relate to and the processes that they are part of, a lot of which are out in the real world and in our interactions. The original article is spot on that there is no AGI pathway in the current research direction. But there are huge incentives for ignoring this.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#195

Post by _flux »

p1esk wrote: post #95
PheonixPharts wrote: post #77

Earlier quoted context omitted.

> Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. I have kids so I'm presuming I'm allowed to have an opinion here. This is ignoring the fact that babies are not just learning labels, they're learning the whole of language, motion planning, sensory processing, etc. Once they have the basics down concept acquisition time shrinks rapidly and kids can easily learn their new fa…

illusion that these models are learning like children do, when in reality they are not even entirely capable of learning novel concepts Now imagine how much would your kid learn if the only input he ever received was a sequence of words?

Are you saying it's not fair for LLMs, because of the way they are taught is different?

The difference is that we don't know better methods for them, but we do know of better methods for people.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#196

Post by _flux »

VirusNewbie wrote: post #107
clipsy wrote: post #93

Earlier quoted context omitted.

Did he require 9,999 more examples of horses before learning the difference?

In another comment I replied that 3D high fidelity images do end up being thousands of training samples, so the answer is yes.

Are you suggesting that if a group of kids were given a book of zoo animals before going to the zoo, they would have difficulties identifing any new animals, because they only have seen one picture of each?

Re: ARC Prize – a $1M+ competition towards open AGI progress

#197

Post by zurfer »

geor9e wrote: post #147

I found them all extremely easy for a while, but then I couldn't figure out the rules of this one at all: e6de6e8f https://i.imgur.com/ExMFGqU.png

yeah it's off somehow. rule 1: start at the green dot?

rule 2: glue the left outer piece to the bottom

rule 3: overlap every now and then :D

rule 4: invert some of the pieces every now and then

Re: ARC Prize – a $1M+ competition towards open AGI progress

#198

Post by slicerdicer1 »

Animats wrote: post #180

> the only eval which measures AGI. That's a stretch. This is a problem at which LLMs are bad. That does not imply it's a good measure of artificial general intelligence. After working a few of the problems, I was wondering how many different transformation rules the problem generator has. Not very many, it seems. So the problem breaks down into extracting the set of transformation rules from the data, then applying…

AGI is not when the AI is good at some particular thing, AGI is when we have nothing left at which the AI is bad at (compared to humans).

Re: ARC Prize – a $1M+ competition towards open AGI progress

#199

Post by neoneye2 »

mark_l_watson wrote: post #32
pmayrgundter wrote: post #14

This claim that these tests are easy for humans seems dubious, and so I went looking a bit. Melanie Mitchell chimed in on Chollet's thread and posted their related test [ConceptARC]. In it they question the ease of Chollet's tests: "One limitation on ARC’s usefulness for AI research is that it might be too challenging. Many of the tasks in Chollet’s corpus are difficult even for humans, and the corpus as a whole migh…

I saw Melanie’s post and I am intrigued by an easier AGI suite. I would like some experimenting done by individuals like myself snd smaller organizations.

Melanie is coauthor/supervisor of ConceptARC, that can be tried here: https://neoneye.github.io/arc/?dataset=ConceptARC

Re: ARC Prize – a $1M+ competition towards open AGI progress

#200

Post by whyever »

dimask wrote: post #194
PheonixPharts wrote: post #77

Earlier quoted context omitted.

> Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. I have kids so I'm presuming I'm allowed to have an opinion here. This is ignoring the fact that babies are not just learning labels, they're learning the whole of language, motion planning, sensory processing, etc. Once they have the basics down concept acquisition time shrinks rapidly and kids can easily learn their new fa…

> How many homework questions did your entire calc 1 class have? I'm guessing less than 100 and (hopefully) you successfully learned differential calculus. Not just that: people learn mathematics mainly by _thinking over and solving problems_, not by memorising solutions to problems. During my mathematics education I had to practice solving a lot of problems dissimilar what I had seen before. Even in the theory part,…

I think there is a component of memorizing solutions. For example, for mathematical proofs there is a set of standard "tricks" that you should have memorized.
Post reply on HN