Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ARC Prize – a $1M+ competition towards open AGI progress

arcprize.org

Post reply on HN
Print view View original

111–120 of 351 posts

Re: ARC Prize – a $1M+ competition towards open AGI progress

#112

Post by lostmsu »

resource0x wrote: post #98
com2kid wrote: post #53

Earlier quoted context omitted.

> humans do not need 10,000 examples to tell the difference between cats and dogs, I swear, not enough people have kids. Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?" And there is a dev…

I haven't seen 1000 cats in my entire life. I'm sure I learned how to tell a dog from a cat after being exposed to just a single instance of each.

I'm sure you saw over 1B images of cats though, assuming 24 images per second from vision.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#114

Post by allanrbo »

salamo wrote: post #21

This is super cool. I share Francois' intuition that the presently data-hungry learning paradigm is not only not generalizable but unsustainable: humans do not need 10,000 examples to tell the difference between cats and dogs, and the main reason computers can today is because we have millions of examples. As a result, it may be hard to transfer knowledge to more esoteric domains where data is expensive, rare, and ha…

If a human eye works at say 10 fps, then 8 minutes with a cat is about 10k images :-D

Re: ARC Prize – a $1M+ competition towards open AGI progress

#115

Post by mewpmewp2 »

amelius wrote: post #39
VirusNewbie wrote: post #36

Earlier quoted context omitted.

>: humans do not need 10,000 examples to tell the difference between cats and dogs well, maybe. We view things in three dimensions at high fidelity: viewing a single dog or cat actually ends up being thousands of training samples, no?

Yes, but we do not call a couch in a leopard print a leopard. Because we understand that the print is secondary to the function.

But we have a lot more sensory input and context to verify all of that.

If you kept training LLMs with all that data, it would be interesting to see what the results would be.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#116

Post by ilaksh »

Maybe this is a dumb question, but in order to pass, is the program or model only allowed to use the 400 training tasks? I assume it is allowed to train on other data, just not the actual public test tasks?

Things like SORA and gpt-4o that use [diffusion transformers etc. or whatever the SOTA is for multimodal large models] seem to be able to generalize quite well. Have these latest models been tested against this task?

Re: ARC Prize – a $1M+ competition towards open AGI progress

#118

Post by mewpmewp2 »

HarHarVeryFunny wrote: post #40
lacker wrote: post #6

I really like the idea of ARC. But to me the problems seem like they require a lot of spatial world knowledge, more than they require abstract reasoning. Shapes overlapping each other, containing each other, slicing up and reassembling pieces, denoising regular geometric shapes, you can call them "core knowledge" but to me it seems like they are more like "things that are intuitive to human visual processing". Would…

I just did the first 5 of the "public eval set" without having looked at the "public training set", and found them easy enough. If we're defining AGI as at least human level, then the AGI should also be able to do these without seeing any more examples. I don't think there's any rules about what knowledge/experience you build into your solution.

AGI should obviously be able to do them. But AI being able to do those 100 percent wouldn't be evidence of AGI however. It is a very narrow domain.
Post reply on HN