Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ARC Prize – a $1M+ competition towards open AGI progress

arcprize.org

Post reply on HN
Print view View original

301–310 of 351 posts

Re: ARC Prize – a $1M+ competition towards open AGI progress

#301

Post by HarHarVeryFunny »

bubblyworld wrote: post #141
mewpmewp2 wrote: post #118

Earlier quoted context omitted.

AGI should obviously be able to do them. But AI being able to do those 100 percent wouldn't be evidence of AGI however. It is a very narrow domain.

Why not? If the only thing that can solve problem X is AGI (e.g. humans), and something else comes along that solves it, then rationally that should be evidence that the something else is AGI right? Unless you have strong prior beliefs (like "computers can't be AGI") or something else that's problem specific ("these problems can be solved by these techniques which don't count as AGI"). So I guess that's my real quest…

A implies B, doesn't mean than B implies A. That's a basic logical fallacy.

AGI can add 1+1 correctly, but an ability to do that is not a test for AGI.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#302

Post by laurent_du »

TheDudeMan wrote: post #68

Where did the money come from? How about put it toward alignment research instead of accelerating capabilities?

It comes from Knoop and Chollet's pockets. You are welcome to spend your own money to further whatever matters most to you.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#303

Post by ein0p »

Tepix wrote: post #206
ein0p wrote: post #172

Earlier quoted context omitted.

Not to mention that babies receive petabytes of visual input to go with other stimuli. It’s up for debate how sample efficient humans actually are in the first few years of their lives.

Hardly. Visual acuity is quite low (limited to a tiny area of the FoV), your brain is filling in all the blanks for you.

Even at that resolution (about 0.35 MP per eye for just the fovea before any processing) napkin math suggests 7.3T per day. Over 5 years you get about 13PB if my math is right, assuming 16 waking hours per day.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#304

Post by dwaltrip »

ben_w wrote: post #231
TeMPOraL wrote: post #214

Earlier quoted context omitted.

I think they're saying that it's silly to claim humans learn with less data than LLMs, when humans are ingesting a continuous video, audio, olfactory and tactile data stream for 16+ hours a day, every day. It takes at least 4 years for a human children to be in any way comparable in performance to GPT-4 on any task both of them could be tested on; do people really believe GPT-4 was trained with more data than a 4 yea…

> do people really believe GPT-4 was trained with more data than a 4 year old? I think it was; the guesstimate I've seen is GPT-4 was trained on 13e12 tokens, that over 4 years is 8.9e9/day or about 1e5/s. Then it's a question of how many bits per token — my expectation is 100k/s is more than the number of token-equivalents we experience, even though it's much less than the bitrate even of just our ears let alone our…

Interesting analysis, makes sense. I wonder how we should account for the “pre-built” knowledge that is transferred to a newborn genetically and from the environment at conception and during gestation. Of course things like epi-genetics also come into play.

The analogies get a little blurry here, but perhaps we can draw a distinction between information that an infant gets from their higher-level senses (e.g. sight, smell, touch, etc) versus any lower-level biological processes (genetics, epi-genetics, developmental processes, and so on).

The main point is that there is a fundamental difference: LLMs have very little prior knowledge [1] while humans contain an immense amount of information even before they begin learning through the senses.

We need to look at the billions of years of biological evolution, millions of years of cultural evolution, and the immense amounts of environmental factors, all which shape us before birth and before any “learning” occurs.

[1] The model architecture probably counts as hard-coded prior knowledge contained before the model begins training, but it is a ridiculously small amount of information compared to the complexity of living organisms.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#305

Post by ks2048 »

neoneye2 wrote: post #201
ks2048 wrote: post #137

This is interesting. I've been looking at the data today and made a helper to quickly view the ARC dataset: https://kts.github.io/arc-viewer/ So you can view 100 per page instead of clicking through one-by-one: https://kts.github.io/arc-viewer/page1/

Nice overview/details. Do you plan on adding more metrics? Idea for a metric: - Number of pixels that stays the same between input/output. - Histogram changes.

Thanks, yeah lots more to look into. Just getting started! Thanks for your work. Your "Awesome ARC" page looks really helpful.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#306

Post by tshadley »

mkl wrote: post #238

I did https://arcprize.org/play?task=05a7bcf2 correctly, but one of the examples doesn't match the rule I used. Are the examples supposed to contain mistakes/noise? Did I find a bug? Did I get the rule wrong? Here's how I understand the rule: yellow blobs turn green then spew out yellow strips towards the blue line, and the width of the strips is the number of squares the green blobs take up along the blue line. The…

Sure looks like a typo. Contact author?

https://x.com/fchollet https://x.com/arcprize https://x.com/mikeknoop

Re: ARC Prize – a $1M+ competition towards open AGI progress

#307

Post by jolt42 »

neoneye2 wrote: post #204
jolt42 wrote: post #105

On puzzle #23 (id: 11e1fe23), I'm sure there's more than one possible valid answer from the examples given. You can't tell if the expected distance is from the gray square or from the RGB squares.

The task is here. https://neoneye.github.io/arc/edit.html?dataset=ARC&task=11e... There are many examples where the test is slightly OOD (out of distribution), so the solver will have to generalize.

Not sure what you mean. There's a viable answer that's marked incorrect. The examples should show the pattern well enough to eliminate possible wrong answers, correct?

Re: ARC Prize – a $1M+ competition towards open AGI progress

#308

Post by lopuhin »

mkl wrote: post #238

I did https://arcprize.org/play?task=05a7bcf2 correctly, but one of the examples doesn't match the rule I used. Are the examples supposed to contain mistakes/noise? Did I find a bug? Did I get the rule wrong? Here's how I understand the rule: yellow blobs turn green then spew out yellow strips towards the blue line, and the width of the strips is the number of squares the green blobs take up along the blue line. The…

yes looks like a bug in the example to me, feel free to report to https://github.com/fchollet/ARC-AGI/issues :)

Re: ARC Prize – a $1M+ competition towards open AGI progress

#309

Post by naasking »

captaincaveman wrote: post #210
allanrbo wrote: post #114

Earlier quoted context omitted.

If a human eye works at say 10 fps, then 8 minutes with a cat is about 10k images :-D

I'd say that was more like a single instance, one interaction with a thing.

One interaction that captures a multidimensional, multisensory set of perceptions. In an ML training set, say for visual recognition, this would consist at least of hundreds of images from many angles, in different poses and varied lighting.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#310

Post by michaelhodel »

neoneye2 wrote: post #75
Geee wrote: post #74

Any details on how these tests were created? I.e. which kind of program was used for generation.

I think the ARC-AGI tasks was manually drawn with an early version of fchollet's editor. Recently Michael Hodel has reverse engineered 400 of the tasks, so more tasks can be generated. Interestingly it can generate python programs that solves the tasks too. https://github.com/michaelhodel/re-arc

No, his re-arc code does not enable generating more tasks, it merely allows to generate more examples for the already existing training tasks. Also, it can't generate task-solving programs either, it's author merely also provided a solution program for each generator program to verify the validity of the generated examples.
Post reply on HN