Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ARC Prize – a $1M+ competition towards open AGI progress

arcprize.org

Post reply on HN
Print view View original

181–190 of 351 posts

Re: ARC Prize – a $1M+ competition towards open AGI progress

#181

Post by 8organicbits »

visarga wrote: post #171
8organicbits wrote: post #160

Earlier quoted context omitted.

> The system made of humans+LLMs is an AGI. Pay no attention to the man behind the curtain. This type of thinking would claim that mechanical turk is AGI, or perhaps that human+pen and paper is AGI. While they are great tools, that's not how I'd characterize them.

> Pay no attention to the man behind the curtain. I could say the same for us, pay no attention to the other humans who are behind the curtain. Humans in isolation are dumb, limited, and can get nowhere with understanding the world. Intelligence is mostly nurture over nature, the collective activity of society nurtures intelligence. It's smart because it learns from many diverse experiences and has a common language…

The A in AGI stands for artificial, so a human+LLM system would not qualify as it has a natural, human component. That doesn't mean it's not an interesting topic, or that it won't help humans discover our world better, it's just the wrong label. Remove the human and you'd just have LLMs talking nonsense at each other. It's not surprising that you get an intelligent system when you include natural intelligence.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#182

Post by mischa_u »

ildon wrote: post #138
levocardia wrote: post #54

François Chollet's original paper is incredibly insightful and I'm consistently shocked more people don't talk about it. Some parts are quite technical but at a high level it is the best answer to "what do we mean by general intelligence?" that I've yet seen. Defining intelligence as an efficiency of learning , after accounting for any explicit or implicit priors about the world, makes it much easier to understand wh…

Do you remember the title/where to find it?

"On the Measure of Intelligence" https://arxiv.org/abs/1911.01547

Re: ARC Prize – a $1M+ competition towards open AGI progress

#183

Post by lucianbr »

bubblyworld wrote: post #178
lucianbr wrote: post #162

Earlier quoted context omitted.

That makes no sense at all. Any problem is initially only solvable by humans, until some technology is developed to solve it. Calculating a logarithm was at some point only doable by humans, and then digital computers came along. This would be in your view evidence that digital computers are AGI!? As in, an 8086 with some math code is AGI. We've had it for decades now, only nobody noticed :)

It's just Bayes theorem - there are basically two variables that control how strong the evidence is: * How likely you think AGI is in general. * How solvable you think the problem is, independently of what's solving it. In the cases you've brought up that latter probability is very high, which means that they are extremely weak evidence that computers are AGI. So we agree! In this case the latter probability seems to…

We don't agree. You're now saying anything is evidence of anything, which just makes the word "evidence" meaningless.

In real life, when people say "A is evidence of B" they mean strong evidence, or even overwhelming evidence. You just backpedalled by redefining evidence to mean anything and nothing, so you can salvage an obviously false claim.

Nobody in the real world says "rain is evidence of aliens" with the implicit assumption that it's just extremely weak evidence. The way English is used by people makes that sentence simply false, as is yours that anything previously not solved is evidence of AGI.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#184

Post by bubblyworld »

lucianbr wrote: post #183
bubblyworld wrote: post #178

Earlier quoted context omitted.

It's just Bayes theorem - there are basically two variables that control how strong the evidence is: * How likely you think AGI is in general. * How solvable you think the problem is, independently of what's solving it. In the cases you've brought up that latter probability is very high, which means that they are extremely weak evidence that computers are AGI. So we agree! In this case the latter probability seems to…

We don't agree. You're now saying anything is evidence of anything, which just makes the word "evidence" meaningless. In real life, when people say "A is evidence of B" they mean strong evidence, or even overwhelming evidence. You just backpedalled by redefining evidence to mean anything and nothing, so you can salvage an obviously false claim. Nobody in the real world says "rain is evidence of aliens" with the impli…

We're talking about a specific problem here - the competition in the OP. Not aliens in the rain.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#185

Post by educasean »

bubblyworld wrote: post #179
educasean wrote: post #170

Earlier quoted context omitted.

This flies directly in the face of technologies such as Deep Blue and AlphaGo. They excel in tiny domains previously thought to be the pinnacle of intelligence, and now they dominate humans. Are they AGI in your definition?

See my response to the other commenter. In these cases as well I would conclude it's very weak evidence of AGI, so I don't think we disagree. Edit: I think maybe the disagreement here is about the nature of evidence. I think there can be evidence that something is AGI even if it isn't, in fact, AGI. You seem to believe that if there's any evidence that something is AGI, it must be AGI, I think?

I personally don't find this line of rhetoric useful or relevant. Let's agree to disagree.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#187

Post by nphard85 »

AIorNot wrote: post #72
salamo wrote: post #21

This is super cool. I share Francois' intuition that the presently data-hungry learning paradigm is not only not generalizable but unsustainable: humans do not need 10,000 examples to tell the difference between cats and dogs, and the main reason computers can today is because we have millions of examples. As a result, it may be hard to transfer knowledge to more esoteric domains where data is expensive, rare, and ha…

There’s a great episode from Darkwish Patels podcast discussing this today https://youtu.be/UakqL6Pj9xo?si=iDH6iSNyz1Net8j7

Dwarkesh*

Re: ARC Prize – a $1M+ competition towards open AGI progress

#188

Post by neoneye2 »

parentheses wrote: post #127
neoneye2 wrote: post #37

I'm Simon Strandgaard and I participated in ARCathon 2022 (solved 3 tasks) and ARCathon 2023 (solved 8 tasks). I'm collecting data for how humans are solving ARC tasks, and so far collected 4100 interaction histories ( https://github.com/neoneye/ARC-Interactive-History-Dataset ). Besides ARC-AGI, there are other ARC like datasets, these can be tried in my editor ( https://neoneye.github.io/arc/ ). I have made some vi…

The UX of your solution entry is _way_ better than the ARC site itself.

That warms my heart. Thank you.

The short story. I needed something that could render thumbnails of tasks, so I could visual debug what was going on in my solver. However I have never gotten around to make the visual inspection tool. After having the thumbnail renderer, mid january 2024, then it eventually turned into what it is now.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#189

Post by bubblyworld »

educasean wrote: post #185
bubblyworld wrote: post #179

Earlier quoted context omitted.

See my response to the other commenter. In these cases as well I would conclude it's very weak evidence of AGI, so I don't think we disagree. Edit: I think maybe the disagreement here is about the nature of evidence. I think there can be evidence that something is AGI even if it isn't, in fact, AGI. You seem to believe that if there's any evidence that something is AGI, it must be AGI, I think?

I personally don't find this line of rhetoric useful or relevant. Let's agree to disagree.

Okay, that's fair. But to be clear - this is a theorem of probability theory, not rhetoric.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#190

Post by isaacfrond »

visarga wrote: post #166

Why doesn't Chollet just make a challenge that reads like "Solve cancer", surely there is no solution in any books. If the AI is really AGI it could presumably do it. But not even the whole human society can do it in one go, it's a slow iterative process of ideation and validation. Even though this is a life and death matter, we can't simply solve it. This is why AGI won't look like we expect, it will be a continuati…

Exactly. Because I'm sure that the minute some program aces the ARC test, we'll all say, ahhh, but that, that wasn't real intelligence. And they would be right, if you solve the ARC test, you can do ARC like puzzles. Say something about your reasoning abilities I guess, but it surely does not say you have super human intelligence.
Post reply on HN