Live data from Hacker News

ARC Prize – a $1M+ competition towards open AGI progress

arcprize.org

321–330 of 351 posts

Re: ARC Prize – a $1M+ competition towards open AGI progress

#321
Just to let you know I found your website unreadable due to:

- annoying animated background

- white text on black background

- annoying font choices

Which is unfortunate because (as I found when I used Firefox reader mode) you're discussing important and interesting stuff.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#322

Why doesn't Chollet just make a challenge that reads like "Solve cancer", surely there is no solution in any books. If the AI is really AGI it could presumably do it. But not even the whole human society can do it in one go, it's a slow iterative process of ideation and validation. Even though this is a life and death matter, we can't simply solve it. This is why AGI won't look like we expect, it will be a continuati…

> Why doesn't Chollet just make a challenge that reads like "Solve cancer", surely there is no solution in any books.

Why doesn't a baby just run a marathon before it learns to walk? Because you've got to learn to walk before you can run.

> But not even the whole human society can do it in one go, it's a slow iterative process of ideation and validation.

So you break it down into little steps, which is what is being done here.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#323
post #163

ARC is a noble endeavour but mistakes visual/spatial reasoning for reasoning and thus fails.

No, I don't think it does. I think that the ideas in a system that could solve this type of problem would be highly generalisable to other tasks.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#324
The website gives an example:

    {
      "train": [
        {"input": [[1, 0], [0, 0]], "output": [[1, 1], [1, 1]]},
        {"input": [[0, 0], [4, 0]], "output": [[4, 4], [4, 4]]},
        {"input": [[0, 0], [6, 0]], "output": [[6, 6], [6, 6]]}
      ],
      "test": [
        {"input": [[0, 0], [0, 8]], "output": [[8, 8], [8, 8]]}
      ]
    }
But why restrict yourself to JSON that codes for 2-d coloured grids? Why not also allow:

    {
      "train": [
        {"input": [[1, 0], [0, 0]], "output": 1},
        {"input": [[0, 0], [4, 0]], "output": 4},
        {"input": [[0, 0], [6, 0]], "output": 6}
      ]
    }
Where the rule might be to output the biggest number in the input, or add them up (and the solver has to work out which).

Re: ARC Prize – a $1M+ competition towards open AGI progress

#325
post #53
post #21

This is super cool. I share Francois' intuition that the presently data-hungry learning paradigm is not only not generalizable but unsustainable: humans do not need 10,000 examples to tell the difference between cats and dogs, and the main reason computers can today is because we have millions of examples. As a result, it may be hard to transfer knowledge to more esoteric domains where data is expensive, rare, and ha…

> humans do not need 10,000 examples to tell the difference between cats and dogs, I swear, not enough people have kids. Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands. One thing kids do is they'll ask for confirmation of their guess. You'll be reading a book you've read 50 times before and the kid will stop you, point at a dog in the book, and ask "dog?" And there is a dev…

> Now, is it 10k examples? No, but I think it was on the order of hundreds, if not thousands.

If I was presented with 10 pictures of 2 species I'm unfamiliar with, about as different as cats and dogs, I expect I would be able to classify further images as either, reasonably accurately.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#326

What is the fastest way to get up to speed with techniques that led to the current SOTA?

Check out the SOTA resources on the guide https://arcprize.org/guide Happy to answer any questions you have along the way (I'm helping run ARC Prize)

Appreciate you and the team for putting this together, it's a lot of fun just brainstorming potential techniques

Re: ARC Prize – a $1M+ competition towards open AGI progress

#328
post #307

Earlier quoted context omitted.

The task is here. https://neoneye.github.io/arc/edit.html?dataset=ARC&task=11e... There are many examples where the test is slightly OOD (out of distribution), so the solver will have to generalize.

Not sure what you mean. There's a viable answer that's marked incorrect. The examples should show the pattern well enough to eliminate possible wrong answers, correct?

IRL problems are often underspecified, and despite minor mistakes, humans manages to solve the puzzle.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#329

Earlier quoted context omitted.

I started by understanding. I could multiply by repeat addition (each addition counted one at a time with the aid of fingers) before I had the 10x10 addition table memorized. I learned university level calculus before I had more than half of the 10x10 multiplication table memorized, and even that was from daily use, not from deliberate memorization. There wasn't a day in my life where I could recite the full table. M…

> I started by understanding. I could multiply by repeat addition How did you learn what the symbols for numbers mean and how addition works? Did you literally just see "1 + 3 = 4" one day and intuit the meaning of all of those symbols? Was it entirely obvious to you from the get-go that "addition" was the same as counting using your fingers which was also the same as counting apples which was also the same as these…

There's a difference between memorizing meanings of words (addition is same as counting this and then the other thing, "3" means three things) and memorizing methods (table of single digit addition/multiplication to do them faster in your head). You were arguing the second, I'm a counterexample. I agree about the first, everyone learns language by memorization (some rote, some by use), but language is not math.

Re: ARC Prize – a $1M+ competition towards open AGI progress

#330

Earlier quoted context omitted.

Why not? If the only thing that can solve problem X is AGI (e.g. humans), and something else comes along that solves it, then rationally that should be evidence that the something else is AGI right? Unless you have strong prior beliefs (like "computers can't be AGI") or something else that's problem specific ("these problems can be solved by these techniques which don't count as AGI"). So I guess that's my real quest…

A implies B, doesn't mean than B implies A. That's a basic logical fallacy. AGI can add 1+1 correctly, but an ability to do that is not a test for AGI.

This is not what I'm saying. Consider the following statement:

"Absence of evidence is evidence of absence."

Presumably you would call this a simple logical fallacy for the same reason, but a little reflection would show that in many cases such a statement is true! It depends on context, in this case your estimate of how well your search covered the possible search space.

Evidence is a continuous variable - things can be weak evidence, strong evidence... There's a whole spectrum. I just take issue with statements like "X is zero evidence of Y" because often you can do a lot better than that with the information at hand.

Post reply on HN