Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

François Chollet: The Arc Prize and How We Get to AGI [video]

youtube.com

Post reply on HN
Print view View original

21–30 of 230 posts

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#21

Post by roenxi »

By both definitions of intelligence in the presentation we should be saying "how we got to AGI" in the past tense. We're already there. AI's can deal with situations they weren't prepared for in any sense that a human can. They might not do well, but they'll have a crack at it. We can trivially build systems that collect data and do a bit more offline training if that is what someone wants to see, but there doesn't really seem to be a commercial need for that right now. Similarly, AIs can whip most humans at most domains that require intelligence.

I think the debate hqas been flat-footed by the speed all this happened. We're not talking AGI any more, we're talking about how to build superintelligences hitherto unseen in nature.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#23

Post by cubefox »

roenxi wrote: post #21

By both definitions of intelligence in the presentation we should be saying "how we got to AGI" in the past tense. We're already there. AI's can deal with situations they weren't prepared for in any sense that a human can. They might not do well, but they'll have a crack at it. We can trivially build systems that collect data and do a bit more offline training if that is what someone wants to see, but there doesn't r…

Well, there is also robotics, active inference, online learning, etc. Things animals can do well.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#24

Post by ben_w »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

You're not alone in this; I expect us to have not yet enumerated all the things that we ourselves mean by "intelligence".

But conversely, not passing this test is a proof of not being as general as a human's intelligence.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#25

Post by OtomotO »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

You're not alone in this, no.

My definition of AGI is the one I was brought up with, not an ever moving goal post (to the "easier" side).

And no, I also don't buy that we are just stochastic parrots.

But whatever. I've seen many hypes and if I don't die and the world doesn't go to shit, I'll see a few more in the next couple of decades

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#26

Post by NetRunnerSu »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

To pass Arc, you need a living model with sentient abilities, not the dead frog now.

https://news.ycombinator.com/item?id=44488126

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#27

Post by blibble »

hackinthebochs wrote: post #15
zamderax wrote: post #11

Earlier quoted context omitted.

People are allowed to evolve opinions. It seems to me he believes that a combination of transformer and program synthesis are key. The big unknown at the moment is how to do program search.

Absolutely. Presumably there is some specific considerations or evidence that helped him evolve his opinion. I would be interested in seeing a writeup about it. With him having been a very public advocate against AGI, a writeup of his evolution seems appropriate and would be very edifying for a lot of people.

> Presumably there is some specific considerations or evidence that helped him evolve his opinion.

suitcases full of money?

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#28

Post by NetRunnerSu »

ben_w wrote: post #24
qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

You're not alone in this; I expect us to have not yet enumerated all the things that we ourselves mean by "intelligence". But conversely, not passing this test is a proof of not being as general as a human's intelligence.

Unfortunately, we did it. All that is left is to assemble the parts.

https://news.ycombinator.com/item?id=44488126

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#29

Post by cubefox »

hackinthebochs wrote: post #10

Has Chollet ever talked about his change of heart regarding AGI? It wasn't that long ago when he was one of the loudest voices decrying even the concept of AGI, let alone us being on the path to creating it. Now he's an advocate and has his own prize dataset? Seems rather convenient to change your tune once hundreds of billions are being thrown at AGI (not that I would blame him).

ARC-AGI was introduced in 2019:

https://arxiv.org/abs/1911.01547

GPT-3 didn't come out until 2020.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#30

Post by nxobject »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

I understand Chollet is transparent that the "branding" of the ARC-AGI-n suites is meant to be suggestive of its purpose, than substantial.

However, it does rub me the wrong way - as someone who's cynical of how branding can enable breathless AI hype by bad journalism. A hypothetical comparison would be labelling SHRDLU's (1968) performance on Block World planning tasks as "ARC-AGI-(-1)".[0]

A less loaded name like (bad strawman option) "ARC-VeryToughSymbolicReasoning" should capture how the ARC-AGI-n suite is genuinely and intrinsically very hard for current AIs, and what progress satisfactory performance on the benchmark suite would represent. Which Chollet has done, and has grounded him throughout! [1]

[0] https://en.wikipedia.org/wiki/SHRDLU [1] https://arxiv.org/abs/1911.01547

Post reply on HN