Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

François Chollet: The Arc Prize and How We Get to AGI [video]

youtube.com

Post reply on HN
Print view View original

11–20 of 230 posts

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#11

Post by zamderax »

hackinthebochs wrote: post #10

Has Chollet ever talked about his change of heart regarding AGI? It wasn't that long ago when he was one of the loudest voices decrying even the concept of AGI, let alone us being on the path to creating it. Now he's an advocate and has his own prize dataset? Seems rather convenient to change your tune once hundreds of billions are being thrown at AGI (not that I would blame him).

People are allowed to evolve opinions. It seems to me he believes that a combination of transformer and program synthesis are key. The big unknown at the moment is how to do program search.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#13

Post by [deleted] »

hackinthebochs wrote: post #10

Has Chollet ever talked about his change of heart regarding AGI? It wasn't that long ago when he was one of the loudest voices decrying even the concept of AGI, let alone us being on the path to creating it. Now he's an advocate and has his own prize dataset? Seems rather convenient to change your tune once hundreds of billions are being thrown at AGI (not that I would blame him).

[deleted]

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#14

Post by Davidzheng »

whiplash451 wrote: post #7
qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

You're not the only one. ARC-AGI is a laudable effort, but its fundamental premise is indeed debatable: "We argue that human cognition follows strictly the same pattern as human physical capabilities: both emerged as evolutionary solutions to specific problems in specific evironments" (from page 22 of On the Measure of Intelligence) https://arxiv.org/pdf/1911.01547

But I believe that because of this "even edge" thing which people call of AI weakenesses being not necessarily same of humans, once we run out of these tests which AI is worse than humans it will actually in effect be very much superhuman. My main evidence for this is leela-zero the Go AI who struggled with ladders and some other aspects of Go play well into the superhuman regime (in go it's easier to see when it's superhuman bc you can have elos and play win-rates etc and there's less room for debates)

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#15

Post by hackinthebochs »

zamderax wrote: post #11
hackinthebochs wrote: post #10

Has Chollet ever talked about his change of heart regarding AGI? It wasn't that long ago when he was one of the loudest voices decrying even the concept of AGI, let alone us being on the path to creating it. Now he's an advocate and has his own prize dataset? Seems rather convenient to change your tune once hundreds of billions are being thrown at AGI (not that I would blame him).

People are allowed to evolve opinions. It seems to me he believes that a combination of transformer and program synthesis are key. The big unknown at the moment is how to do program search.

Absolutely. Presumably there is some specific considerations or evidence that helped him evolve his opinion. I would be interested in seeing a writeup about it. With him having been a very public advocate against AGI, a writeup of his evolution seems appropriate and would be very edifying for a lot of people.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#16

Post by loki_ikol »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

Well for most, the next steps are probably towards removing the highly deterministic and discrete characteristics of current approaches (we certainly don't think in lock steps). Those have no measures. Even the creative aspect is undermined by those characteristics.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#17

Post by kubb »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

AGI isn't defined anywhere, so it can be anything you want.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#18

Post by AnimalMuppet »

avmich wrote: post #5
qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

Roughly speaking, the job of a medical doctor is to diagnose the patient - and then, after the diagnosis is made, to apply the healing from the book, corresponding to the diagnosis. The diagnosis is pattern matching (again, roughly). It kinda suggests that a lot of "intelligent" problems are focused on pattern matching, and (relatively straightforward) application of "previous experience". So, pattern matching can br…

Pattern matching is instinct. (Or at least, instinct is a kind of pattern matching. And once you learn the patterns, pattern matching can become almost instinctual). And that's fine, for things that fit the pattern. But a human-level intelligence can also deal with problems for which there is no pattern. (I mean, not always successfully - finding a correct solution to a novel problem is difficult. But it is within the capability of at least some humans.)

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#19

Post by oldge »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

Today’s llms are fancy autocomplete but lack test time self learning or persistent drive. By contrast, an AGI would require: – A goal-generation mechanism (G) that can propose objectives without external prompts – A utility function (U) and policy π(a│s) enabling action selection and hierarchy formation over extended horizons – Stateful memory (M) + feedback integration to evaluate outcomes, revise plans, and execute real-world interventions autonomously Without G, U, π, and M operating llms remain reactive statistical predictors, not human level intelligence.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#20

Post by TheAceOfHearts »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

Getting a high score on ARC doesn't mean we have AGI and Chollet has always said as much AFAIK, it's meant to push the AI research space in a positive direction. Being able to solve ARC problems is probably a pre-requisite to AGI. It's a directional push into the fog of war, with the claim being that we should explore that area because we expect it's relevant to building AGI.
Post reply on HN