Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

François Chollet: The Arc Prize and How We Get to AGI [video]

youtube.com

Post reply on HN
Print view View original

131–140 of 230 posts

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#131

Post by ben_w »

IanCal wrote: post #105
ben_w wrote: post #95

Earlier quoted context omitted.

That's certainly a valid way of looking at their abilities at any given task — "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim". But when the question is "are they going to more important to the economy than humans?", then they have to be good at basically everything a human can do, otherwise we just see a variant of Amdahl's law in action and the…

> are they going to more important to the economy than humans?", then they have to be good at basically everything a human can do, I really don’t think that’s the case. A robot that can stack shelves faster than a human is more valuable at that job than someone who can move items and also appreciate comedy. One that can write software more reliably than person X is more valuable than them at that job even if X is wel…

> I really don’t think that’s the case. A robot that can stack shelves faster than a human is more valuable at that job than someone who can move items and also appreciate comedy.

Yes, until all the shelves are stacked and that is no longer your limiting factor.

> One that can write software more reliably than person X is more valuable than them at that job even if X is well rounded and can do cryptic crosswords and play the guitar.

Cryptic crosswords and guitar playing are already something computers can do, so they're not great examples.

Consider a different example: "computer" used to be a job title of a person who computes. A single Raspberry Pi model zero, given away for free on a magazine cover at launch, can do this faster than the entire human population combined even if we all worked at the speed of the world record holder 24/7. But that wasn't enough to replace all human labour.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#132

Post by tedy1996 »

lostphilosopher wrote: post #89
TheAceOfHearts wrote: post #20

Earlier quoted context omitted.

Getting a high score on ARC doesn't mean we have AGI and Chollet has always said as much AFAIK, it's meant to push the AI research space in a positive direction. Being able to solve ARC problems is probably a pre-requisite to AGI. It's a directional push into the fog of war, with the claim being that we should explore that area because we expect it's relevant to building AGI.

We don't really have a true test that means "if we pass this test we have AGI" but we have a variety of tests (like ARC) that we believe any true AGI would be able to pass. It's a "necessary but not sufficient" situation. Also ties directly to the challenge in defining what AGI really means. You see a lot of discussions of "moving the goal posts" around AGI, but as I see it we've never had goal posts, we've just got…

I have graduated with a degree in Software engineering and i am bilingual (Bulgarian and English). Currently AI is better than me in everything except adding big numbers or writing code in really niche topics - for example code golfing a Brainfuck interpreter or writing a Rubiks cube solver. I believe AGI has been here for at least a year now.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#133

Post by MPSimmons »

kordlessagain wrote: post #111
TheAceOfHearts wrote: post #20

Earlier quoted context omitted.

Getting a high score on ARC doesn't mean we have AGI and Chollet has always said as much AFAIK, it's meant to push the AI research space in a positive direction. Being able to solve ARC problems is probably a pre-requisite to AGI. It's a directional push into the fog of war, with the claim being that we should explore that area because we expect it's relevant to building AGI.

ARC is definitely about achieving AGI and it doesn't matter whether we "have" it or not right now. That is the goal: > where he introduced the "Abstract and Reasoning Corpus for Artificial General Intelligence" (ARC-AGI) benchmark to measure intelligence So, a high enough score is a threshold to claim AGI. And, if you use an LLM to work these types of problems, it becomes pretty clear that passing more tests indicate…

>a high enough score is a threshold to claim AGI

I'm pretty sure he said that AGI would achieve a high score, not that a high score was indicative of AGI

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#134

Post by KoolKat23 »

NetRunnerSu wrote: post #51
KoolKat23 wrote: post #32

Earlier quoted context omitted.

I'd say we're not far off. Looking at the human side, it takes a while to actually learn something. If you've recently read something it remains in your "context window". You need to dream about it, to think about, to revisit and repeat until you actually learn it and "update your internal model". We need a mechanism for continuous weight updating. Goal-generation is pretty much covered by your body constantly drip-f…

Yes, you're right, that's what we're doing. https://github.com/dmf-archive/PILF

Very interesting, thanks for the link.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#135

Post by tedy1996 »

IanCal wrote: post #105
ben_w wrote: post #95

Earlier quoted context omitted.

That's certainly a valid way of looking at their abilities at any given task — "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim". But when the question is "are they going to more important to the economy than humans?", then they have to be good at basically everything a human can do, otherwise we just see a variant of Amdahl's law in action and the…

> are they going to more important to the economy than humans?", then they have to be good at basically everything a human can do, I really don’t think that’s the case. A robot that can stack shelves faster than a human is more valuable at that job than someone who can move items and also appreciate comedy. One that can write software more reliably than person X is more valuable than them at that job even if X is wel…

AFAIK 55% of PRs written by latest GPT model get approved.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#136

Post by ipunchghosts »

visarga wrote: post #86

I think intelligence is search. Search is exploration + learning. So intelligence is not in the model or in the environment, but in their mutual dance. A river is not the banks, nor the water, but their relation. ARC is just a frozen snapshot of the banks, not the dynamic environment we have.

I agree strongly with this take but find it hard to convince others of it. Instead, people keep thinking there is a magic bullet to discover resulting in a lot of wasted resources and money.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#137

Post by fvdessen »

MPSimmons wrote: post #130
lostphilosopher wrote: post #89

Earlier quoted context omitted.

We don't really have a true test that means "if we pass this test we have AGI" but we have a variety of tests (like ARC) that we believe any true AGI would be able to pass. It's a "necessary but not sufficient" situation. Also ties directly to the challenge in defining what AGI really means. You see a lot of discussions of "moving the goal posts" around AGI, but as I see it we've never had goal posts, we've just got…

I don't think we actually even have a good definition of "This is what AGI is, and here are the stationary goal posts that, when these thresholds are met, then we will have AGI". If you judged human intelligence by our AI standards, then would humans even pass as Natural General Intelligence? Human intelligence tests are constantly changing, being invalidated, and rerolled as well. I maintain that today's modern LLMs…

Turing test is not really that meaningful anymore because you can always detect the AI by text and timing patterns rather than actual intelligence. In fact the most reliable way to test for AI is probably to ask trivia questions on various niche topics, I don't think any human has as much breath of general knowledge as current AIs.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#138

Post by fvdessen »

tedy1996 wrote: post #132
lostphilosopher wrote: post #89

Earlier quoted context omitted.

We don't really have a true test that means "if we pass this test we have AGI" but we have a variety of tests (like ARC) that we believe any true AGI would be able to pass. It's a "necessary but not sufficient" situation. Also ties directly to the challenge in defining what AGI really means. You see a lot of discussions of "moving the goal posts" around AGI, but as I see it we've never had goal posts, we've just got…

I have graduated with a degree in Software engineering and i am bilingual (Bulgarian and English). Currently AI is better than me in everything except adding big numbers or writing code in really niche topics - for example code golfing a Brainfuck interpreter or writing a Rubiks cube solver. I believe AGI has been here for at least a year now.

I suggest you to try to let the AI think through race conditions scenarios in asynchronous programs; it is not that good at these abstract reasoning tasks.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#139

Post by QuadmasterXLII »

MPSimmons wrote: post #130
lostphilosopher wrote: post #89

Earlier quoted context omitted.

We don't really have a true test that means "if we pass this test we have AGI" but we have a variety of tests (like ARC) that we believe any true AGI would be able to pass. It's a "necessary but not sufficient" situation. Also ties directly to the challenge in defining what AGI really means. You see a lot of discussions of "moving the goal posts" around AGI, but as I see it we've never had goal posts, we've just got…

I don't think we actually even have a good definition of "This is what AGI is, and here are the stationary goal posts that, when these thresholds are met, then we will have AGI". If you judged human intelligence by our AI standards, then would humans even pass as Natural General Intelligence? Human intelligence tests are constantly changing, being invalidated, and rerolled as well. I maintain that today's modern LLMs…

The current definition and goal of AGI is “Artificial intelligence good enough to replace every employee for cheaper” and much of the difficulty people have in defining it is cognitive dissonance about the goal.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#140

Post by ta8645 »

wat10000 wrote: post #84
maaaaattttt wrote: post #56

Earlier quoted context omitted.

Given your premise (which I agree with) I think the issue in general comes from the lack of a good, broadly accepted definition of what AGI is. My initial comment originates from the fact that in my internal definition, an AGI would have a de facto understanding of the physics of "our world". Or better, could infer them by trial and error. But, indeed, it doesn't have to be the case. (The other advantage of the Zelda…

I'd say the issue is the lack of a good, broadly accepted definition of what I is. We all know "smart" when we see it, but actually defining it in a rigorous way is tough.

This difficulty is interesting in and of itself.

When people catalogue the deficiencies in AI systems, they often (at least implicitly) forgive all of our own such limitations. When someone points to something that an AI system clearly doesn't understand, they say that proves it isn't AGI. But if you point at any random human, who fails at the very same task, you wouldn't say they lack "HGI", even if they're too personally limited to ever be taught the skill.

All of which, is to say, I don't think pointing at a limitation of an AI system, really proves it lacks AGI. It's a more slippery definition, than that.

Post reply on HN