Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

François Chollet: The Arc Prize and How We Get to AGI [video]

youtube.com

Post reply on HN
Print view View original

151–160 of 230 posts

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#151

Post by krackers »

bogtog wrote: post #91

I wonder how much slow progress on ARC can be explained by their visual properties making them easy for humans but hard for LLMs. My impression is that models are pretty bad at interpreting grids of characters. Yesterday, I was trying to get Claude to convert a message into a cipher where it converted a 98-character string into 7x14 grid where the sequential letters moved 2-right and 1-down (i.e., like a knight it ch…

I thought so too back when the test was first released, but now that we have multimodal models which can take images directly as input, shouldn't this point be moot?

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#152

Post by [deleted] »

MPSimmons wrote: post #130
lostphilosopher wrote: post #89

Earlier quoted context omitted.

We don't really have a true test that means "if we pass this test we have AGI" but we have a variety of tests (like ARC) that we believe any true AGI would be able to pass. It's a "necessary but not sufficient" situation. Also ties directly to the challenge in defining what AGI really means. You see a lot of discussions of "moving the goal posts" around AGI, but as I see it we've never had goal posts, we've just got…

I don't think we actually even have a good definition of "This is what AGI is, and here are the stationary goal posts that, when these thresholds are met, then we will have AGI". If you judged human intelligence by our AI standards, then would humans even pass as Natural General Intelligence? Human intelligence tests are constantly changing, being invalidated, and rerolled as well. I maintain that today's modern LLMs…

[deleted]

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#153

Post by adastra22 »

ummonk wrote: post #114
adastra22 wrote: post #106

Earlier quoted context omitted.

They, and the other posters posting similar things, don't mean human-like intelligence, or even the rigorously defined solving of unconstrained problem spaces that originally defined Artificial General Intelligence (in contrast to "narrow" intelligence"). They mean an artificial god, and it has become a god of the gaps: we have made artificial general intelligence, and it is more human-like than god-like, and so to m…

Right, but there is a very clear term for that which they should be using: ASI

Agree, and you shouldn't be downvoted. It is a pet peeve of me as well.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#154

Post by cubefox »

sweezyjeezy wrote: post #149
cubefox wrote: post #144

Earlier quoted context omitted.

He only says this because otherwise he would have to say that - OpenAI's o3 counts as "AGI" when it did unexpectedly beat the ARC-AGI benchmark or - Explicitly admit that he was wrong when assuming that ARC-AGI would test for AGI

FWIW the original ARC was published in 2019, just after GPT-2 but a while before GPT-3. I work in the field, I think that discussing AGI seriously is actually kind of a recent thing (I'm not sure I ever heard the term 'AGI' until a few years ago). I'm not saying I know he didn't feel that, but he doesn't talk in such terms in the original paper.

> We argue that ARC can be used to measure a human-like form of general fluid intelligence and that it enables fair general intelligence comparisons between AI systems and humans.

https://arxiv.org/abs/1911.01547

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#155

Post by asah »

oldge wrote: post #19
qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

Today’s llms are fancy autocomplete but lack test time self learning or persistent drive. By contrast, an AGI would require: – A goal-generation mechanism (G) that can propose objectives without external prompts – A utility function (U) and policy π(a│s) enabling action selection and hierarchy formation over extended horizons – Stateful memory (M) + feedback integration to evaluate outcomes, revise plans, and execute…

we're closer than you think...

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#156

Post by NetRunnerSu »

tsurba wrote: post #65
NetRunnerSu wrote: post #49

Earlier quoted context omitted.

Minimize prediction errors.

But are we close to doing that in real-time on any reasonably large model? I don’t think so.

This is not about reasoning , this is about continuous learning and perpetual learning .

https://github.com/dmf-archive/PILF

https://dmf-archive.github.io/docs/posts/beyond-snn-plausibl...

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#157

Post by jcelerier »

fvdessen wrote: post #138
tedy1996 wrote: post #132

Earlier quoted context omitted.

I have graduated with a degree in Software engineering and i am bilingual (Bulgarian and English). Currently AI is better than me in everything except adding big numbers or writing code in really niche topics - for example code golfing a Brainfuck interpreter or writing a Rubiks cube solver. I believe AGI has been here for at least a year now.

I suggest you to try to let the AI think through race conditions scenarios in asynchronous programs; it is not that good at these abstract reasoning tasks.

the huge majority of humans aren't either though

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#158

Post by bgwalter »

thatguy0900 wrote: post #70
jacquesm wrote: post #68

Let's not. Seriously. I absolutely love François and have used his work extensively. But looking around me at the social impact of AI I am really not convinced that this is what the world needs right now and that if we can stave off the turning point for another decade or two that humanity will likely benefit from that. The last thing we need is to inject yet another instability into a planet that is already fighting…

It doesn't matter what should or should not happen. Technology will continue to race forward at breakneck speed while everyone involved pats each other on the back for making a bunch of money before the consequences hit

The Iranian nuclear program disagrees. There could be treaties and the analogue of the IAEA for "AI".

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#159

Post by hollerith »

lo_zamoyski wrote: post #73
nessbot wrote: post #72

Earlier quoted context omitted.

technology doesn't just advance itself

This is true. We have a choice...in principle. But in practice, it's like stopping an arms race.

Which arms race? The nuclear arms race of the Cold War? That one got stopped.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#160

Post by goatlover »

MPSimmons wrote: post #130
lostphilosopher wrote: post #89

Earlier quoted context omitted.

We don't really have a true test that means "if we pass this test we have AGI" but we have a variety of tests (like ARC) that we believe any true AGI would be able to pass. It's a "necessary but not sufficient" situation. Also ties directly to the challenge in defining what AGI really means. You see a lot of discussions of "moving the goal posts" around AGI, but as I see it we've never had goal posts, we've just got…

I don't think we actually even have a good definition of "This is what AGI is, and here are the stationary goal posts that, when these thresholds are met, then we will have AGI". If you judged human intelligence by our AI standards, then would humans even pass as Natural General Intelligence? Human intelligence tests are constantly changing, being invalidated, and rerolled as well. I maintain that today's modern LLMs…

Because an important part of being a Natural general Intelligence is having a body and interacting with the world. Data from Star Trek is a good example of an AGI.
Post reply on HN