Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

François Chollet: The Arc Prize and How We Get to AGI [video]

youtube.com

Post reply on HN
Print view View original

101–110 of 230 posts

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#101

Post by mindcrime »

kubb wrote: post #17
qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

AGI isn't defined anywhere, so it can be anything you want.

Oh, it's defined in lots of places. The problem is.. it's defined in lots of places!

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#102

Post by ltbarcly3 »

There is some kind of massive brigading happening on this thread. Lots of thoughtful comments are downmodded or flagged (including mine, which I thought was pretty thoughtful. I even said poop instead of shit.).

https://news.ycombinator.com/item?id=44492241

My comment was basically instantly flagged. I see at least 3 other flagged comments that I can't imagine deserve to be flagged.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#104

Post by andoando »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

Who says intelligence is anything more than "pattern matching"? Everything is patterns

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#105

Post by IanCal »

ben_w wrote: post #95
kypro wrote: post #55

Earlier quoted context omitted.

I find the "what is intelligence?" discussion a little pointless if I'm honest. It's similar to asking a question like does it mean to be a "good person" and would we know whether an AI or person is really "good"? While understanding why a person or AI is doing what it's doing can be important (perhaps specifically in safety contexts) at the end of the day all that's really going to matter to most people is the outco…

That's certainly a valid way of looking at their abilities at any given task — "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim". But when the question is "are they going to more important to the economy than humans?", then they have to be good at basically everything a human can do, otherwise we just see a variant of Amdahl's law in action and the…

> are they going to more important to the economy than humans?", then they have to be good at basically everything a human can do,

I really don’t think that’s the case. A robot that can stack shelves faster than a human is more valuable at that job than someone who can move items and also appreciate comedy. One that can write software more reliably than person X is more valuable than them at that job even if X is well rounded and can do cryptic crosswords and play the guitar.

Also many tasks they can be worse but cheaper.

I do wonder how many tasks something like o3 or o3 pro can’t do as well as a median employee.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#106

Post by adastra22 »

ummonk wrote: post #98
TheAceOfHearts wrote: post #20

Earlier quoted context omitted.

Getting a high score on ARC doesn't mean we have AGI and Chollet has always said as much AFAIK, it's meant to push the AI research space in a positive direction. Being able to solve ARC problems is probably a pre-requisite to AGI. It's a directional push into the fog of war, with the claim being that we should explore that area because we expect it's relevant to building AGI.

"Being able to solve ARC problems is probably a pre-requisite to AGI." - is it? Humans have general intelligence and most can't solve the harder ARC problems.

They, and the other posters posting similar things, don't mean human-like intelligence, or even the rigorously defined solving of unconstrained problem spaces that originally defined Artificial General Intelligence (in contrast to "narrow" intelligence").

They mean an artificial god, and it has become a god of the gaps: we have made artificial general intelligence, and it is more human-like than god-like, and so to make a god we must have it do XYZ precisely because that is something which people can't do.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#107

Post by sva_ »

qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

It is a necessary condition, but not a sufficient one.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#108

Post by echelon »

TheAceOfHearts wrote: post #20
qoez wrote: post #4

I feel like I'm the only one who isn't convinced getting a high score on the ARC eval test means we have AGI. It's mostly about pattern matching (and some of it ambiguous even for humans what the actual true response aught to be). It's like how in humans there's lots of different 'types' of intelligence, and just overfitting on IQ tests doesn't in my mind convince me a person is actually that smart.

Getting a high score on ARC doesn't mean we have AGI and Chollet has always said as much AFAIK, it's meant to push the AI research space in a positive direction. Being able to solve ARC problems is probably a pre-requisite to AGI. It's a directional push into the fog of war, with the claim being that we should explore that area because we expect it's relevant to building AGI.

My problem with AGI is the lack of a simple, concrete definition.

Can we formalize it as giving out a task expressible in, say, n^m bytes of information that encodes a task of n^(m+q) real algorithmic and verification complexity -- then solving that task within a certain time, compute, and attempt bounds?

Something that captures "the AI was able to unwind the underlying unspoken complexity of the novel problem".

I feel like one could map a variety of easy human "brain teaser" type tasks to heuristics that fit within some mathematical framework and then grow the formalism from there.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#109

Post by abeppu »

acegod wrote: post #103

[dead]

I think you're basically saying that ARC-AGI doesn't achieve a goal that _it didn't set_. The point of ARC-AGI is not to benchmark LLMs specifically. The point is to measure fluid intelligence in a way which supports comparisons between models and between models and humans. It's not the obligation of the test to be tailored to the form of model that's most popular now.

Re: François Chollet: The Arc Prize and How We Get to AGI [video]

#110

Post by lawlessone »

How do we define AGI?

I would have thought/considered AGI to be something that is constantly aware, a biological brain is always on. An LLM is on briefly while it's inferring.

A biological brain constantly updates itself adds memories of things. Those memories generally stick around.

Post reply on HN