Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-6 Astra on ARC-AGI-3

arcprize.org

Post reply on HN
Print view View original

81–90 of 131 posts

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#81

Post by meander_water »

fxd wrote: post #72

“AGI” never made sense to me. It’s a purely marketing term right? I’ve ignored it thinking it would go away, but it keeps coming up. I get that consciousness differs from intelligence and that our waking awareness of life is a complete mystery. Knowledge and thus intelligence however I consider as actively being solved by these large ML models. That is, with the right combination of machinery and know-how, you’ll get…

The OpenAI charter defines it as:

"highly autonomous systems that outperform humans at most economically valuable work"

https://time.com/article/2026/08/26/openai-sam-altman-interv...

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#82

Post by layer8 »

p1esk wrote: post #80
layer8 wrote: post #77

Earlier quoted context omitted.

Not sure why you are bringing up consciousness, that’s largely orthogonal to intelligence. AGI is usually taken to mean the capability to match or surpass human intelligence across all conceivable cognitive tasks, as opposed to being limited to certain kinds of tasks, or to not matching the general level of human intelligence in some respect. Intelligence, and hence AGI, doesn’t require consciousness or emotions or s…

capability to match or surpass human intelligence across all conceivable cognitive tasks What human intelligence do you mean? Genius? Professional? Educated? Random person? “Dumb” person?

“General”.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#83

Post by an0malous »

Was OpenAI able to run ARC-AGI-3 tests previously so that they could build a custom harness for the specific tests in the set? Even with the standard harness, if they knew the problems ahead of them they could have used supervised reinforcement learning to teach the model how to solve these specific tests.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#84

Post by p1esk »

layer8 wrote: post #82
p1esk wrote: post #80

Earlier quoted context omitted.

capability to match or surpass human intelligence across all conceivable cognitive tasks What human intelligence do you mean? Genius? Professional? Educated? Random person? “Dumb” person?

“General”.

Sorry, I don’t know what this means when applied to intelligence

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#86

Post by modeless »

$360 per puzzle. When they tested people it took about 10 minutes per puzzle. If price/performance keeps falling at the same rate it has been, this will cost less than US minimum wage humans within two years. Three for Phillipines minimum wage.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#87

Post by at1as »

I like Erdos problems as a benchmark. Models continue to solve them, but a pretty tepid rate now that the low hanging fruit has been taken.

From https://epoch.ai/latest/announcing-frontiermath-erdos

> Only GPT-6 Astra solved anything: 2 of the 68 problems. It disproved problem 74 by finding a counterexample, at a cost of $218 and 15 hours of working time, and it proved problem 126, at a cost of $247 and 16 hours

> Across all attempts, GPT-6 Astra solved 5 of the 68 problems at least once: the two above, plus problem 1, which it disproved, and problem 548 and problem 571, which it proved. Most of the remaining problems were attempted between two and five times in total (172 attempts), and none was solved. Reaching these five solutions took over $220,000 of compute across all attempts, compared with roughly $20,000 for the benchmark run itself.

Which implies a genuine improvement in capability, but there's a still a very long tail ahead that models will continue to need to improve to capture.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#88

Post by drdeca »

p1esk wrote: post #80
layer8 wrote: post #77

Earlier quoted context omitted.

Not sure why you are bringing up consciousness, that’s largely orthogonal to intelligence. AGI is usually taken to mean the capability to match or surpass human intelligence across all conceivable cognitive tasks, as opposed to being limited to certain kinds of tasks, or to not matching the general level of human intelligence in some respect. Intelligence, and hence AGI, doesn’t require consciousness or emotions or s…

capability to match or surpass human intelligence across all conceivable cognitive tasks What human intelligence do you mean? Genius? Professional? Educated? Random person? “Dumb” person?

I think the distinction between these options probably doesn’t matter alll that much if the threshold you’re using is either “random person” or higher? Maybe bump it up to “random educated person”?

They are still different concepts of course, but I imagine that once one is achieved, the others aren’t far off.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#89

Post by mdp2021 »

ranyume wrote: post #74
mdp2021 wrote: post #65

Earlier quoted context omitted.

> Because the IQ is No, it is just because they have difficulties at the bench. > how gives we don't measure them by We'd measure them by all the tests available. Not all test are usable in all circumstances.

> No, it is just because they have difficulties at the bench. I'll put it in another way. A "gifted kid" can be measured incredibly well on an IQ test, but fail miserably at incredibly normal but very difficult tasks such as consoling someone for their loss and managing family crisis. This is a clear example where an IQ measure doesn't translate to a person being capable of meaningfully changing their environments fo…

We have always been very aware that IQ does not measure all skills - nonetheless, it does measure one.

Nobody says that the IQ test would "measure intelligence". We know it does test a form of it.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#90

Post by an0malous »

an0malous wrote: post #83

Was OpenAI able to run ARC-AGI-3 tests previously so that they could build a custom harness for the specific tests in the set? Even with the standard harness, if they knew the problems ahead of them they could have used supervised reinforcement learning to teach the model how to solve these specific tests.

Actually I can answer my own question: we know that they have had previous access to the tests because they’ve run older models against the same benchmark.

I wouldn’t put it past a company like OpenAI with a long history of lying and being deceptive to record the tests and benchmaxx ARC. They have trillions of dollars of incentive to cheat any way they can.

Post reply on HN