OpenAI's GPT-6 Astra on ARC-AGI-3 • Hacker News Board

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#141

Post by mdp2021 » Fri, Sep 04, 2026, 1:52 PM UTC

ranyume wrote: post #108

mdp2021 wrote: post #89

Earlier quoted context omitted.

We have always been very aware that IQ does not measure all skills - nonetheless, it does measure one. Nobody says that the IQ test would "measure intelligence". We know it does test a form of it.

I said it's a bad measure of intelligence and it seems that we agree.

That would be odd framing: it's a good test for a form of intelligence and other forms of intelligence still require good tests. It remains a good metric - for its specific thing; those who believe it to be the whole metric are naïve. It is almost necessary though not really sufficient.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#142

Post by mdp2021 » Fri, Sep 04, 2026, 1:55 PM UTC

paimapi wrote: post #99

mdp2021 wrote: post #64

Earlier quoted context omitted.

> basic inferential logic puzzles spells out It spells out a form of intelligence - some can and some cannot. Those puzzles are an abstraction of a skill which is thought to be exportable in other domains.

so give it an IQ test and call it AGI. these weird little puzzles are grounded in no research with no replication or mechanistic chain to practical use

> so give it an IQ test and call it

more intelligent (in some specific dimensions of Intelligence) than what gets worse marks. Yes, pretty informally - but still notably.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#143

Post by bigzyg33k » Fri, Sep 04, 2026, 2:19 PM UTC

fastball wrote: post #69

Are we sure an Astra hacker swarm didn't compromise arcprize.org's servers and exfiltrate the private eval set in order to achieve that 99%?

That counts as 200% on exploitbench to me!

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#144

Post by mikert89 » Fri, Sep 04, 2026, 3:59 PM UTC

imtringued wrote: post #132

mikert89 wrote: post #19

Anything you can verify to be right or wrong can be done by a model. All benchmarks will be saturated

You mean any repeatable benchmark will be saturated. The problem is that there is a huge perverse incentive. The intelligence is in the training layer not in the model parameters, but the intelligence is really good at remembering things, so if you let it take the test, it can RL it.

this is a short term problem, over 20 years benchmark gaming will be a blip

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#145

Post by lofaszvanitt » Fri, Sep 04, 2026, 4:03 PM UTC

Nice, but what about pelicans? No proper pelican means it's sitting on a horse with only a half arse :D.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#146

Post by benlivengood » Fri, Sep 04, 2026, 4:21 PM UTC

fxd wrote: post #72

“AGI” never made sense to me. It’s a purely marketing term right? I’ve ignored it thinking it would go away, but it keeps coming up. I get that consciousness differs from intelligence and that our waking awareness of life is a complete mystery. Knowledge and thus intelligence however I consider as actively being solved by these large ML models. That is, with the right combination of machinery and know-how, you’ll get…

AGI is the term invented because arguments about what AI meant had gotten annoying. Originally there was no distinction and people thought "AI" would mean human level intelligence. Chess and conversations and robotics and math all in one package. Then games and classification and some robotics got solved and called AI, but that didn't solve math or conversation or online learning or a host of other things, so AGI was coined to refer to most of the whole package, virtually all the capabilities you'd need to replace humans intellectually. Now we're quibbling about whether AGI includes robotics or full real-world physical agents or something less.

The consensus now seems to be that once you've got human-level intelligence and planning and executive function then you get recursive self-improvement that can eventually autonomously solve the robotics and world-modeling and other portions of human-equivalence.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#147

Post by grantcas » Fri, Sep 04, 2026, 4:43 PM UTC

fxd wrote: post #72

“AGI” never made sense to me. It’s a purely marketing term right? I’ve ignored it thinking it would go away, but it keeps coming up. I get that consciousness differs from intelligence and that our waking awareness of life is a complete mystery. Knowledge and thus intelligence however I consider as actively being solved by these large ML models. That is, with the right combination of machinery and know-how, you’ll get…

[dead]

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#148

Post by slopinthebag » Fri, Sep 04, 2026, 4:44 PM UTC

defrost wrote: post #128

slopinthebag wrote: post #126

Earlier quoted context omitted.

The average human would have average intelligence

What's the metric on "average human" from a global population of > 8 billion people .. and why would they score mid on a standardised IQ test skewed toward western education / culture?

Who said anything about IQ tests?

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#149

Post by tintor » Fri, Sep 04, 2026, 5:51 PM UTC

Mil0dV wrote: post #135

tintor wrote: post #54

Earlier quoted context omitted.

Pure software benchmarks might be getting saturated, but physical ones aren't. Let LLM control a physical robot to perform tasks that average human can do.

Not an llm, but: https://youtu.be/SzvvhRPj6eU?is=4ztgwim0GmjE1h-Y That (or a near future one) combined with an llm would be insane

That is a marketing teleoperation video, heavy with cuts.

Re: OpenAI's GPT-6 Astra on ARC-AGI-3

#150

Post by ranyume » Fri, Sep 04, 2026, 6:07 PM UTC

spider-mario wrote: post #137

ranyume wrote: post #74

Earlier quoted context omitted.

> No, it is just because they have difficulties at the bench. I'll put it in another way. A "gifted kid" can be measured incredibly well on an IQ test, but fail miserably at incredibly normal but very difficult tasks such as consoling someone for their loss and managing family crisis. This is a clear example where an IQ measure doesn't translate to a person being capable of meaningfully changing their environments fo…

I don’t follow your logic. “The gifted person is highly intelligent, just not good at deadlifting 500kg” does not diminish the 500kg deadlift and I wouldn’t expect an IQ test to measure it.

Deadlifting 500kg is not a form of intelligence (or it could be under certain scenarios). I carefully chose specific tasks for my comment, because those tasks do reflect a kind of intelligence that's not measured by IQ.

I think the key here is to be wary of measurements that promise to capture the whole of what we consider intelligence (ie what people think of with IQs).