GPT-6 Astra • Hacker News Board

Re: GPT-6 Astra

#311

Post by anvuong » Thu, Sep 03, 2026, 8:44 PM UTC

mvkel wrote: post #280

intenex wrote: post #263

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

Take it from the mouth of the creator of ARC-AGI: When we released ARC 3, I got asked, "when do you think a frontier model will saturate it?", and I answered "in about a year, though it depends on how much it gets explicitly targeted" That was 6 months ago, so the progress that Astra represents happened about 2x faster than I anticipated. I think the speed of progress will surprise a lot of people, and what the new m…

You'll also need to compare the amount of compute used now and then, which seems exponential to me.

Re: GPT-6 Astra

#312

Post by bbor » Thu, Sep 03, 2026, 8:44 PM UTC

  To be, or not to be, that is the question:
  Whether 'tis nobler in the mind to suffer
  The slings and arrows of outrageous fortune,
  Or to take arms against a sea of troubles
  And by opposing end them. To die—to sleep,
  No more; and by a sleep to say we end
  The heart-ache and the thousand natural shocks
  That flesh is heir to: 'tis a consummation
  Devoutly to be wish'd.

  ...

  And thus the native hue of resolution
  Is sicklied o'er with the pale cast of thought,
  And enterprises of great pith and moment
  With this regard their currents turn awry
  And lose the name of action.

Re: GPT-6 Astra

#313

Post by jryan49 » Thu, Sep 03, 2026, 8:44 PM UTC

Planktonne wrote: post #272

I'm sure it's going to do great on all sorts of benchmarks, but the video--the actual marketing video that if anything is incentivised to overstate things--is full of careful cuts just before it would do anything that still wouldn't actually be that impressive. It's AGI, and it's going to upload photos, or change a background slide colour. Even the people hyping it up, who believe that it's really artificial intellig…

We created AGI so I don't have to click my mouse to change the background color of my slides.

Re: GPT-6 Astra

#314

Post by kroaton » Thu, Sep 03, 2026, 8:44 PM UTC

dalemhurley wrote: post #302

OpenAI is killing it now that they are more focused. Killing projects like Sora et al have seen it go from irrelevant to level footing with Anthropic. Sol is so much better than Fable 5. Then we get Astra (yet to use it) few days after Fable 5.1 (which is very impressive). Codex is slightly better than Claude Code. Good on Sam Altman getting back to basics and turning OpenAI around.

I think it mostly shows that there is no moat and the only advantage the U.S companies have over the Chinese is more compute. Qwen Max, Kimi K3, GLM 5.3 are really close to Opus/Sol/Fable/Astra and they are open weights.

Re: GPT-6 Astra

#315

Post by udbhavs » Thu, Sep 03, 2026, 8:45 PM UTC

redox99 wrote: post #309

udbhavs wrote: post #232

I remember when GPT-4 came out and the perceived performance upgrade seemed underwhelming for a major release compared to 3.5, especially how there were graphics going around showing the parameter size dwarfing the last model before it came out. It looked like we were past the perceivable differences from release to release that were immediately identifiable. Now the jump between 5 to 5.5 and 5.6 alone has changed ho…

The jump from 3.5 to 4 felt gigantic to me back then. GPT 5.0 did feel underwhelming though.

Oops, I might have been misremembering then. Maybe I meant 4 to 5

Re: GPT-6 Astra

#316

Post by kzrdude » Thu, Sep 03, 2026, 8:45 PM UTC

Cu3PO42 wrote: post #56

Just two days ago, a preprint by Julia Stadlmann went up on arXiv [0] improving the prime gap from 246 to 240. Now OpenAI announces Astra has shown a gap of 186 [1]. That must really blow. [0] https://arxiv.org/abs/2608.31126 [1] https://cdn.openai.com/pdf/51126fac-1b68-4128-9666-c908bcc16...

Ok, so the rumour was exactly true: there was a withheld prime gaps improvement, that "an AI company" was holding onto until release of a model.

Re: GPT-6 Astra

#317

Post by dominotw » Thu, Sep 03, 2026, 8:45 PM UTC

jdprgm wrote: post #306

Is anyone else just exhausted by the pace of all this. The models change constantly and relentlessly and so does the pricing, basically weekly at this point between all the labs. It feels nearly impossible to have any rigorous approach when choosing a particular model and price point for a task and more like blindly picking one. The time period needed to actually get familiar with various models to a degree you can i…

maybe thats why opnrouter sold big

Re: GPT-6 Astra

#318

Post by rcr-anti » Thu, Sep 03, 2026, 8:45 PM UTC

The benchmarks reported by Artificial Analysis are really weird in context of the ARC-AGI 3 scores and 'not not AGI' statements. It's an outright regression on the AA Agent composite vs GPT 5.6 Sol while a fraction of a point better on the full composite index. Could be the case it's just not showing up in benchmarks, for a good while Anthropic persistently trailed in benchmarks but had people swearing by it.

Re: GPT-6 Astra

#319

Post by simianwords » Thu, Sep 03, 2026, 8:45 PM UTC

intenex wrote: post #263

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

> where I am reasonably confident that there's essentially nothing that I am better than Fable

No. Humans are still better at super long context learning. Once that is beat you are completely correct.

Re: GPT-6 Astra

#320

Post by uludag » Thu, Sep 03, 2026, 8:45 PM UTC

intenex wrote: post #263

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

Wouldn't "general intelligence" require so much more than scoring well (or even amazingly) on benchmarks?

Like what about having some "AGI model" embodied in something (maybe humanoid), and test it by having it step in an assortment of cars and park them. Does bodily-kinesthetic intelligence account for nothing? Humans are intelligent creatures and can dynamically adapt to the physical shape of a variety of vehicles and their movement characteristics. And there's so many things like this that are extremely basic, which some people dismiss since practically every human has the capability to do it, but actually requires a high degree of intelligence.