Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-6 Astra

openai.com

Post reply on HN
Print view View original

331–340 of 1001 posts

Re: GPT-6 Astra

#331

Post by bbor »

applfanboysbgon wrote: post #292
intenex wrote: post #248

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

My definition of AGI certainly doesn't entail passing a benchmark that some random person arbitrarily labelled AGI to make it sound cooler.

And my definition of climate change doesn't entail passing some arbitrary benchmarks[1] that some random person arbitrarily labelled a problem to make it sound more dangerous.

It's obvious that these scientists are in bad faith, as they've invested way too much of their lives into the field being real -- they're just playing up the data. Common sense tells me that winter is still happening, anyway; what's the big fuss?

(/s, cause you never know these days)

[1] https://upload.wikimedia.org/wikipedia/commons/e/e2/The_Plan...

Re: GPT-6 Astra

#332

Post by codruterdei »

I was actually wondering when they will release the new Opel Astra model. Good and reliable car, wondering if we can say the same thing about this model and its impact on the market.

Re: GPT-6 Astra

#334

Post by upupupandaway »

jdprgm wrote: post #288

Is anyone else just exhausted by the pace of all this. The models change constantly and relentlessly and so does the pricing, basically weekly at this point between all the labs. It feels nearly impossible to have any rigorous approach when choosing a particular model and price point for a task and more like blindly picking one. The time period needed to actually get familiar with various models to a degree you can i…

> The models change constantly and relentlessly and so does the pricing, basically weekly at this point between all the labs.

A dev in my team saw a new model and changed one application to use said model (essentially changing the contents of a url). One week later I received an escalation from the CTO of the company that our pace of weekly usage was in the millions of dollars (rather than low hundred thousands). Turns out that the new model was 5x more expensive but no one noticed.

Re: GPT-6 Astra

#335

Post by XCSme »

It's fun, but every new model release makes me even less interested to create cool stuff. Like, what's the point, if the next AI can do it in 5 seconds?

Re: GPT-6 Astra

#336

Post by upupupandaway »

Pikamander2 wrote: post #310
jdprgm wrote: post #288

Is anyone else just exhausted by the pace of all this. The models change constantly and relentlessly and so does the pricing, basically weekly at this point between all the labs. It feels nearly impossible to have any rigorous approach when choosing a particular model and price point for a task and more like blindly picking one. The time period needed to actually get familiar with various models to a degree you can i…

That's how cutting edge tech has always worked. Imagine buying a shiny new PC in the 90s only to see it become practically obsolete within a year.

Or you could buy a PC with a Celeron CPU, which was obsolete way before launch.

Re: GPT-6 Astra

#337

Post by hypfer »

intenex wrote: post #248

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

What does "AGI" or "effective AGI" even mean, and why should anyone even care whether this unclear thing has been "reached" or not?

Computer chips got faster, but 2026 edition. Why the artificial ceiling/category/goal labelled "AGI"?

I'd much rather like to talk about what this enables, instead of discussing whether a category someone made up applies here or not.

Re: GPT-6 Astra

#338

Post by qsort »

intenex wrote: post #248

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

> The ARC-AGI-3 scorecard is extremely misleading (...)

True.

> Regardless, the result is still valid (...)

If you think the game is rigged, the virtuous thing to do is to point that out and refuse to partecipate; making up your own rules is something I just don't understand, especially since the rule-abiding result would still have been SOTA.

> in the sense of passing the most famous benchmark designed specifically to measure AGI progress

The benchmark does not measure AGI progress or progress towards superhuman intelligence, as explicitly stated by the creators.

On the AGI question: surely you realize this depends on how we define the term? For example, one of the definitions OpenAI originally gave is "capable of doing most economically valuable work", which almost certainly Astra, as impressive as it is, would fall short of. I'm not saying it's a good definition, but as far as I'm concerned it's as good as any. More importantly, I don't think that it would change much if we said yes or no. I'm only bothering to take a position if it amounts to something.

This can feel as "moving the goalposts", and to some extent it is, but if done honestly "moving the goalposts" is how you make progress. Had you asked me 10 years ago I would have said that anything that could hold a conversation like GPT-4 could would probably have been wildly superhuman at almost everything. It shouldn't be hard to find ways GPT-4 was lacking, though. We see new things, we reassess and try again: that's how it's supposed to work.

Re: GPT-6 Astra

#339

Post by wilg »

siva7 wrote: post #76
isoprophlex wrote: post #66

> We have found that GPT-6 Astra is more capable of controlling its own CoT than GPT 5.6-Sol, and less likely to include incriminating information in its CoT. In adversarial settings (where we push the model to evade our monitors) we find that the model is able to remain undetected when strategically underperforming in evaluations (sandbagging) and can sometimes evade our internal monitors when asked to perform certa…

Sounds fun. As fun as their press release claiming it is the most safety aligned model ever.

These are not mutually exclusive ideas

Re: GPT-6 Astra

#340

Post by upupupandaway »

dalemhurley wrote: post #284

OpenAI is killing it now that they are more focused. Killing projects like Sora et al have seen it go from irrelevant to level footing with Anthropic. Sol is so much better than Fable 5. Then we get Astra (yet to use it) few days after Fable 5.1 (which is very impressive). Codex is slightly better than Claude Code. Good on Sam Altman getting back to basics and turning OpenAI around.

Their ads business is also doing well. Not "will recover all compute costs" well, but crossed $1b in a few months.
Post reply on HN