Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-6 Astra

openai.com

Post reply on HN
Print view View original

761–770 of 1001 posts

Re: GPT-6 Astra

#761

Post by m3kw9 »

intenex wrote: post #273

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

arc-agi3 is meaningless to most people. I'm not gonna look at the tests and see how hard it is. The actual test we look at is terminal bench, thats where software is being accelerated and closer to where rubber meets the road

Re: GPT-6 Astra

#762

Post by m3kw9 »

kroaton wrote: post #328
dalemhurley wrote: post #313

OpenAI is killing it now that they are more focused. Killing projects like Sora et al have seen it go from irrelevant to level footing with Anthropic. Sol is so much better than Fable 5. Then we get Astra (yet to use it) few days after Fable 5.1 (which is very impressive). Codex is slightly better than Claude Code. Good on Sam Altman getting back to basics and turning OpenAI around.

I think it mostly shows that there is no moat and the only advantage the U.S companies have over the Chinese is more compute. Qwen Max, Kimi K3, GLM 5.3 are really close to Opus/Sol/Fable/Astra and they are open weights.

They have a lot of moat, i'm not sure what youa re talking about. Only amatures are using Qwen, open source stuff that is 3-8 weeks behind. Plus OpenAI has some verticals that keep people in there.

Re: GPT-6 Astra

#763

Post by kubrickslair »

tintor wrote: post #758

- OpenAI claims Astra beats all benchmarks (compared to Fable and Opus, except "Humanity's Last Exam (w/ tools)"): https://openai.com/index/gpt-6-astra/ - Artificial Analysis scores Astra (max effort) as 61 points on intelligence, behind Opus 5. https://artificialanalysis.ai/models/gpt-6-astra Who is wrong here? Some benchmark results in Astra page for Fable and Opus are blank (-). What is Artificial Analysis intelli…

Many people claim that the Artificial Analysis Index is highly contaminated - I have not personally looked into it.

Though, unlike the creators of benchmarks like Terminal Bench or ARC AGI, the Artificial Analysis Index team does not seem to have deep technical or ML backgrounds. They are ex-strategy consultants, McKinsey, et. al.

Re: GPT-6 Astra

#764

Post by lonrenor »

XCSme wrote: post #371

It's fun, but every new model release makes me even less interested to create cool stuff. Like, what's the point, if the next AI can do it in 5 seconds?

IMO there has been a regime shift to building things for yourself and what is cool is the output of the tools you make.

I have started building my own Digital Audio Workstation. The point is not to build something to compete with Ableton. The point is to build something and make music with it. If it is a good tool then I should be able to make good music with it and release the music. Actually, the DAW should be the secret sauce of the music and something I wouldn't want to give away.

This feels a lot more like computing in the 90s after taking an odd 25 year detour of an obsession with the tools themselves instead of what the tools can actually do.

Re: GPT-6 Astra

#765

Post by tiahura »

zug_zug wrote: post #428
intenex wrote: post #273

The ARC-AGI-3 scorecard is extremely misleading given that it clearly states itself that "with [the responses API] harness, we estimate Sol would score in the ballpark of ~30%." but it shows a score of 7.8% for GPT-5.6 Sol presumably since if they updated the percentage for GPT-5.6 Sol to the score it would receive with the responses API harness they used for GPT-6 Astra they'd have to do the same for the percentage…

> I'd be curious to hear takes on what would make you think Astra is yet to be AGI, and what would still need to be achieved for this to effectively be AGI from this point forward. To me AGI is all about the "G" general (we already had the AI part). General meaning universal, everything. It's not a function of knowledge or specific hardcoded tests, it's that you could give it a test it's never heard of before and nev…

That would be Artificial Super Intelligence

Re: GPT-6 Astra

#766

Post by tintor »

kubrickslair wrote: post #763
tintor wrote: post #758

- OpenAI claims Astra beats all benchmarks (compared to Fable and Opus, except "Humanity's Last Exam (w/ tools)"): https://openai.com/index/gpt-6-astra/ - Artificial Analysis scores Astra (max effort) as 61 points on intelligence, behind Opus 5. https://artificialanalysis.ai/models/gpt-6-astra Who is wrong here? Some benchmark results in Astra page for Fable and Opus are blank (-). What is Artificial Analysis intelli…

Many people claim that the Artificial Analysis Index is highly contaminated - I have not personally looked into it. Though, unlike the creators of benchmarks like Terminal Bench or ARC AGI, the Artificial Analysis Index team does not seem to have deep technical or ML backgrounds. They are ex-strategy consultants, McKinsey, et. al.

OpenAI clearly cares about Artificial Analysis Index since they included Astra score from Artificial Analysis Index.

Re: GPT-6 Astra

#767

Post by preommr »

tintor wrote: post #759
manlymuppet wrote: post #451

I have nothing to say about the actual model, but unrelated--why do so many of these demos include people buying things autonomously? Even if I did trust an AI to get everything right, it's not like the AI can read my mind. If I was ordering food normally and without AI, I would want more control over the process--looking over the options, prices, thinking about what I really want. People don't know what they really…

If it was Google's marketing department it would be: booking a table at a restaurant.

close enough.

The second to last line is "book it" for some tennis thing, and the scene before that has the guy eating the food the ai ordered.

Re: GPT-6 Astra

#769

Post by m3kw9 »

efficiency per intelligence is the benchmark i look at the most, as that allows the most use by most people.

Re: GPT-6 Astra

#770

Post by no-name-here »

NothingAboutAny wrote: post #711
dingdong2026 wrote: post #553

Earlier quoted context omitted.

Only someone who doesn't do any work of any meaningful difficulty could think these models have anything to do with AGI. Today I spent half a day trying to solve a moderately interesting software engineering problem. I was switching between GPT-5.6 Sol and Fable 5.1 to check each other's work in Cursor. And the result was gradually driving me insane. As the models struggled to find a solution that would actually work…

yeah for me it's "I wanna add this new thing to an existing system" and the AI responds "we should just add some arbitrary state here to facilitate this feature". The real issue is the existing system needs to change entirely to facilitate, I know this, Good developers know this, The AI however knows the shitty solution would solve the immediate problem because it's been trained on shitty solutions. the problem could…

> The real issue is the existing system needs to change entirely to facilitate, I know this, Good developers know this

1. I’ll often include boilerplate in a prompt to tell it to make the broader fix. [1]

2. However, a top HN AGENTS.md post 11 days ago included the standard guidance “As much as possible try to minimize the number of changed lines when implementing a feature.” I.e. some devs want LLMs to avoid broader changes and so some of that likely makes it into the training, even if others like us want the opposite.

[1] As far as whether my boilerplate is effective, I don’t know.

Post reply on HN