The FrontierCode 1.1 Extended benchmark is the only benchmark that aligns with my actual LLM experiences and Astra isn't significantly better or cheaper. All this celebration, and yet it's only on-par with an already existing model? I don't get it.
GPT-6 Astra
231–240 of 1001 posts
Re: GPT-6 Astra
#232Argh! I hit a wrong keyboard shortcut and moved the entire thread. Please stand by... it will all come back shortly
500 upvotes with 2 comments would have been a new record. ;)
Re: GPT-6 Astra
#233I hope they don't `fable` it and block people from doing they daily jobs with it, by introducing huge amounts of restrictions that are not really needed.
Re: GPT-6 Astra
#234I wonder how they were able to get it to get 99.9% on ARC-AGI-3. That seems truly insane.
Re: GPT-6 Astra
#235[deleted]
Re: GPT-6 Astra
#236I decided to front run and added support for it in Dirac (coding agent) a couple of hours ago, using best guess pricing: input/output/cache: $10/$50/$1.
Re: GPT-6 Astra
#237Argh! I hit a wrong keyboard shortcut and moved the entire thread. Please stand by... it will all come back shortly
Luckily there’s a standard keyboard shortcut for “undo” as well. ;)
Re: GPT-6 Astra
#238[dead]
Re: GPT-6 Astra
#239[flagged]
Re: GPT-6 Astra
#240This is wild: OpenAI is basically declaring that AGI is here. https://www.theverge.com/ai-artificial-intelligence/989601/o... “If we fast-forward a couple of years, and we look back and say, ‘When was it, really, that AGI was created?’ I think it’s going to be about this time, and I think it might be about this model,” OpenAI president Greg Brockman said during a Thursday press briefing. Later in the call, he added,…
I almost feel like I need just as much healthy skepticism toward hn comments that have the automatic reflex of dismissing performance gains, as much as I need a similar form of skepticism toward AI claims. It feels like (from what I'm understanding) the harnessed result on ARC-AGI-3 is not exactly playing by the normal rules that would tell us how much of a leap this really is. Nothing wrong with harnesses, but if there's one thing they aren't, it's an indicator of generality in performance gains.
So I think it's a bit of a misleading signal and we should wait for more independent vetting. I think the middle ground is that these are improvements worthy of the "GPT-6" label but still well short of a true "this is AGI moment" that would truly put the question to rest.