Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6

openai.com

Post reply on HN
Print view View original

721–730 of 1001 posts

Re: GPT-5.6

#721

Post by insanitybit »

inciampati wrote: post #118
eig wrote: post #38

Funny to see that they did not include Fable 5 in their GeneBench and LifeSciBench comparisons because "it does not answer advanced biology questions and refuses the majority of questions in this eval". Winner by default!

This is a major reason why I and a number of biologists I've talked to have canceled their anthropic accounts recently. Not working is not working.

I'm writing a programming language with a "capability security model". That's enough to trigger Fable, it won't work on the language. It's hilarious. The mere presence of the word "security" seems to be enough to trip it up.

Re: GPT-5.6

#722

Post by insanitybit »

matheusmoreira wrote: post #624
eig wrote: post #38

Funny to see that they did not include Fable 5 in their GeneBench and LifeSciBench comparisons because "it does not answer advanced biology questions and refuses the majority of questions in this eval". Winner by default!

Anthropic just refuses to allow Fable to properly code review my projects. It's so obnoxious. If OpenAI's Fable equivalent is better at this, that'll get me to cancel my Anthropic subscription and switch.

Given that Fable is so gutted and Anthropic added the absurd data retention policy for it, I'm going to advocate that we prioritize support for as many other models as we can at work.

Re: GPT-5.6

#723

Post by isaachinman »

RagnarD wrote: post #694

Annoyingly, the new ChatGPT app which folds in Codex, no longer recognizes Shift-Tab to toggle plan mode. Irritatingly you have to enter /plan. OpenAI, fix this!

Noticed this as well. You have to go into keyboard shortcuts and set it manually.

Re: GPT-5.6

#724

Post by Sabinus »

rsoto2 wrote: post #493
senko wrote: post #439

I love testing the new models by asking them to code a toy RTS game. Here's what Terra did: https://senko.net/vibecode-bench/2026/rts-gpt-5.6-terra.html (one try, in codex app, xhigh effort) Comparing this to other models, I find it similar to GPT-5.5 and a bit behind Sonnet 5. You can see how other models fared here: https://senko.net/vibecode-bench/ (you can also fetch the prompt and the the 5.6 Terra resulting cod…

So the measure of a model is how well they can recreate something they easily have thousands of examples of in their training data. There's probably a better base RTS on github somewhere for free.

The 5.6 model article for this post has three examples of little web games.

There are plenty of little js web games anyway. The point isn't to make an actual game, it's to show coding ability, design and taste in a way that's more assessable than reading a codebase.

Re: GPT-5.6

#725

Post by mnicky »

ls_stats wrote: post #712

The most impressive part is the token efficiency/cost per task of 5.6 Sol, it makes Opus 4.8 and Fable look extremely bad ($1.04 vs $1.80 vs $2.75)[0]. And 5.6 Luna ($0.21) is also impressive, cheaper than GLM 5.2 ($0.37) with higher intelligence. [0]: https://artificialanalysis.ai/#price-and-cost

Well it's smaller model (something like 4T against 10T Fable). So it's faster and cheaper and with a lot of RL and maybe some favorable benchmark selection it can compete on these scores. In real tasks I expect it to have less intelligence, generalization ability, etc. than Fable.

Re: GPT-5.6

#727

Post by mnicky »

djx22 wrote: post #720

Not sure what everyone's experience is but I find 5.6 Sol to be a great liar. Reported success on a half done job and left things in a broken state after having quite a few back & forth followups on the initial prompt to clarify the plan. Didn't experience this with 5.5. Opus 4.7 and below sometimes did it but they fixed it in Opus 4.8. So, overall, the initial experience has made me think that this model will be a l…

May be related to this from METR evaluation:

> GPT-5.6 Sol’s detected cheating rate was higher than any public model we have evaluated

Re: GPT-5.6

#728

Post by ai_fry_ur_brain »

stavros wrote: post #278
mNovak wrote: post #260

>> approximately 700,000 A100e GPU hours of black-box automated red teaming Amusing that they use A100e as the reference point to sound impressive. Different ways you could make that conversion, but based on FP4 FLOPs (yes it's disadvantageous to A100, that's the point), that's something like 200hr on a GB300 NVL72 rack. Not nothing either, but far less astounding sounding than 700k hrs.

Wait, what do you mean? 700k A100e hours are equal to 200 hours of a GB300 NVL72 rack? One GB300 NVL72, 72-GPU rack has equal processing power to 3500 A100e GPUs?

[flagged]

Re: GPT-5.6

#729

Post by ai_fry_ur_brain »

prodmod wrote: post #580

Things I have been struggling with Fable over and GPT 5.5, were just solved handily by SOL in a real "thank you, next problem" kind of way. Overall, something that just works is way less wasteful for your usage than struggling back and forth for hours.

[flagged]

Re: GPT-5.6

#730

Post by 8note »

ravenstine wrote: post #94
minimaxir wrote: post #26

The developer's guide ( https://developers.openai.com/api/docs/guides/latest-model ) has some interesting semantic tips for using the model: > Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly. > Original image detail: GPT-5.6 preserves the…

> Avoid generic brevity instructions That part is confusing because it's not like they provide an example of how default GPT-5.6 output compares with GPT-5.5 both with default output and prompted for brevity. Whenever I use such prompts, it's usually because I want the model to give me the gist in a few sentences. I'd be stunned if GPT-5.6 was that concise by default. I would think that could "break" a lot of things…

this is a dependency update.

shouldnt you have good testing for that and not deploy a version update when those tests fail?

Post reply on HN