Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6

openai.com

Post reply on HN
Print view View original

801–810 of 1001 posts

Re: GPT-5.6

#801

Post by andai »

ComputerGuru wrote: post #737
tshaddox wrote: post #485

Earlier quoted context omitted.

If you conceptualize this as “there is an appropriate amount of brevity for each situation” then it would be expected for a better model to use different amounts of brevity if it gets better at determining the appropriate amount. My view is that popular models by default output wildly excessive amounts of prose for nearly every use case, so if this changes in a new model that’s a pure win.

> wildly excessive amounts of prose Not just prose. I think this is part of the reason why you see ridiculous code with insane error handling and type checking even for impossible cases.

This is one reason I switched back to Claude after testing various alternatives a few months ago. Claude ended up writing much more elegant code.

Although I was surprised that I could get very Claude like results from Chinese models though by just telling it to make the code elegant.

Reminds me of the old days with art AI where you had to put "+good -bad" in the prompt because otherwise it would assume you just wanted random quality outputs, because it had been trained on random quality inputs...

Re: GPT-5.6

#802

Post by andai »

10xDev wrote: post #234
meetpateltech wrote: post #154

GPT-5.6 Sol sets a new SOTA on ARC-AGI-3: 7.8% Sol is the first verified frontier model to ever beat an ARC-AGI-3 game https://arcprize.org/results/openai-gpt-5-6

Seeing the dramatic differences in scores just going from high to xhigh is just another demonstration of the bitter lesson: Just keep scaling search and learning. We are probably going to need a lot more GPUs.

I said a few months ago, "man, Opus is great, but sometimes when talking with it I have the feeling like, this thing should be about 10 times bigger."

When Mythos was announced after that, I was pleasantly surprised to hear about it. But when it turned out to be only two times bigger, I was a little disappointed!

(I am even more disappointed with the safety filters, but that's kind of a separate discussion... "Fortunately" I find that I can usually edit my prompt by single character and get through...)

Re: GPT-5.6

#803

Post by baq »

ryandvm wrote: post #399
Razengan wrote: post #279

Earlier quoted context omitted.

> We are probably going to need a lot more GPUs. Or a breakthrough in algorithms etc. The human brain, heck all bio brains, are proof that you don't need a lot of power or size for intelligence.

The human brain has 80 billion neurons and a 100 trillion synapses. I think you're underselling the processing power of that warm chunk of meat. The real message of the last 15 years has actually been the opposite: if you throw enough processing power at it, intelligence emerges.

It’s still 20W. We have living proof what is possible within 20W. The message has always been clear - try to get silicon computer to be as power efficient as the brain is as it is obviously possible.

Re: GPT-5.6

#804

Post by andai »

snthpy wrote: post #793
bevekspldnw wrote: post #331

Earlier quoted context omitted.

These aren’t raw base models they are the result of a ton of RLHF and various adjustments. Bitter lesson wildly overstated in this context.

Not just RLHF but also RLVR, and isn't that the litter lesson though? My sense of the Sutton Dwarkesh interview was that he was calling out that he didn't mean just longer datasets, but rather learning through exploration and that's exactly RL.

They just need more contact with reality. That's what RL is right? Contact with narrow subsets of reality.

Re: GPT-5.6

#805

Post by senko »

ciefa wrote: post #516
senko wrote: post #439

I love testing the new models by asking them to code a toy RTS game. Here's what Terra did: https://senko.net/vibecode-bench/2026/rts-gpt-5.6-terra.html (one try, in codex app, xhigh effort) Comparing this to other models, I find it similar to GPT-5.5 and a bit behind Sonnet 5. You can see how other models fared here: https://senko.net/vibecode-bench/ (you can also fetch the prompt and the the 5.6 Terra resulting cod…

Very neat! Can't wait for the Sol version!

Here it is: https://senko.net/vibecode-bench/2026/rts-gpt-5.6-sol.html

Clearly much better than the Terra version. I'd say its on par with Fable, and the observed differencies are more due to random luck and open-ended prompt, rather than model capability. (Edit: after some more testing, perhaps not on par - somewhere between Opus and Fable, is a better description).

Fable did better pathfinding and has more terrain variety, visually the map looks better, especially soft edges of the fog of war. And the enemies.

Sol took more care with tiny ux details, added help, and more building varieties.

Re: GPT-5.6

#806

Post by fodkodrasz »

elAhmo wrote: post #230
minimaxir wrote: post #26

The developer's guide ( https://developers.openai.com/api/docs/guides/latest-model ) has some interesting semantic tips for using the model: > Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly. > Original image detail: GPT-5.6 preserves the…

> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%. A shorter prompt results in half as much tokens spend? I find this very hard to believe.

So the user must be concise, but cannot ask the model to be concise... because it hurts the model...

Re: GPT-5.6

#807

Post by andai »

XCSme wrote: post #601
10xDev wrote: post #234

Earlier quoted context omitted.

Seeing the dramatic differences in scores just going from high to xhigh is just another demonstration of the bitter lesson: Just keep scaling search and learning. We are probably going to need a lot more GPUs.

Not always, in some cases, changing to a higher reasoning makes the AI doubt itself too much, and skip over the correct answer by overcomplicating the problem and polluting the context. It would be nice to see on which categories of problems the extra thinking makes it better and on which it makes it worse.

I think I have this problem but with my human brain.

Re: GPT-5.6

#808

Post by ainch »

vatsachak wrote: post #247
osti wrote: post #226

Earlier quoted context omitted.

Mythos probably wouldn't, otherwise they'd have included it in their release. Next version of Mythos probably will though. And yeah.. Reality has not been kind to LeCun.

Are you joking? They spend billions of dollars training LLMs to get a 7.8% on arc agi 3 whereas DINO models are near sota in image classification, provide meaningful embeddings to the point where image segmentation is just PCA. The spend on DINO cannot be more than five million (correct me if I'm wrong) JEPA is just getting started

Yann is a big SSL guy but I don't think he was involved in the original DINO - he's not listed as a co-author or anything.

Re: GPT-5.6

#809

Post by andai »

drdrey wrote: post #761
monk_grilla wrote: post #675

Earlier quoted context omitted.

This is the first I have herd of this benchmark. Can someone explain how it in any way indicates how close we are to "AGI"? Replay of Sol attempting the game: https://arcprize.org/replay/83543d22-8e1e-439a-8809-129ff1d9... It seems a weird and arbitrary challenge for a language model to be expected to perform. It also seems like there are some harness/visual issues even in the first few steps, where it states that it…

the problems are general and abstract, domain specific knowledge and memorization don't help. Figuring out the rules, the goal, the controls, and how to solve in a reasonable budget all indicate some level of general ability.

So basically they're well suited for like, an octopus or a crow?

I was thinking about those species earlier in the context of, what does intelligence mean outside of language.

The benchmark appears to be testing the same thing. Although I don't know how much transfer there would be between this data set and the kind of situations a crow or an octopus would encounter.

Edit: Huh, it's just a Game boy game? I just did a couple of the tasks. It looks like C64 era game to me. Navigating levels. A lot of overlap with animal intelligence then.

Re: GPT-5.6

#810

Post by davedx »

roncesvalles wrote: post #791
ravenstine wrote: post #94

Earlier quoted context omitted.

> Avoid generic brevity instructions That part is confusing because it's not like they provide an example of how default GPT-5.6 output compares with GPT-5.5 both with default output and prompted for brevity. Whenever I use such prompts, it's usually because I want the model to give me the gist in a few sentences. I'd be stunned if GPT-5.6 was that concise by default. I would think that could "break" a lot of things…

I think it is widely known by now that instructions to alter the LLM's "tone", things like asking it to adopt a persona ("you are the world's best programmer"), and overly broad directives ("make no mistakes") always gives poor results. Just state directly what you want. If you want something very specific, add more information. "Prompt engineering" is pseudoscience. To put it another way, you will only get the bench…

It's a bit more nuanced than that. Earlier models definitely benefited a lot more from prompt engineering. I remember this distinctly from building data pipelines to do things like extract data from PDFs over the last year or two - there are numerous "tricks" like negative prompting, including the right number of examples, massaging the mock data in the JSON examples so it wasn't "too realistic", and so on. I saw how this impacted recall by running evals, so it wasn't pseudoscience.

But what has happened is the models have gotten better - which OpenAI is making explicit for some cases in this release. You need that stuff less and less as they become more human and better at inferring what's required implicitly.

You still do need to be explicit, and you probably always will, but you don't need as much "engineering" of the way you're asking for things with more recent models.

Post reply on HN