Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Previewing GPT‑5.6 Sol: a next-generation model

openai.com

Post reply on HN
Print view View original

401–410 of 797 posts

Re: Previewing GPT‑5.6 Sol: a next-generation model

#401

Post by nyrikki »

bob1029 wrote: post #350
gandreani wrote: post #177

Easily the most interesting part of this announcement is buried in the second to last paragraph: "We're also launching GPT‑5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity." 750 tokens/s on a frontier model is going to be extremely interesting. I doubt this new vers…

At a certain rate we will be able to move towards continuous / real-time inference systems. The discrete, turn based solutions are quite confining with how they must be trained. Continuous and real-time would fundamentally alter the domain. From an information theory perspective we are still in dial-up territory with regard to the actual information rate. 750 tokens per second would be a really bad dialup connection.…

We still have the problem that auto regressive decoders are memory bound.

The new Blackwell hardware combined with TensorRT-LLM and speculative decoding consistently can hit 1,000 TPS/user barrier, comparing to closer to ~250 TPS/user (out of 10k+/TPS on the server)

Is there something I missed, this looks more like 14.4 to 56 on a 64kbps backing channel modem story. I have no doubt that there are still massive gains to be found, but they seem to be using existing constraints more efficiently, not that fios is coming.

I don’t have the budget to work on the foundational model scale, but with a draft model 10x–20x faster than target and an 60-80 acceptance rate I can see how they could promise 750/TPS (with a lot of other hard work) but I would appreciate where I should look to figure out what I am missing.

Re: Previewing GPT‑5.6 Sol: a next-generation model

#402

Post by dyauspitr »

bob1029 wrote: post #350
gandreani wrote: post #177

Easily the most interesting part of this announcement is buried in the second to last paragraph: "We're also launching GPT‑5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity." 750 tokens/s on a frontier model is going to be extremely interesting. I doubt this new vers…

At a certain rate we will be able to move towards continuous / real-time inference systems. The discrete, turn based solutions are quite confining with how they must be trained. Continuous and real-time would fundamentally alter the domain. From an information theory perspective we are still in dial-up territory with regard to the actual information rate. 750 tokens per second would be a really bad dialup connection.…

And more importantly those 10 million tokens/s should cost fractions of a penny. Tokens need to be dirt cheap so I hope they build out massive solar+battery powered data centers asap.

Re: Previewing GPT‑5.6 Sol: a next-generation model

#403

Post by b112 »

mikepurvis wrote: post #373
bob1029 wrote: post #350

Earlier quoted context omitted.

At a certain rate we will be able to move towards continuous / real-time inference systems. The discrete, turn based solutions are quite confining with how they must be trained. Continuous and real-time would fundamentally alter the domain. From an information theory perspective we are still in dial-up territory with regard to the actual information rate. 750 tokens per second would be a really bad dialup connection.…

Is there anyone exploring or writing about this in public? I've felt for a while that the turn-based model was not quite right, but also felt too stupid and ill-informed to have much of an opinion about what else it could be.

I have an active 'sleep' mode, where when the user is AFK the LLM goes into a loop with a sleep 10 between turns, and determines (via tool use) if something should be done. That's still a 'turn' in a way, but it's all the LLM just sort of sitting around like a human would, pondering what to do next.

But I could imagine after each space(eg, word) having a 27b model on a nice rig, with thinking off, doing a quick look at the sentence and determine if it should interrupt and start a real turn with thinking on. Which kind of is non-turn based in a way. If you're typing fast, it might hit that run every 3 or 4 words, but that's sort of how a human might be when a person is talking to them. That is, waiting for enough info to interrupt, if needed.

There might be a way to process chunks of a sentence using commas as break points, eg for comma delimitated phrases in sentences, so the whole sentence doesn't need to be re-processed each "should I break in" assessment at word break.

Could be fascinating. Could actually do some of this right now.

I don't think this is what the parent poster was thinking, but the idea even at this level seems fun.

Re: Previewing GPT‑5.6 Sol: a next-generation model

#404

Post by girvo »

paxys wrote: post #216
tontinton wrote: post #186

Earlier quoted context omitted.

Yep this is a glimpse into the future of 500+ t/s, which is in my opinion the next big thing that validates Jevon's paradox (the models are already smart enough)

Faster tokens = more reasoning loops, so it can actually make the models smarter as well.

Yeah! So at a much smaller scale, being able to boost Step 3.7 Flash up to 40tk/s on my Spark-alike with proper triple head MTP was the thing that made it superior to Qwen 3.6 27B in wall clock time despite Step reasoning more

A lot of the open Chinese models get their results through huge reasoning loops. Being able to boost decode perf is what will make them worth it, and I’m sure OpenAI and Anthropic could do similar (if they aren’t already)

Re: Previewing GPT‑5.6 Sol: a next-generation model

#405

Post by gck1 »

enraged_camel wrote: post #326
gck1 wrote: post #162

Earlier quoted context omitted.

If it's anything like ClaudeCode's ultracode, it's nothing new or revolutionary. It's essentially a bunch of subagents being called by a deterministic script written by the main model thread, each eating tokens for lunch and output of which is synthesized by an orchestrator agent.

>> If it's anything like ClaudeCode's ultracode, it's nothing new or revolutionary. OpenAI flat out copying Anthropic is a pretty funny development. It's strong evidence that they've been in catch-up mode.

Eh, pretty much everyone that spent some time tweaking their harness already had a homemade 'ultracode' long before Anthropic did it.

OpenAI is just way more careful with what features they add or enable by default in their harness. Anthropic's harness is a junk drawer of random features, with a new feature added every few hours. It feels like they're in panic mode, dropping random things to see what sticks when models are eventually commoditized.

I prefer OpenAI way - slow and steady.

Re: Previewing GPT‑5.6 Sol: a next-generation model

#406

Post by nubg »

A question I always have is, how to the AI labs safeguard the leak of their model? Training a cutting edge model basically cost a minimum of hundreds of millions of dollars. And its all contained within a file. Okay, that file might be 500GB large, but its still just one blob that is worth almost a billion dollars. And they need to train new models every few weeks, have lots of people with access to it to debug it, run inference etc. I wonder when we will see the first leaks? Imagine if e.g. Opus 4.8 got leaked. Wouldnt that bankrupt Anthropic?

Re: Previewing GPT‑5.6 Sol: a next-generation model

#407

Post by seekup »

markrages wrote: post #372
rasengan wrote: post #363

Earlier quoted context omitted.

To be fair, whenever I join a pre-existing code-base [1], it's the same. I have no idea and have to map it out ;) [1] Not AI codebases (and of course, AI code bases I guess)

I can't be the only one whose memory is so bad that I am like this in my own code base.

I seem to remember - but cannot find, even with an AI boost - someone's "law of computing" or somesuch describing the amount of time that has to pass before code you wrote is indistinguishable to you from code written by someone else. At any rate the interval is not so long.

Re: Previewing GPT‑5.6 Sol: a next-generation model

#408

Post by pipes »

markrages wrote: post #372
rasengan wrote: post #363

Earlier quoted context omitted.

To be fair, whenever I join a pre-existing code-base [1], it's the same. I have no idea and have to map it out ;) [1] Not AI codebases (and of course, AI code bases I guess)

I can't be the only one whose memory is so bad that I am like this in my own code base.

You are not. :) my memory is disturbingly fried.

Re: Previewing GPT‑5.6 Sol: a next-generation model

#409

Post by hhh »

jumploops wrote: post #327

If you used GPT-5.5 over the last 24 hours or so, you may have already had access to 5.6. I've been running some tests on a harness we're building, and suddenly saw a jump in a few points yesterday. I reran the vanilla codex benchmark and saw an ~88% score on Terminal Bench 2.1 from GPT-5.5 on vanilla Codex. The biggest indicator, beyond the score, was that 3 tests which frequently hit "safety" blockers with 5.5 star…

these things can just change with infrastructure changes rather than be some mysterious A/B testing.

Re: Previewing GPT‑5.6 Sol: a next-generation model

#410

Post by jonp888 »

tedsanders wrote: post #45
low_tech_punk wrote: post #16

all the emphasis on cyber security. feels like a reaction to anthropic, not a real next generation.

Yeah, we'll share a lot more details and evals when we can release GPT-5.6 widely. We focused on cyber (and bio) here to help explain why it's being held back for now. We would have loved to launch it to everyone - it's the best coding model I've ever used - and we plan to do so as soon as we can ('coming weeks'). (I work at OpenAI.)

So now have to be worried that I'm going to killed by an AI designed nerve agent that someone has cooked up in their shed?

FFS. I hate this world so much. I wish I could just flip a switch and never have to hear about or have anything to do with AI ever again.

Do you ever stop to think about the horrific dystopia you and your acolytes are creating?

Post reply on HN