Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerating GPT-5.6 Sol Ultrafast

cerebras.ai

Post reply on HN
Print view View original

91–100 of 295 posts

Re: Accelerating GPT-5.6 Sol Ultrafast

#91

Post by tristanMatthias »

auspiv wrote: post #83
tristanMatthias wrote: post #80

> GPT-5.6 Sol on Ultrafast mode, delivering up to 750 output tokens per second https://taalas.com/products/ > delivering 17k tokens per second per user on Llama 3.1 8B model. Obviously this is a much smaller model, but I really can't wait for ASICs to take over the LLM space. Imagine running a model like Sol/Fable (even half the size with 60-70% of it's intelligence) on your own ASIC hardware.

I'd take qwen3.6 (3.8 as of tomorrow) 27B running at 17k per second first on the way to Sol/Fable! And then dsv4-flash-0731!

Right! Not sure that will happen since Talaas was just bought by AMD, so more likely a US model. Still, we can dream.

Re: Accelerating GPT-5.6 Sol Ultrafast

#92

Post by notfromhere »

tomrod wrote: post #77
sixtyj wrote: post #72

Earlier quoted context omitted.

Output from Cerebras with GPT model is 750 tokens per second. Don’t blink. (Chatjimmy has 14,200 TPS.)

ChatJimmy is a much smaller model and, AFAIK, has no reasoning capability. Absolutely insane raw speed, like a supercar, while Sol is more like a freight truck.

Anything will be fast if you etch it straight to silicon

Re: Accelerating GPT-5.6 Sol Ultrafast

#93

Post by johnfn »

This does look pretty incredible, but don't forget that incredible token thoroughput can only necessarily solve certain bottlenecks. If your e2e tests take an hour, they'll still take an hour after Ultracode. If the agent runs a 10 minute typecheck after a change, that will still take 10 minutes. grep over a massive codebase is still just as slow, etc. I say this not to take away from this accomplishment but just to ensure everyone here keeps a clear head about what it means - 14x faster tokens does not mean it completes every task 14x faster.

I suspect Humanity's Last Exam is without tool-calls, making it kind of the perfect benchmark to highlight how fast Ultrafast is, but not really the same as the everyday work you or I do.

Re: Accelerating GPT-5.6 Sol Ultrafast

#94

Post by YetAnotherNick »

philipportner wrote: post #42
fcarraldo wrote: post #19

Earlier quoted context omitted.

Can anyone explain why Cerberus needs to be _fast_ instead of _cheap_? I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s. Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.

You'd need hundreds of GB alone for the KV cache of each user. For something like LLama 3 405B you need ~67GB at ~130k tokens. A single CS-3 has 44GB on-chip sram. So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference. https://newsletter.semianalysis.com/p/cerebras-faster-tokens... goes quite in-depth.

LLama 3 405B had the most unoptimized kv cache usage by far. Deepseek v4 pro uses 2.4GB for the same context length[1].

[1]: https://vllm.ai/blog/2026-04-24-deepseek-v4

Re: Accelerating GPT-5.6 Sol Ultrafast

#95

Post by rvz »

iamcoder18 wrote: post #13

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…

Been waiting since Cerebras-GPT. [0]

[0] https://news.ycombinator.com/item?id=35490837

Re: Accelerating GPT-5.6 Sol Ultrafast

#96

Post by [deleted] »

johnfn wrote: post #93

This does look pretty incredible, but don't forget that incredible token thoroughput can only necessarily solve certain bottlenecks. If your e2e tests take an hour, they'll still take an hour after Ultracode. If the agent runs a 10 minute typecheck after a change, that will still take 10 minutes. grep over a massive codebase is still just as slow, etc. I say this not to take away from this accomplishment but just to…

[deleted]

Re: Accelerating GPT-5.6 Sol Ultrafast

#97

Post by mNovak »

tristanMatthias wrote: post #80

> GPT-5.6 Sol on Ultrafast mode, delivering up to 750 output tokens per second https://taalas.com/products/ > delivering 17k tokens per second per user on Llama 3.1 8B model. Obviously this is a much smaller model, but I really can't wait for ASICs to take over the LLM space. Imagine running a model like Sol/Fable (even half the size with 60-70% of it's intelligence) on your own ASIC hardware.

ASIC makes it sound like it's a single chip, but in reality serving trillion-param models on Cerebras requires a full cluster (as in multiple racks, MW of power).

Some interesting twitter analysis here:

https://x.com/bleysg/status/2073937651150029084

Re: Accelerating GPT-5.6 Sol Ultrafast

#98

Post by sscaryterry »

piyh wrote: post #16
iamcoder18 wrote: post #13

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…

Feels like the 90's again where single threaded speed is improving fast. ASICs and wafer scale rather than node shrinks, but end result to me the consumer feels the same.

Yep, https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law

Re: Accelerating GPT-5.6 Sol Ultrafast

#99

Post by fragmede »

fcarraldo wrote: post #19
WarmWash wrote: post #17

Earlier quoted context omitted.

The stake in the side of cerebras has always been that the economics are pretty poor. Who knows if they will subsidizes it to mitigate sticker shock, but it's a safe assumption that it will be scarily expensive. However if you are in a "cost is no obstacle, speed is god" position, it will likely be pure magic.

Can anyone explain why Cerberus needs to be _fast_ instead of _cheap_? I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s. Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.

To use a car analogy, they built. Ferrari and technically you could put a tow hitch on it and go slow and carry a bunch of cargo instead of 1 passenger and go real fast, that's just not what they've built. There's one really fast core to send tokens at and get tokens back out at insane speeds.

Re: Accelerating GPT-5.6 Sol Ultrafast

#100

Post by msdz »

sixtyj wrote: post #85
tomrod wrote: post #77

Earlier quoted context omitted.

ChatJimmy is a much smaller model and, AFAIK, has no reasoning capability. Absolutely insane raw speed, like a supercar, while Sol is more like a freight truck.

At such output speed, I wouldn’t expect reasoning. (But I didn’t know it, thanks.) 700 TPS with reasoning is awesome and it speeds things up. Cerebras as public traded company is worth keeping an eye what they produce.

> At such output speed, I wouldn’t expect reasoning.

As the sibling comment to yours mentioned, if they had a reasoning model “hardware-ified” onto a custom chip (as is their plan for IIRC this or next year, a new ASIC), it’d output fast decode speeds for the regular output as well as reasoning sections. Both would be ≈equally fast.

Post reply on HN