Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerating GPT-5.6 Sol Ultrafast

cerebras.ai

Post reply on HN
Print view View original

61–70 of 295 posts

Re: Accelerating GPT-5.6 Sol Ultrafast

#61

Post by whimsicalism »

zozbot234 wrote: post #55
iamcoder18 wrote: post #13

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…

Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.

i assume they meant serially....

Re: Accelerating GPT-5.6 Sol Ultrafast

#62

Post by buybackoff »

This is something I'm ready to pay for. Not more per token, but I will be happy to burn through 20x Pro subscription as fast as I consume my Plus weekly limit now, with 10x more tokens per unit of time. I've learned how to deal with and steer Sol medium quite efficiently, but at the same time I realize it's so slow for the small tasks it can do well, and still so unreliable for open-ended tasks.

Re: Accelerating GPT-5.6 Sol Ultrafast

#63

Post by yetihehe »

bakies wrote: post #47
applfanboysbgon wrote: post #32

Earlier quoted context omitted.

No. This conveys a fundamental misunderstanding of how anything pertaining to programming works. This will never happen, ever . For example, take... printf("Hello, world"); vs. a plausible illustration of how it might be compiled down to machine code... 48 65 6C 6C 6F 2C 20 77 6F 72 6C 64 48 83 EC 28 48 8D 0D F5 0F 00 00 E8 F0 00 00 00 33 C0 48 83 C4 28 C3 The latter now takes up 10x as many tokens (= 10x the cost/ti…

I mean you're right, but what about a different tokenizer and some guardrails... trained on compiled code... I just learned that gpt-5.6 will talk to you in base64 without tool calls so maybe it's possible.

It's good enough at decoding hex from some packet dumps. And I was doing that even with 5.5. And it was good at decompiling some code (with tools) and searching for offsets of buffers and commands. Found viable exploit that allowed me to rescue broken update system in devices I was maintaining for my company (it was broken by chatgpt forgetting -v in hexdump, heh).

Re: Accelerating GPT-5.6 Sol Ultrafast

#64

Post by evmaki »

applfanboysbgon wrote: post #32
yetihehe wrote: post #23

Earlier quoted context omitted.

Maybe then LLM's will switch to outputting raw machine code?

No. This conveys a fundamental misunderstanding of how anything pertaining to programming works. This will never happen, ever . For example, take... printf("Hello, world"); vs. a plausible illustration of how it might be compiled down to machine code... 48 65 6C 6C 6F 2C 20 77 6F 72 6C 64 48 83 EC 28 48 8D 0D F5 0F 00 00 E8 F0 00 00 00 33 C0 48 83 C4 28 C3 The latter now takes up 10x as many tokens (= 10x the cost/ti…

For what it's worth, this theoretical LLM-based compiler would probably generate LLVM or some other intermediate representation rather than platform-specific machine code.

Re: Accelerating GPT-5.6 Sol Ultrafast

#65

Post by datadrivenangel »

jaggederest wrote: post #28
owentbrown wrote: post #25

Whoa. This looks both powerful and expensive. My prediction is that, this time next year, top developers outside ai labs will be spending 50k USD+ on inference. Within labs, I've heard spend is already far beyond this per developer.

I mean I don't think $50k is the ceiling, unless you're talking about actual cash out. Claude code subscriptions right now can easily clear you $25-35k a year in nominal value for $2400 out of pocket cost. Given sufficient budget and scope, I could certainly productively burn a half million dollars in tokens a year or more. I think that's where we're headed anyway, buying a 2nd or 5th claude max subscription feels sl…

at the corporate level you pay full price.

And a moderately heavy user dev can easily spend a few $K a month, so yeah. Not impossible, but a high cost, and the diminishing returns definitely kick in

Re: Accelerating GPT-5.6 Sol Ultrafast

#66

Post by stillpointlab »

I haven't wrapped my head around what level of reasoning this involves. Is it equivalent to max?

I didn't like Sol initially but it is growing on me the more I use it. Its personality is a bit flat and I caught it taking shortcuts a few times. But once I learned how to interact with it, I'm genuinely warming up to it. I find that it writes code that has fewer bugs even than Fable (although, to be fair I reach for Fable when the task is less well defined).

If this has similar performance to Sol at max reasoning level, this would be a compelling reason to shift even more of my work (maybe the majority) to this model.

Re: Accelerating GPT-5.6 Sol Ultrafast

#67

Post by UltraSane »

fcarraldo wrote: post #19
WarmWash wrote: post #17

Earlier quoted context omitted.

The stake in the side of cerebras has always been that the economics are pretty poor. Who knows if they will subsidizes it to mitigate sticker shock, but it's a safe assumption that it will be scarily expensive. However if you are in a "cost is no obstacle, speed is god" position, it will likely be pure magic.

Can anyone explain why Cerberus needs to be _fast_ instead of _cheap_? I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s. Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.

Because companies are willing to pay proportional to (tokens/s)^k for some value of k >1

Re: Accelerating GPT-5.6 Sol Ultrafast

#68

Post by imtringued »

yetihehe wrote: post #23
applfanboysbgon wrote: post #18

This kills the crab. Compilation time will be a genuine bottleneck for slop coding if this becomes the standard generation rate over the next few years. Go, Zig or even C99 with TCC for dev builds, any language that can get you systems-level performance (or close to it) in a dev environment where you can iterate in ms rather than minutes is going to be immensely more appealing than generating a potential prototype in…

Maybe then LLM's will switch to outputting raw machine code?

That still requires a specification language to constrain the generated code even if the implementation is produced directly.

Why? Because you are defining the implementation based on its observable behaviour rather than as a rule set to be followed.

Re: Accelerating GPT-5.6 Sol Ultrafast

#69

Post by UltraSane »

black_knight wrote: post #40
arw0n wrote: post #26

Earlier quoted context omitted.

What do you need speed for? That's a genuine question, I feel like the limiting factor already is my creativity, attention span and budget. And I'm not even yet optimizing cost by batching things like review to slow local models over night, or schedule tasks to take full advantage of my subscriptions.

I just finished waiting almost four hours for Fable to write 700 lines of code, based on my three paragraph prompt. Some speed on these harder tasks would definitely be welcome! It also spent almost 800k tokens on these lines…

I'm very curios what the code is doing.

Re: Accelerating GPT-5.6 Sol Ultrafast

#70

Post by datadrivenangel »

thraway3837 wrote: post #9

This is really cool. Someone here commented about similarity between this and hardware advancements for AV encode/decode. I think it's only a matter of time before miniaturization can have a thumbnail sized user-replaceable accessory that contains the LLM built onto the hardware. I admit I don't know how any of that works, but would be amazing to experience. Fully local, fully offline, ultra fast local inference bett…

In 5-10 years nice smartphones will be able to run ChatGPT (~gpt3-4) class models. A memory rich laptop (highend mac/framework) can run GPT-OSS:120b or full Gemma4 at very interactive speeds.

High end phones can already run the smaller models at enough speed to be probably useful, especially for background/overnight photo tagging and curation and things like that.

Post reply on HN