Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerating GPT-5.6 Sol Ultrafast

cerebras.ai

Post reply on HN
Print view View original

241–250 of 295 posts

Re: Accelerating GPT-5.6 Sol Ultrafast

#241

Post by shussjjs »

TacticalCoder wrote: post #188
Daneel_ wrote: post #168

Earlier quoted context omitted.

If anything, writing "7x" is actually just a poor emulation of writing "7×". Saying something is "done at 7x speed" should be read as "done at seven times speed" not as "done at seven x speed". So using the 'times' (multiplication) symbol is the better form in my opinion; it just happens to be significantly easier to type "x" instead, which is how we got here.

But speech evolves. And we all pronounce "ten ex" , not "ten times" . Actual speech usage very clearly pronounce the letter 'x'. So it's 10x. And no need for Unicode codepoints.

A number plus an “x” is used beyond Trumpistan. I have never in my life heard anyone say “ten ex”.

Re: Accelerating GPT-5.6 Sol Ultrafast

#242

Post by grey-area »

dannyw wrote: post #169
aetherspawn wrote: post #130

Earlier quoted context omitted.

I discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency. Dollar for tokens, Sol and Fable are the same price. However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task. We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day. And, due to less guardrails, Sol also performed be…

We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company. The amount of workloads we can shift with an advisor model pattern continues to grow. It’s seriously amazing.

What are you using it for?

Re: Accelerating GPT-5.6 Sol Ultrafast

#246

Post by literallywho »

anthonypasq wrote: post #24

I'd just like to point out that the largest model Cerebras has ever served is Kimi K2.6 which is 1T parameters, so that either means that theyve had a breakthrough on the hardware engineering side of things, or GPT-5.6 Sol is likely a lot smaller than people think. If it truly is only ~1-2T parameters, then this kinda kills 2 narratives for me. 1. all the handwringing about open source catching up via Kimi K3 (3T par…

Don't frontier labs distill their own bigger models into smaller ones? Opus 5 was probably distilled from Fable/Mythos. Since Chinese labs now have competitive models, they can distill those into their smaller version in Kimi K3.1 or something and achieve better intelligence per parameter results.

Re: Accelerating GPT-5.6 Sol Ultrafast

#247

Post by philipportner »

YetAnotherNick wrote: post #94
philipportner wrote: post #42

Earlier quoted context omitted.

You'd need hundreds of GB alone for the KV cache of each user. For something like LLama 3 405B you need ~67GB at ~130k tokens. A single CS-3 has 44GB on-chip sram. So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference. https://newsletter.semianalysis.com/p/cerebras-faster-tokens... goes quite in-depth.

LLama 3 405B had the most unoptimized kv cache usage by far. Deepseek v4 pro uses 2.4GB for the same context length[1]. [1]: https://vllm.ai/blog/2026-04-24-deepseek-v4

Good point, thanks! I haven't been keeping up with most of the new model internals.

Re: Accelerating GPT-5.6 Sol Ultrafast

#248

Post by fireant »

obmelvin wrote: post #121
andriy_koval wrote: post #52

Earlier quoted context omitted.

not sure how "top developers" are defined here, but there is huge diminishing return curve starts kicking in after $200/month price point for typical eng work.

I agree with the diminishing returns on spend, but worth noting that when on an Enterprise seat and paying API rates, I'd say that you can easily spend above 200/mo before seeing the curve begin to flatten Obviously there are a ton of ways to spend money / tokens and people have different levels of experience that will put this ceiling at very different levels for different people.

I think that we are starting to see that API inference prices for US labs are excessive and subscription prices are closer to real costs so 200/m can be realistic longer term.

Re: Accelerating GPT-5.6 Sol Ultrafast

#249

Post by aetherspawn »

w4yai wrote: post #143
ChadMoran wrote: post #142

Earlier quoted context omitted.

The comment you're replying to almost feels like it was written by a bot or am I crazy?

I agree. Weird to use and characters. Maybe written on phone, but certainly not on keyboard.

Written “on an iPhone” - yep, seems to automatically switch the quotes.

Re: Accelerating GPT-5.6 Sol Ultrafast

#250

Post by awestroke »

krzyk wrote: post #233
dannyw wrote: post #169

Earlier quoted context omitted.

We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company. The amount of workloads we can shift with an advisor model pattern continues to grow. It’s seriously amazing.

By "advisor model pattern" you mean https://claude.com/blog/the-advisor-strategy ? How do you implement that outside of claude code?

the easier way to do this, with any harness, is to use an expensive main agent that is told to delegate all code reading, writing, exploration, research etc to weaker subagents to conserve tokens. It's an inversion of the pattern but the resulting split is the same
Post reply on HN