Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerating GPT-5.6 Sol Ultrafast

cerebras.ai

Post reply on HN
Print view View original

181–190 of 295 posts

Re: Accelerating GPT-5.6 Sol Ultrafast

#181

Post by vatsachak »

Topfi wrote: post #33
yetihehe wrote: post #23

Earlier quoted context omitted.

Maybe then LLM's will switch to outputting raw machine code?

A great future for us all to look forward to. Human readability gone, any changes must go through a providers API so cost+restrictions are a constant and we'll need far more tokens for even simple operations. Call me old fashioned, still like to understand the code I merge, especially since I've seen Fable and Sol make sometimes utterly egregious mistakes in simple, well scoped requests that erode any trust I can mus…

Yeah and sometimes even worse, they do things correctly but in most spaghetti way.

I've had to rewrite my whole codebase. It's just the thrill of getting things done quick. Not getting things done right.

Re: Accelerating GPT-5.6 Sol Ultrafast

#182

Post by giovannibonetti »

jlund-molfese wrote: post #145
ChadMoran wrote: post #142

Earlier quoted context omitted.

The comment you're replying to almost feels like it was written by a bot or am I crazy?

What kind of bot would say `less guardrails` instead of `fewer guardrails`? I guess someone could instruct an LLM to deliberately make mistakes, but isn't that too paranoid?

In some romantic languages less and fewer are the same word, so it is a common mistake for people that have them as their mother tongue.

Re: Accelerating GPT-5.6 Sol Ultrafast

#183

Post by bpbp-mango »

fg137 wrote: post #21

> allowing Sol Ultrafast to accelerate your most time-sensitive, mission-critical work Curious, what are some of the use cases?

Agents would be excellent 'on call' team members. able to log in to production and diagnose issues, fix or raise issues, post outages etc.

Re: Accelerating GPT-5.6 Sol Ultrafast

#184

Post by jgalt212 »

dannyw wrote: post #169
aetherspawn wrote: post #130

Earlier quoted context omitted.

I discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency. Dollar for tokens, Sol and Fable are the same price. However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task. We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day. And, due to less guardrails, Sol also performed be…

We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company. The amount of workloads we can shift with an advisor model pattern continues to grow. It’s seriously amazing.

what has a single company accomplished with tens of millions of token spend?

Re: Accelerating GPT-5.6 Sol Ultrafast

#185

Post by satvikpendem »

theplumber wrote: post #171
ChadMoran wrote: post #142

Earlier quoted context omitted.

The comment you're replying to almost feels like it was written by a bot or am I crazy?

Peak under your skin a bit. Something weird is going on. I think we are bots/robots(sic)

Related, Under The Skin with Scarlett Johansson is an incredible movie.

Re: Accelerating GPT-5.6 Sol Ultrafast

#186

Post by maleldil »

samatman wrote: post #138
navigate8310 wrote: post #134

Earlier quoted context omitted.

What stops the LLM not to iteratively think and expound upon before emitting the final tokens?

Nothing at all. They're not designed to, so they don't. Change that, and they would. The question is the wrong one. The right question: why aren't frontier models designed to work that way? The answer: it's slow and expensive. The other answer: that's basically what you're selecting with "Medium", "High" and so on, how many tokens they'll blow on muttering to themselves before they get back to you with an answer. The…

Reasoning models will frequently backtrack and re-assess what they've said so far. That's one reason test-time scaling is so powerful.

Re: Accelerating GPT-5.6 Sol Ultrafast

#187

Post by TacticalCoder »

perching_aix wrote: post #140
sixtyj wrote: post #72

Earlier quoted context omitted.

Output from Cerebras with GPT model is 750 tokens per second. Don’t blink. (Chatjimmy has 14,200 TPS.)

Never heard of it before, that's fucking insane. Apparently they baked the Llama 3.1 8B model weights [0] into silicon (the actual hardware is called Taalas HC1). I guess for the trillion parameter models this would not scale due to cost? Imagine buying GPT 6 in the form of a PCI-E card, pulling these speeds, with up to 120 cct agent sessions. It'd be beyond wild. [0] the weights are also using some cut down small fo…

> Never heard of it before, that's fucking insane.

They've been acquired by AMD. Those saying the model sucks are completely missing the point: it was a proof-of-concept.

The question is: what happens to a model like Anthropic's Fable 5 that does, what, 70 tokens/s (and requires lots of output tokens) when the latest open-weights model is etched on silicon and does 14 000 tokens/s?

Shall the better model still have the upper hand or will the raw speed compensate?

Re: Accelerating GPT-5.6 Sol Ultrafast

#188

Post by TacticalCoder »

Daneel_ wrote: post #168
christoff12 wrote: post #116

Earlier quoted context omitted.

An irrational gripe of mine is how GPT uses 7× instead of 7x. I recognize that the former is the multiplication symbol, but I don't think it should be used that way.

If anything, writing "7x" is actually just a poor emulation of writing "7×". Saying something is "done at 7x speed" should be read as "done at seven times speed" not as "done at seven x speed". So using the 'times' (multiplication) symbol is the better form in my opinion; it just happens to be significantly easier to type "x" instead, which is how we got here.

But speech evolves. And we all pronounce "ten ex", not "ten times". Actual speech usage very clearly pronounce the letter 'x'.

So it's 10x. And no need for Unicode codepoints.

Re: Accelerating GPT-5.6 Sol Ultrafast

#189

Post by TranAndrewA »

dannyw wrote: post #169
aetherspawn wrote: post #130

Earlier quoted context omitted.

I discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency. Dollar for tokens, Sol and Fable are the same price. However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task. We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day. And, due to less guardrails, Sol also performed be…

We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company. The amount of workloads we can shift with an advisor model pattern continues to grow. It’s seriously amazing.

Luna came out about a month ago, you're saying that the cost saving from switching to Luna has saved your company $20 000 000+ in 1 months spending on API usage?

Re: Accelerating GPT-5.6 Sol Ultrafast

#190

Post by rpdillon »

zozbot234 wrote: post #55
iamcoder18 wrote: post #13

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…

Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.

750t/s tells the story. This isn't purely (or even mostly) driven by parallelization.
Post reply on HN