Why did OpenAI partner with Cerebras when they've already built their own chip, Jalapeño?
Accelerating GPT-5.6 Sol Ultrafast
131–140 of 295 posts
Re: Accelerating GPT-5.6 Sol Ultrafast
#132Unless I have read over it, besides the animation in the intelligence vs speed graph which only mentions internal data and not whether they truly reran the AA suite, there is no actually solid statement on the important aspect of performance. Neither the Cerebras or OpenAI post [0] outright state that this performs exactly the same as regular 5.6 Sol. I feel if this was 1:1 just Sol but much faster, they'd (rightfull…
"delivering up to 750 output tokens per second and without any quality compromise" seems pretty definitive.
It’s suspicious that they repeat “without any quality compromise” instead of just outright saying it’s the exact same model. It’s like a motel that insists they have clean beds.
Re: Accelerating GPT-5.6 Sol Ultrafast
#133I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…
Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.
[deleted]
Re: Accelerating GPT-5.6 Sol Ultrafast
#134People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…
What stops the LLM not to iteratively think and expound upon before emitting the final tokens?
Re: Accelerating GPT-5.6 Sol Ultrafast
#135I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…
I discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency. Dollar for tokens, Sol and Fable are the same price. However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task. We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day. And, due to less guardrails, Sol also performed be…
Wait isn't Fable like 2x more expensive if we compare under 272k tokens
Re: Accelerating GPT-5.6 Sol Ultrafast
#136Earlier quoted context omitted.
why intel and amd ? these are cerebras wafers? i know people are joking about the sol ultrafast prices (its unlikely to be accessible for average joes) but this shows scaling wafer cores works for inference boost which makes me very excited, sol ultrafast will be as slow as it will get if that makes sense. at these token speeds , we will see a much deeper economic impact.
Right now the bottle neck is not the CPU, so people aren‘t spending big $ on them. But with this ultra fast mode, CPU becomes a bigger part of the bottleneck and thus Intel and AMD can charge more $$$.
I noticed it myself. It's not just that the models are faster, but that by becoming more capable, they can take on larger tasks, which require larger compute.
Re: Accelerating GPT-5.6 Sol Ultrafast
#137Earlier quoted context omitted.
"delivering up to 750 output tokens per second and without any quality compromise" seems pretty definitive.
It’s suspicious that they repeat “without any quality compromise” instead of just outright saying it’s the exact same model. It’s like a motel that insists they have clean beds.
Or an 'asbestos free!' label on a cereal box
Re: Accelerating GPT-5.6 Sol Ultrafast
#138People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…
What stops the LLM not to iteratively think and expound upon before emitting the final tokens?
Nothing at all. They're not designed to, so they don't. Change that, and they would.
The question is the wrong one. The right question: why aren't frontier models designed to work that way? The answer: it's slow and expensive.
The other answer: that's basically what you're selecting with "Medium", "High" and so on, how many tokens they'll blow on muttering to themselves before they get back to you with an answer. There's more to it, but not that much more.
Re: Accelerating GPT-5.6 Sol Ultrafast
#139I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…
Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.
Presumably when they're talking about benchmarking, they're using a fixed amount of resources, like everyone else does when benchmarking?
Re: Accelerating GPT-5.6 Sol Ultrafast
#140I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…
Output from Cerebras with GPT model is 750 tokens per second. Don’t blink. (Chatjimmy has 14,200 TPS.)
Never heard of it before, that's fucking insane.
Apparently they baked the Llama 3.1 8B model weights [0] into silicon (the actual hardware is called Taalas HC1).
I guess for the trillion parameter models this would not scale due to cost? Imagine buying GPT 6 in the form of a PCI-E card, pulling these speeds, with up to 120 cct agent sessions. It'd be beyond wild.
[0] the weights are also using some cut down small format, but HC2 will have regular FP4 supposedly, and support for 20B params on one die