161–170 of 295 posts
Earlier quoted context omitted.
The task the majority of people here care about (and which makes up a huge part of AI lab revenue these days) is collaborative coding with a single human which is not embarrassingly parallel.
I am aware. I was talking about HLE. Talking about how fast you can run through HLE in a serial manner is like talking about how how fast your non-simd single threaded cpu ray tracer is. Its simply not how any sensible person would tackle this problem.
> Compared with output speeds reported by Artificial Analysis GPT-5.6 Sol on Ultrafast mode runs 11x faster than Fable 5, and 5x faster than Opus 4.8 on Fast mode. Awesome work. I'm personally very excited for faster models/inference. I think speed is underrated to some degree in the current conversation. For a while, I was using Cursor's Composer quite a lot, even over frontier models, just because of how darn fast…
What do you need speed for? That's a genuine question, I feel like the limiting factor already is my creativity, attention span and budget. And I'm not even yet optimizing cost by batching things like review to slow local models over night, or schedule tasks to take full advantage of my subscriptions.
well, I am.
look at what the market thinks of CPU manufacturers and general computation now that agentic workflows have taken up, all went to the moon after being picked over in favor of GPUs and RAM for years
most computers have been idling, waiting for human input, for decades, and if there was a computationally intensive process it was offloaded to GPUs a long time ago, over the last decade, so CPUs and general processors have remained idle, relegated to just defined conditional statements to switch between tasks with no reasoning capability to occupy compute
now, there are reasoning capabilities to tell a CPU what to do (as a byproduct of the varied processes). Cerebras is not a CPU, it is a special purpose chip for inference, but is hosting LLMs that tell CPUs of all its clients what to do faster than a human can. Outside of Cerebras, LLMs are not doing much to optimize compute of the system they're affecting, as they're reading or compiling code when being used for coding, very few processes are intensive and the CPU is just waiting as if a human was using it because the LLM can't digest and output information fast enough. The CPU ecosystem is very mature for general and varied tasks, but is underutilized.
To the what: any kind of compositing or configurations that humans do, agents can do. AutoCAD, video editing, sequencing in music, all forms of media, all forms of configuration done digitally. right now they rely on snapshots to see and react, and this increases the 'framerate' per say, and rapid and relentless iteration they can do.
Earlier quoted context omitted.
At such output speed, I wouldn’t expect reasoning. (But I didn’t know it, thanks.) 700 TPS with reasoning is awesome and it speeds things up. Cerebras as public traded company is worth keeping an eye what they produce.
> At such output speed, I wouldn’t expect reasoning. As the sibling comment to yours mentioned, if they had a reasoning model “hardware-ified” onto a custom chip (as is their plan for IIRC this or next year, a new ASIC), it’d output fast decode speeds for the regular output as well as reasoning sections. Both would be ≈equally fast.
Earlier quoted context omitted.
ChatJimmy is a much smaller model and, AFAIK, has no reasoning capability. Absolutely insane raw speed, like a supercar, while Sol is more like a freight truck.
The knowledge of ChatJimmy is terrible. Even Qwen on my iPhone is better.
Earlier quoted context omitted.
> At such output speed, I wouldn’t expect reasoning. As the sibling comment to yours mentioned, if they had a reasoning model “hardware-ified” onto a custom chip (as is their plan for IIRC this or next year, a new ASIC), it’d output fast decode speeds for the regular output as well as reasoning sections. Both would be ≈equally fast.
Yeah, I thought reasoning was literally just chain of thought in the output token stream, with the model itself adding delimiters to indicate what part of the output is internal reasoning, and what part is an answer to the user. Is that wrong?
Unless I have read over it, besides the animation in the intelligence vs speed graph which only mentions internal data and not whether they truly reran the AA suite, there is no actually solid statement on the important aspect of performance. Neither the Cerebras or OpenAI post [0] outright state that this performs exactly the same as regular 5.6 Sol. I feel if this was 1:1 just Sol but much faster, they'd (rightfull…
Earlier quoted context omitted.
No quality compromise/degradation is something I have had this industry, including especially OpenAI, claim multiple times in the past and I have more than once been able to verify that it was in fact not the case. Examples being gpt-3.5-turbo vs text-davinci-003, GPT-4-Turbo and all the other post training checkpoints they had under one name (which was a major bug bear for me back then witnessing degradations with n…
There's probably some quantization and other processing needed to get Sol to fit on a Celebras chip, so it's not an exact 1:1 copy of the Sol model running on Nvidia GPUs, because it's not running on GPUs doing abstract math calculations anymore, so they can run benchmarks at them and say it's similar enough, but no one believes in benchmarks in this new field of performance that's more computer astrology than comput…
I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…
An irrational gripe of mine is how GPT uses 7× instead of 7x. I recognize that the former is the multiplication symbol, but I don't think it should be used that way.
Saying something is "done at 7x speed" should be read as "done at seven times speed" not as "done at seven x speed". So using the 'times' (multiplication) symbol is the better form in my opinion; it just happens to be significantly easier to type "x" instead, which is how we got here.
I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…
I discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency. Dollar for tokens, Sol and Fable are the same price. However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task. We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day. And, due to less guardrails, Sol also performed be…
The amount of workloads we can shift with an advisor model pattern continues to grow.
It’s seriously amazing.