Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerating GPT-5.6 Sol Ultrafast

cerebras.ai

Post reply on HN
Print view View original

191–200 of 295 posts

Re: Accelerating GPT-5.6 Sol Ultrafast

#191

Post by fragmede »

macNchz wrote: post #125
csallen wrote: post #103

People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…

This is foundationally similar to a lesson I've found from years of pre-LLM software development: builds that turn around in 500ms instead of 5 minutes fundamentally change the way you can work as a software engineer. I think a lot of the same applies to working with LLMs. I'm not sure, though, what the path from where we are today to some future state of high speed token abundance actually looks like...I think there…

There was one point in my career where the setup took a couple of hours so I'd only get two, maybe three chances per day. I reviewed the code very carefully before I hit send.

Re: Accelerating GPT-5.6 Sol Ultrafast

#193

Post by rokob »

TranAndrewA wrote: post #189
dannyw wrote: post #169

Earlier quoted context omitted.

We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company. The amount of workloads we can shift with an advisor model pattern continues to grow. It’s seriously amazing.

Luna came out about a month ago, you're saying that the cost saving from switching to Luna has saved your company $20 000 000+ in 1 months spending on API usage?

It makes sense at a company that sells AI as a service. Perhaps to make presentations...

Re: Accelerating GPT-5.6 Sol Ultrafast

#194

Post by bigyabai »

exabrial wrote: post #180

Right now, the "economic model" of AI is "who has the best model", or really weights. That'll go away eventually, just like operating systems eventually became free. Instead, it's going to come down to selling inference hardware. We'll likely see the "apple" model where a custom OS runs on their hardware, but we'll probably also see more things like Cerebras become commodity hardware instead of kilowatt-class datacen…

> we'll probably also see more things like Cerebras become commodity hardware instead of kilowatt-class datacenter only hardware.

Cerebras uses a unreal amount of SRAM to make these dies feasible. If AI weights become small enough to fit into commodity-scale Cerebras chips like that, you might as well load it into unified memory instead and run inference on a GPGPU-capable SOC instead. CUDA-style acceleration makes much more sense at that scale, especially if your use-case is just realtime conversational AI on a smartphone.

Re: Accelerating GPT-5.6 Sol Ultrafast

#195

Post by andai »

loufe wrote: post #108
csallen wrote: post #103

People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…

Going back between two different company's AI tools when facing a tricky architecture question often surfaces holes in an approach I'd been building. Similarly, if I ever get a bit too vibey and don't carefully review code changes myself, the blast radius is generally significantly resolved by a carefully tuned "did you consider x, y, and z" skill after a first draft partnered with a "deploy an adversarial review age…

When I was doing some important work, I had 2 worktrees, each running a frontier model from a different company, and I passed their responses between them in markdown files and had them critique them. I thought they'd prefer their own solutions, but they frequently preferred the other model's solutions.

I went back and forth like this until we converged on a solution "everyone" was satisfied with.

---

Context: I'm a solo dev and working outside my area of expertise, so I'm leaning heavily on LLMs. It's not great (definitely too vibey for my taste, and I keep running into issues) but the alternative is spending the next few years studying several specializations instead of shipping. So this is the "least bad" thing I could come up with.

I'm definitely increasingly making time for "learning sprints" to catch up on specific knowledge gaps. For example today I spent 2 hours debugging something because I was missing a fact that would have taken me 1 minute to learn...

I went from "fully hand crafted" to "fully vibed" (when Fable came out), back to "fully hand crafted" (once I realized I no longer understand the code!), and now I'm at "making very careful use of AI, asking for the smallest possible changes, and double checking everything"...

Re: Accelerating GPT-5.6 Sol Ultrafast

#196

Post by trollbridge »

andai wrote: post #120
iamcoder18 wrote: post #13

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…

I'm finding Luna suprisingly adequate for my work. I slept on it due to the benchmarks, but it's very fast and even on low reasoning I'm finding it more than adequate for "menial" work. (The speed is crucial for "interactive" work -- if a model is fast enough it goes from "async" to "real time", subjectively, which is a huge difference.) In fact, I'd say it's overqualified for the kind of work I'm doing, because it s…

Luna and Terra are great, but so are DeepSeek Flash, MiMo, Qwen 3.6, and even Gemma for some things.

MiMo Pro has had UltraSpeed for a while.

Re: Accelerating GPT-5.6 Sol Ultrafast

#197

Post by moralestapia »

moralestapia wrote: post #76
baq wrote: post #74

Earlier quoted context omitted.

Amadahl’s law dictates it’s approximately always better (as in, more efficient computationally) to have one super fast thing than many slower things doing the same job in parallel.

Such a deep and smart insight. But why is that important if they're measuring time?

And just like that, GP disappears forever ...

Re: Accelerating GPT-5.6 Sol Ultrafast

#198

Post by csallen »

bluerooibos wrote: post #157
csallen wrote: post #103

People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…

> But what if it didn't? Or what if it output the final result in the first place, without having to repeatedly prompt it to check its own work to trick it into a better answer?

Maybe having the optionality is better? I could imagine a world in which it does what you say, and people wish it didn't, so they have both the option to get the fast and slightly worse answer + the option to ask for reviews to get slower but better answers. And more control over how those reviews are performed. Which is the world we live in today.

Re: Accelerating GPT-5.6 Sol Ultrafast

#199

Post by energy123 »

csallen wrote: post #103

People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…

That's an argument for cheap massive parallelism, which is what's behind the recent math breakthroughs, not expensive low latency serial compute.

Re: Accelerating GPT-5.6 Sol Ultrafast

#200

Post by danielmarkbruce »

csallen wrote: post #103

People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…

100% this. I do some work where frontier LLMs really are quite bad on a first pass... yet with what I consider a basically silly agent architecture which sort of amounts to "do this three ways, check, revise, choose the best, rethink, check, check, revise ......" I get to remarkably good results. It was through this process I realized.... humans do this, on the fly, without thinking.

I end up spending a lot on inference, it's incredibly slow, and the architecture really does seem like overkill at first glance. But it works magic.

Post reply on HN