Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerating GPT-5.6 Sol Ultrafast

cerebras.ai

Post reply on HN
Print view View original

251–260 of 295 posts

Re: Accelerating GPT-5.6 Sol Ultrafast

#251

Post by cl3misch »

csallen wrote: post #103

People underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop…

> In fact, one could almost imagine it baked into the assistant/harness

...it already is, via thinking/effort. That wouldn't be possible if a LLM at usable quality wasn't fast enough to allow at least some amount of "thinking" (i.e. hidden text generation).

But your point still stands: we could get massive quality gains by allowing even more thinking by default, if it was fast enough.

Re: Accelerating GPT-5.6 Sol Ultrafast

#252

Post by namibj »

ashdksnndck wrote: post #221
TranAndrewA wrote: post #189

Earlier quoted context omitted.

Luna came out about a month ago, you're saying that the cost saving from switching to Luna has saved your company $20 000 000+ in 1 months spending on API usage?

OpenAI and Anthropic are believed to be making billions per month in revenue selling tokens to enterprises. It’s not that surprising for enterprise SaaS they would have individual customers making up ~1% of their total sales. Although, perhaps a bit more surprising those few customers are posting about it here. Another interpretation would this is a counterfactual savings, like they previously paid $1M for y tokens,…

A second angle on the counterfactual savings would be Luna telling them not to pursue a potential session with the expected/extrapolated (from which sessions they did ignore Luna on, e.g. just to keep efficiency statistics current) sunk costs at time of getting shut down used to derive the quoted number.

Re: Accelerating GPT-5.6 Sol Ultrafast

#253

Post by munksbeer »

andai wrote: post #120
iamcoder18 wrote: post #13

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…

I'm finding Luna suprisingly adequate for my work. I slept on it due to the benchmarks, but it's very fast and even on low reasoning I'm finding it more than adequate for "menial" work. (The speed is crucial for "interactive" work -- if a model is fast enough it goes from "async" to "real time", subjectively, which is a huge difference.) In fact, I'd say it's overqualified for the kind of work I'm doing, because it s…

I'm considering a home project done in Rust. I've not even touched Rust before, I'm a Java dev. It'll be something reasonably complicated, using a bit of unsafe etc. Do you know how Luna does writing Rust code?

Re: Accelerating GPT-5.6 Sol Ultrafast

#254

Post by namibj »

orbital-decay wrote: post #218
woadwarrior01 wrote: post #203

Earlier quoted context omitted.

Evidently, Claude's tokenizer vocabulary size is ~15k[1]. On one hand, it's quite mind blowing. On the other hand, Anthropic models' token (in)efficiency makes a lot of sense in that light. [1]: https://xcancel.com/magikarp_tokens/status/20878591737488549...

Not just that, they normalize everything into lowercase and use a special character to capitalize words (what about languages with non-trivial normalization/capitalization?) and mark beginning and end of each word, all of that diluting already small vocabulary. That smells like manual tuning of what should be done statistically, I wonder what technical merit they saw in that - I know they mentioned better generalizat…

Sadly it's not too counterintuitive; remember the old "how many r's are in the word strawberry"?

Also different tokens for the same named entity/concept if they almost entirely exclusively occur in non-overlapping contexts, and are themselves rare/uncommon in the first place, will result in behavior that's similar to the speech/phrasing/vocabulary registers humans exhibit, where the aspects of the named entity/concept get largely compartmentalized.

The most severe case along these lines were the old BERT models that ran over straight UTF-8 bytes (plus a handful special tokens).

But for the modern post-GPT2 LLMs such radical simplicity seems to mostly not be considered suitable. Note that CJK (the big one in particular, so Chinese semantic and Japanese Kanji) encodes each one into multiple UTF-8 bytes giving some automatic scaling for semantically dense languages; similar effects also apply to e.g. APL code.

Re: Accelerating GPT-5.6 Sol Ultrafast

#255

Post by aetherspawn »

ChadMoran wrote: post #142
minraws wrote: post #135

Earlier quoted context omitted.

Wait isn't Fable like 2x more expensive if we compare under 272k tokens

The comment you're replying to almost feels like it was written by a bot or am I crazy?

beep boop, everyone thinks I’m a robot. :|

Re: Accelerating GPT-5.6 Sol Ultrafast

#256

Post by anentropic »

krzyk wrote: post #226
iamcoder18 wrote: post #13

I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…

Isn't ultrafast just making hundreds of subagents?

No, that's Claude "ultracode"

Re: Accelerating GPT-5.6 Sol Ultrafast

#257

Post by eru »

giovannibonetti wrote: post #182
jlund-molfese wrote: post #145

Earlier quoted context omitted.

What kind of bot would say `less guardrails` instead of `fewer guardrails`? I guess someone could instruct an LLM to deliberately make mistakes, but isn't that too paranoid?

In some romantic languages less and fewer are the same word, so it is a common mistake for people that have them as their mother tongue.

In German the distinction is also different from what English speakers sometimes make.

Re: Accelerating GPT-5.6 Sol Ultrafast

#258

Post by eru »

modeless wrote: post #107
xyzsparetimexyz wrote: post #78

Earlier quoted context omitted.

Yes. but for embarrassingly parallel tasks it barely matters.

The task the majority of people here care about (and which makes up a huge part of AI lab revenue these days) is collaborative coding with a single human which is not embarrassingly parallel.

Maybe not embarrassingly parallel, but can be surprisingly parallel.

Re: Accelerating GPT-5.6 Sol Ultrafast

#259

Post by eru »

andai wrote: post #195
loufe wrote: post #108

Earlier quoted context omitted.

Going back between two different company's AI tools when facing a tricky architecture question often surfaces holes in an approach I'd been building. Similarly, if I ever get a bit too vibey and don't carefully review code changes myself, the blast radius is generally significantly resolved by a carefully tuned "did you consider x, y, and z" skill after a first draft partnered with a "deploy an adversarial review age…

When I was doing some important work, I had 2 worktrees, each running a frontier model from a different company, and I passed their responses between them in markdown files and had them critique them. I thought they'd prefer their own solutions, but they frequently preferred the other model's solutions. I went back and forth like this until we converged on a solution "everyone" was satisfied with. --- Context: I'm a…

> I thought they'd prefer their own solutions, but they frequently preferred the other model's solutions.

I suspect that's downstream from their sycophancy.

Re: Accelerating GPT-5.6 Sol Ultrafast

#260

Post by davidmurdoch »

I don't even have "Priority" (2.5x speed) in my Pro account, only Standard and Fast (1.5, speed). I actually didn't even know their was a level above Fast till now. I don't imagine non enterprise users will get the "UltraFast" mode option why time soon.
Post reply on HN