171–180 of 295 posts
Earlier quoted context omitted.
Wait isn't Fable like 2x more expensive if we compare under 272k tokens
The comment you're replying to almost feels like it was written by a bot or am I crazy?
Earlier quoted context omitted.
I think it's pretty obvious that, in that world, the AIs will simply be tasked with making the compilers faster. It's already happening with their own stack, after all.
Speed is independent of quality. LLMs are currently completely incapable of producing a good compiler. I'm not convinced that's going to change (and if it does change, we're all unemployed).
Would gladly switch over to OpenAI and pay them 2x what I'm paying Claude if this becomes generally available
Fast mode is already 1.5 times faster and 2x more expensive in the Codex subscription plan. If this thing is 14 times faster, then I can imagine running out of my quota in one session.
There is zero chance this will be offered to subscription users.
Earlier quoted context omitted.
ChatJimmy is a much smaller model and, AFAIK, has no reasoning capability. Absolutely insane raw speed, like a supercar, while Sol is more like a freight truck.
The knowledge of ChatJimmy is terrible. Even Qwen on my iPhone is better.
I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a s…
Answering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.
I believe the assumption of the benchmark creators is that it's meant to measure the sequential speed that a single instance of the LLM and piece of hardware can get through all the tasks from start to finish, like running a race. As a rudimentary comparison, sort of like doing prime number calculations as a benchmark of the CPUs in one bare metal server. Of course you'd get a speedup in total number of primes searched if you ran the same software of GIMPS on 8 servers with the same hardware in parallel rather than 1 server.
Of course if you took all the individual questions in humanity's last exam and fed them in parallel into separate queries to Claude that land on separate hardware instances of the claude model you'd get a speed up. Because each question is independent and not related to knowledge/calculations that are performed in any other question it is indeed very open to speed up by breaking it into separately dispatched parallel tasks.
The amount of usage you receive on Codex these days is dismal compared to what it was a few months ago, FYI.
And they charge more for going faster.
As a Codex customer, I am not impressed with their shenanigans over the past few months and I have resolved to master the art of Pi Coding Harness creation and loving it.
Thanks for all the fish, Sam!
Earlier quoted context omitted.
The comment you're replying to almost feels like it was written by a bot or am I crazy?
What kind of bot would say `less guardrails` instead of `fewer guardrails`? I guess someone could instruct an LLM to deliberately make mistakes, but isn't that too paranoid?
That'll go away eventually, just like operating systems eventually became free.
Instead, it's going to come down to selling inference hardware. We'll likely see the "apple" model where a custom OS runs on their hardware, but we'll probably also see more things like Cerebras become commodity hardware instead of kilowatt-class datacenter only hardware.