Earlier quoted context omitted.
Opus 4.5 ~= Opus 4.6 high. Opus 4.5 was nerfed just before or after the release of 4.6.
The models don’t change.
Only nominally...
$500 GPU outperforms Claude Sonnet on coding benchmarks
191–200 of 311 posts
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#192Earlier quoted context omitted.
On paper. There's huge financial incentive to quantize the crap out of a good model to save cash after you've hooked in subscriptions.
And there’s an incentive to publish evidence of this to discourage it, do you have any?
Anybody with more than five years in the tech industry has seen this done in all domains time and again. What evidence you have AI is different, which is the extraordinary claim in this case...
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#193Earlier quoted context omitted.
Models aren't just big bags of floats you imagine them to be. Those bags are there, but there's a whole layer of runtimes, caches, timers, load balancers, classifiers/sanitizers, etc. around them, all of which have tunable parameters that affect the user-perceptible output.
There really always is a man behind the curtain eh?
Often it's literally just that:
https://www.msn.com/en-us/money/other/ai-startup-backed-by-m...
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#194Earlier quoted context omitted.
The models don’t change.
I think the conspiracy theories are silly, but equally I think pretending these black boxes are completely stable once they're released is incorrect as well.
No conspiracy theories. Companies being scumbags, cutting corners, and doctoring benchmarks while denying it. Happens since forever.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#195Earlier quoted context omitted.
"There's no such thing as astroturfing." ok I use Codex regularly and Claude is shit in comparison, from its constant "Oops you're right!!" backtracking to its crap Electron app (if their AI is so good why can't they make a fucking native app for each OS?) Hell right freakin now I asked it to implement something and got a weird "Something went wrong" API error
"Shit", "Crap", "Fucking", "Hell", "Freaking". Maybe you're too easily frustrated. Or your existing code reads like your comments.
Maybe you haven't tried any other AI product with an actual preexisting project. Or blindly trust every BS Claude feeds you.
I haven't had any such frustrations with Codex
Claude is specially annoying because of their submarining and people thinking it's the best
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#196Earlier quoted context omitted.
Yup, they do quite poorly on random non-coding tasks: https://aibenchy.com/compare/minimax-minimax-m2-7-medium/moo...
It’s worth also comparing Qwen 3.5, it’s a very strong model. Different benchmarks give different results, but in general Qwen 3.5, GLM 5, and Kimi K2.5 are all excellent models, and not too far from current SOTA models in capability/intelligence. In my own non-coding tests, they were better than Gemini 3.1 flash. They’re comparable to the best American models from 6 months ago.
[flagged]
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#197Earlier quoted context omitted.
Why is that? The $200 per month subscription comes with a ton of usage. Opus 4.6 is available on the $20 plan too
> The $200 per month subscription comes with a ton of usage. $200 dollars + VAT is half of my rent. I know HN is not a good place to rant on this subject, but I'm often flabbergasted about the number of people here that lives in a bubble with regard to the price of tech. Or just prices in general. I remember someone who said a few years ago (I'm paraphrasing): "You could just use one of the empty room in your house!"…
In the US/Western Europe? Because for devs especially in the former, $200 is pocket change, especially for a core productivity tool. And the rent would be in the $1200 to $3000 easily. Same for houses. Maybe not in NY or SF, but in most of the US there's no shortage of house spaces and redundant rooms.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#198Earlier quoted context omitted.
"Shit", "Crap", "Fucking", "Hell", "Freaking". Maybe you're too easily frustrated. Or your existing code reads like your comments.
Maybe you haven't tried any other AI product with an actual preexisting project. Or blindly trust every BS Claude feeds you. I haven't had any such frustrations with Codex Claude is specially annoying because of their submarining and people thinking it's the best
I use both, read what I need to read and fix small issues myself. Both Agents are pure magic and none of their issues warrant a tantrum on a public forum.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#199Earlier quoted context omitted.
Work pays.
I'm not sure I've correctly understood what you're implying. If it's that I'm not working, well, I'm employed. It it's that I'm not working enough to not have this money... Well, we still go back to the bubble. Not everywhere in the world you can easily find a job that pays you enough, even if you accept to work more. And the employer will not accept to give developers a $200/month subscription, even less for persona…
I guess what was meant is that those tools are generally bought by the employer
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#200Earlier quoted context omitted.
Real world usage suggests otherwise. It's been a known trend for a while. Anthropic even confirmed as such ~6 months ago but said it was a "bug" - one that somehow just keeps happening 4-6 months after a model is released.
Real world usage is unlikely to give you the large sample sizes needed to reliably detect the differences between models. Standard error scales as the inverse square root of sample size, so even a difference as large as 10 percentage points would require hundreds of samples. https://marginlab.ai/trackers/claude-code/ tries to track Claude Opus performance on SWE-Bench-Pro, but since they only sample 50 tasks per day,…
It's hard to trust public, high profile benchmarks because any change to a specific model (Opus 4.5 in this case) can be rejected if they have regressions on SWE-Bench-Pro, so everything that gets to be released would perform well in this benchmark