Earlier quoted context omitted.
Yup, they do quite poorly on random non-coding tasks: https://aibenchy.com/compare/minimax-minimax-m2-7-medium/moo...
GLM 5 here is significantly better than GPT-5.4
[dead]
$500 GPU outperforms Claude Sonnet on coding benchmarks
151–160 of 311 posts
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#152Earlier quoted context omitted.
> The $200 per month subscription comes with a ton of usage. $200 dollars + VAT is half of my rent. I know HN is not a good place to rant on this subject, but I'm often flabbergasted about the number of people here that lives in a bubble with regard to the price of tech. Or just prices in general. I remember someone who said a few years ago (I'm paraphrasing): "You could just use one of the empty room in your house!"…
Work pays.
I'm not sure I've correctly understood what you're implying.
If it's that I'm not working, well, I'm employed.
It it's that I'm not working enough to not have this money... Well, we still go back to the bubble. Not everywhere in the world you can easily find a job that pays you enough, even if you accept to work more. And the employer will not accept to give developers a $200/month subscription, even less for personal use.
If it's that I'm not working enough and I should go freelancing to work as much as I want and get rich (I'm extrapolating). Well, you're right, I could do that. But (at least at first), I would work a lot more for much less money. And even if I become a recognized freelancer, it doesn't change the fact that I'll earn less money compared to the baseline of SF, or even the USA in the tech sector in general. So, bubble again. I could also, like someone said, put the tokens cost into my hourly/daily rate, but I'll be much more expensive than other freelancers.
Also, but that's a "me case" compared to my previous points, health issues can greatly affect how much work you can do.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#153Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#154Claude Code has been bleh or meh at best in my experience. There's so many posts on HN fawning about it lately that it could only be a guerrilla marketing campaign.
"I don't get it. Everyone else is wrong."
"There's no such thing as astroturfing." ok
I use Codex regularly and Claude is shit in comparison, from its constant "Oops you're right!!" backtracking to its crap Electron app (if their AI is so good why can't they make a fucking native app for each OS?)
Hell right freakin now I asked it to implement something and got a weird "Something went wrong" API error
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#155Earlier quoted context omitted.
They do. I'm currently seeing a degradation on Opus 4.6 on tasks it could do without trouble a few months back. Obvious I'm a sample of n=1, but I'm also convinced a new model is around the corner and they preemptively nerf their current model so people notice the "improvement".
Make that 2, I told my friends yesterday "Opus got dumb, new model must be coming".
I swear that difference sessions will route to different quants. Sometimes it's good, sometimes not.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#156Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#157Claude Code has been bleh or meh at best in my experience. There's so many posts on HN fawning about it lately that it could only be a guerrilla marketing campaign.
You still need to give it precise context and instructions when dealing with things that are not web apps or some other software cliche.
The reasoning is great in opus, unbeatable at the moment.
I understand what you mean, it becomes disappointing on more niche or specific work. It’s honestly a good thing to see these models are not really intelligent yet.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#158Earlier quoted context omitted.
> The $200 per month subscription comes with a ton of usage. $200 dollars + VAT is half of my rent. I know HN is not a good place to rant on this subject, but I'm often flabbergasted about the number of people here that lives in a bubble with regard to the price of tech. Or just prices in general. I remember someone who said a few years ago (I'm paraphrasing): "You could just use one of the empty room in your house!"…
You think I don't understand that? I'm friends with people who make little more than that amount per month. But it's not all that relevant to this conversation. It's not like this is the first time economic inequality is a thing. It's just as relevant to me factoring in your salary the next time I go buy a car.
First, I've assumed you were in the bubble I described, but that's not the case, so sorry bout that.
Also, I think it's relevant to the conversation.
You replied to someone who said that "you" (undirected pronoun I suppose) can't afford the SOTA that the $200/month Anthropic subscription comes with a ton of usage. So I interpreted it as a general statement. It wasn't what you meant?
I'm a bit lost about who you're talking to/about in your first comment: the person you respond to, a general statement for everyone reading, or yourself?
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#159Earlier quoted context omitted.
Nobody in this thread claimed that.
The person you were replying to was not talking about SF but you specifically called out SF so you were implying that
The thread started with "$200 is a lot for most of the world", the person I was replying to said "no it's not, now anyone can sell to billions of people", and I said "company success being concentrated in SF shows that that's not true".
I didn't say "only SF can afford $200/mo".
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#160Earlier quoted context omitted.
On paper. There's huge financial incentive to quantize the crap out of a good model to save cash after you've hooked in subscriptions.
And there’s an incentive to publish evidence of this to discourage it, do you have any?
Models aren't just big bags of floats you imagine them to be. Those bags are there, but there's a whole layer of runtimes, caches, timers, load balancers, classifiers/sanitizers, etc. around them, all of which have tunable parameters that affect the user-perceptible output.