Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

$500 GPU outperforms Claude Sonnet on coding benchmarks

github.com

Post reply on HN
Print view View original

51–60 of 311 posts

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#51

Post by [deleted] »

wizee wrote: post #38
XCSme wrote: post #19

Earlier quoted context omitted.

Yup, they do quite poorly on random non-coding tasks: https://aibenchy.com/compare/minimax-minimax-m2-7-medium/moo...

It’s worth also comparing Qwen 3.5, it’s a very strong model. Different benchmarks give different results, but in general Qwen 3.5, GLM 5, and Kimi K2.5 are all excellent models, and not too far from current SOTA models in capability/intelligence. In my own non-coding tests, they were better than Gemini 3.1 flash. They’re comparable to the best American models from 6 months ago.

[deleted]

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#52

Post by Kodiack »

jLaForest wrote: post #49
jacquesm wrote: post #43

Earlier quoted context omitted.

Some of those local model enthusiasts can actually afford solar panels.

You are still incurring a cost if you use the electricity instead of selling it back to the grid

The extent of that heavily depends on where you are. Where I live in NZ, the grid export rates are very low while the import rates are very high.

Our peak import rate is 3x higher than our solar export rate. In other words, we’d need to sell 3 kWh hours of energy to offset the cost of using 1 kWh at peak.

We’re currently in the process of accepting a quote for home batteries. The rates here highly incentivise maximising self-use.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#53

Post by throwaway85825 »

b3ing wrote: post #50

Will open source or local llms kill the big AI providers eventually? If so when? I can see maybe basic chat, not sure about coding and images yet

Financial gravity will kill them when returns don't match stratospheric expectations.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#54

Post by usagisushi »

XCSme wrote: post #19
mmaunder wrote: post #17

I’d encourage devs to use MiniMax, Kimi, etc for real world tasks that require intelligence. The down sides emerge pretty fast: much higher reasoning token use, slower outputs, and degradation that is palpable. Sadly, you do get what you pay for right now. However that doesn’t prevent you from saving tons through smart model routing, being smart about reasoning budgets, and using max output tokens wisely. And optimiz…

Yup, they do quite poorly on random non-coding tasks: https://aibenchy.com/compare/minimax-minimax-m2-7-medium/moo...

Interesting benchmark. It is notable that Gemini-3-Flash outperforms 3.1 Pro. My experience using Flash via Opencode over the past month suggests it is quite underrated.

Needless to say, benchmarks are limited and impressions vary widely by problem domain, harness, written language, and personal preference (simplicity vs detail, tone, etc.). If personal experience is the only true measure, as with wine, solving this discovery gap is an interesting challenge (LLM sommelier!), even if model evolution eventually makes the choice trivial. (I prefer Gemini 3 for its wide knowledge, Sonnet 4.6 for balance, and GLM-5 for simplicity.)

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#55

Post by dmichulke »

jLaForest wrote: post #49
jacquesm wrote: post #43

Earlier quoted context omitted.

Some of those local model enthusiasts can actually afford solar panels.

You are still incurring a cost if you use the electricity instead of selling it back to the grid

Luxembourg: Purchase price = 2 x sales price, mostly due to grid costs.

And this is with no income tax or VAT on sold electricity.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#57

Post by qingcharles »

b3ing wrote: post #50

Will open source or local llms kill the big AI providers eventually? If so when? I can see maybe basic chat, not sure about coding and images yet

Unless there are some really, really major shortcuts found in inference, then it's always going to be hard to run a really great model locally. The costs of the PC + electric will usually be crazy compared to a $20/mo Claude sub.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#58

Post by bluefirebrand »

throwaway85825 wrote: post #53
b3ing wrote: post #50

Will open source or local llms kill the big AI providers eventually? If so when? I can see maybe basic chat, not sure about coding and images yet

Financial gravity will kill them when returns don't match stratospheric expectations.

I hope so too, but I think it's wishful thinking. Be prepared for the mother of all financial bailouts from the world governments to make sure that doesn't happen

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#59

Post by mkw2000 »

mmaunder wrote: post #17

I’d encourage devs to use MiniMax, Kimi, etc for real world tasks that require intelligence. The down sides emerge pretty fast: much higher reasoning token use, slower outputs, and degradation that is palpable. Sadly, you do get what you pay for right now. However that doesn’t prevent you from saving tons through smart model routing, being smart about reasoning budgets, and using max output tokens wisely. And optimiz…

i find kimi to be very very good, minimax not so much

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#60

Post by hollerith »

bluefirebrand wrote: post #58
throwaway85825 wrote: post #53

Earlier quoted context omitted.

Financial gravity will kill them when returns don't match stratospheric expectations.

I hope so too, but I think it's wishful thinking. Be prepared for the mother of all financial bailouts from the world governments to make sure that doesn't happen

I can understand why banks got bailed out by the US gov in 2008, but why would a government feel the need to bail out AI labs?

I hope you are not going to say, "to avoid a global recession or depression caused by the popping of the AI bubble". That would be unnecessary and harmful (in its second-order effects), and governments do have advisors who are competent enough in economics to advise against such a move.

Post reply on HN