[dead]
$500 GPU outperforms Claude Sonnet on coding benchmarks
41–50 of 311 posts
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#42Am I still SOL on AMD (9070 XT) when it comes to this stuff?
No? You can run any model that fits in its VRAM, and you can run larger models with layer/MoE offloading. Ask an AI what the best models you can run on that card are, then ask it for newer models than that. Ask what tuning options to pass to llama.cpp, and what the auto-tuning options are. Use ROCm builds.
It looks like your card has 16GB VRAM? Start with Qwen 3.5 9B Unsloth GGUFs (UD-Q6_K_XL) and branch out from there.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#43Earlier quoted context omitted.
> cheaper than the cost of local electricity only. Can you explain what that means?
I think they mean that the DeepSeek API charges are less than it would cost for the electricity to run a local model. Local model enthusiasts often assume that running locally is more energy efficient than running in a data center, but fail to take the economies of scale into account.
Some of those local model enthusiasts can actually afford solar panels.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#44Earlier quoted context omitted.
> cheaper than the cost of local electricity only. Can you explain what that means?
It means that the electricity you would have to pay if you did the computations yourself would be more expensive than paying them to do it. Part of thst has to do with the fact that China has cheap electricity, also due to their massive push into renewables. Part of that is just economies of scale. A big server farm can run more efficiently than your PC on average.
cheap electric due to their massive push on non renewables. There has been no change in the price of electricity during the renewable shift.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#45[dead]
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#46I’d encourage devs to use MiniMax, Kimi, etc for real world tasks that require intelligence. The down sides emerge pretty fast: much higher reasoning token use, slower outputs, and degradation that is palpable. Sadly, you do get what you pay for right now. However that doesn’t prevent you from saving tons through smart model routing, being smart about reasoning budgets, and using max output tokens wisely. And optimiz…
[flagged]
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#47Earlier quoted context omitted.
Yup, they do quite poorly on random non-coding tasks: https://aibenchy.com/compare/minimax-minimax-m2-7-medium/moo...
It’s worth also comparing Qwen 3.5, it’s a very strong model. Different benchmarks give different results, but in general Qwen 3.5, GLM 5, and Kimi K2.5 are all excellent models, and not too far from current SOTA models in capability/intelligence. In my own non-coding tests, they were better than Gemini 3.1 flash. They’re comparable to the best American models from 6 months ago.
I used qwen 3.5 plus in production, it was really good at instruction following and tool calling.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#48It's a race to the bottom. DeepSeek beats all others (single-shot), and it is ~50% cheaper than the cost of local electricity only. > DeepSeek V3.2 Reasoning 86.2% ~$0.002 API, single-shot > ATLAS V3 (pass@1-v(k=3)) 74.6% ~$0.004 Local electricity only, best-of-3 + repair pipeline
I've tested many open models, Deepseek 3.2 is the only SOTA similar.
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#49Earlier quoted context omitted.
I think they mean that the DeepSeek API charges are less than it would cost for the electricity to run a local model. Local model enthusiasts often assume that running locally is more energy efficient than running in a data center, but fail to take the economies of scale into account.
Some of those local model enthusiasts can actually afford solar panels.
You are still incurring a cost if you use the electricity instead of selling it back to the grid
Re: $500 GPU outperforms Claude Sonnet on coding benchmarks
#50Will open source or local llms kill the big AI providers eventually? If so when? I can see maybe basic chat, not sure about coding and images yet