Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

$500 GPU outperforms Claude Sonnet on coding benchmarks

github.com

Post reply on HN
Print view View original

171–180 of 311 posts

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#171

Post by mongrelion »

sznio wrote: post #105

On that topic, anyone here got a decent local coding AI setup for a 12GB VRAM system? I have a Radeon 6700 XT and would like to run autocomplete on it. I can fit some models in the memory and they run quick but are just a tad too dumb. I have 64GB of system ram so I can run larger models and they are at least coherent, but really slow compared to running from VRAM.

Not the answer that you are looking for, but I am a fellow AMD GPU owner, so I want to share my experience.

I have a 9070 XT, which has 16GB of VRAM. My understanding from reading around a bunch of forums is that the smallest quant you want to go with is Q4. Below that, the compression starts hurting the results quite a lot, especially for agentic coding. The model might eventually start missing brackets, quotes, etc.

I tried various AI + VRAM calculators but nothing was as on the point as Huggingface's built-in functionality. You simply sign up and configure in the settings [1] which GPU you have, so that when you visit a model page, you immediately see which of the quants fits in your card.

From the open source models out there, Qwen3.5 is the best right now. unsloth produces nice quants for it and even provides guidelines [2] on how to run them locally.

The 6-bit version of Qwen3.5 9B would fit nicely in your 6700 XT, but at 9B parameters, it probably isn't as smart as you would expect it to run.

Which model have you tried locally? Also, out of curiosity, what is your host configuration?

[1]: https://huggingface.co/settings/local-apps [2]: https://unsloth.ai/docs/models/qwen3.5

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#172

Post by jakozaur »

bloppe wrote: post #76

Generating big chunks of code is rarely what I want from an agent. They really shine for stuff like combing through logs or scanning dozens of source files to explain a test failure. Which benchmark covers that? I want the debugging benchmark that tests mastery of build systems, CLIs, etc.

Build systems are tested by CompileBench (Quesma's benchmark).

Disclaimer: I'm the founder.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#173

Post by woadwarrior01 »

armchairhacker wrote: post #139
tornikeo wrote: post #107

Earlier quoted context omitted.

On paper. There's huge financial incentive to quantize the crap out of a good model to save cash after you've hooked in subscriptions.

And there’s an incentive to publish evidence of this to discourage it, do you have any?

There's this[1]. Model providers have a strong incentive to switch (a part of) their inference fleet to quantized models during peak loads. From a systems perspective, it's just another lever. Better to have slightly nerfed models than complete downtime.

[1]: https://marginlab.ai/trackers/claude-code/

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#174

Post by layer8 »

Epholys wrote: post #117
weird-eye-issue wrote: post #70

Earlier quoted context omitted.

Why is that? The $200 per month subscription comes with a ton of usage. Opus 4.6 is available on the $20 plan too

> The $200 per month subscription comes with a ton of usage. $200 dollars + VAT is half of my rent. I know HN is not a good place to rant on this subject, but I'm often flabbergasted about the number of people here that lives in a bubble with regard to the price of tech. Or just prices in general. I remember someone who said a few years ago (I'm paraphrasing): "You could just use one of the empty room in your house!"…

The other part of the bubble is assuming working in projects that allow disclosing any code or project details to a generic third party with that kind of power asymmetry.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#176

Post by Razengan »

maipen wrote: post #157
Razengan wrote: post #82

Claude Code has been bleh or meh at best in my experience. There's so many posts on HN fawning about it lately that it could only be a guerrilla marketing campaign.

You still need to give it precise context and instructions when dealing with things that are not web apps or some other software cliche. The reasoning is great in opus, unbeatable at the moment. I understand what you mean, it becomes disappointing on more niche or specific work. It’s honestly a good thing to see these models are not really intelligent yet.

I still don't trust any AI enough to generate or edit code, except for some throwaway experiments, because every time I tried it's been inefficient or too verbose or just plain wrong.

I use it for reviewing existing code, specifically for a components-based framework for Godot/GDScript at [0]. You can view the AGENTS.md and see that it's a relatively simple enough project: Just for 2D games and fairly modular so the AI can look at each file/class individually and have to cross-reference maybe 1-3 dependencies/dependents at most at any time during a single pass.

I've been using Codex, and it's helped me catch a lot of bugs that would have taken a long time on my own to even notice at all. Most of my productivity and the commits from the past couple months are thanks to that.

Claude on the other hand, oh man… It just wastes my time. It's had way more gaffes than Codex, on the exact same code and prompts.

[0] https://github.com/InvadingOctopus/comedot

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#177

Post by weird-eye-issue »

Epholys wrote: post #158
weird-eye-issue wrote: post #142

Earlier quoted context omitted.

You think I don't understand that? I'm friends with people who make little more than that amount per month. But it's not all that relevant to this conversation. It's not like this is the first time economic inequality is a thing. It's just as relevant to me factoring in your salary the next time I go buy a car.

First, I've assumed you were in the bubble I described, but that's not the case, so sorry bout that. Also, I think it's relevant to the conversation. You replied to someone who said that "you" (undirected pronoun I suppose) can't afford the SOTA that the $200/month Anthropic subscription comes with a ton of usage. So I interpreted it as a general statement. It wasn't what you meant? I'm a bit lost about who you're ta…

I assume when somebody says you and is not talking about anyone in particular they mean that it's infeasible for virtually everybody which is certainly not the case. Also you conveniently disregarded the fact that is available on the $20 per month plan.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#178

Post by weird-eye-issue »

stavros wrote: post #166
weird-eye-issue wrote: post #165

Earlier quoted context omitted.

"I guess all those people who live in not-SF just can't be bothered to succeed!"

I explained it in my previous comment, I'm not going to explain it more than that.

Again, if you think that only successful companies are in SF you live in a bubble.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#180

Post by nl »

woadwarrior01 wrote: post #173
armchairhacker wrote: post #139

Earlier quoted context omitted.

And there’s an incentive to publish evidence of this to discourage it, do you have any?

There's this[1]. Model providers have a strong incentive to switch (a part of) their inference fleet to quantized models during peak loads. From a systems perspective, it's just another lever. Better to have slightly nerfed models than complete downtime. [1]: https://marginlab.ai/trackers/claude-code/

So - as the charts say - no statistical difference?

Isn't this link am argument against the point you are making?

Post reply on HN