Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

$500 GPU outperforms Claude Sonnet on coding benchmarks

github.com

Post reply on HN
Print view View original

261–270 of 311 posts

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#261

Post by slashdev »

imiric wrote: post #255
serf wrote: post #236

Earlier quoted context omitted.

I have never read a snide comment on this site that i've been more repulsed by. I think because it's so specifically sharpened to stab at the software developer, my compatriot, one of the foremost primary populations here, rather than just an overall shitty human insult -- and timed to do so when the person opens up in an honest dialogue about what they're doing. But good news: every large software house i've talked…

[flagged]

You only hurt yourself with that attitude. AI might take your job.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#262

Post by imiric »

slashdev wrote: post #261
imiric wrote: post #255

Earlier quoted context omitted.

[flagged]

You only hurt yourself with that attitude. AI might take your job.

> You only hurt yourself with that attitude.

Funny, others seem more hurt by it.

> AI might take your job.

I'm not the one "grieving the loss of his career". :)

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#263

Post by hrmtst93837 »

patshead wrote: post #18
negativegate wrote: post #5

Am I still SOL on AMD (9070 XT) when it comes to this stuff?

No, but yes? OmniCoder 9B at Q6 fits on my 9070 XT with 200k+ tokens of context, and it works pretty well with OpenCode. It is for sure the best local model that I've managed to squeeze onto my GPU, and it even works at 120k context at Q3 on an 8GB RX 580 GPU. I can't imagine trying to using this model on either GPU for real work. I can use much bigger and faster models on the $3 Chutes subscription or $10 OpenCode G…

[flagged]

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#264

Post by scrollop »

hhh wrote: post #104
withinboredom wrote: post #66

Earlier quoted context omitted.

Opus 4.5 ~= Opus 4.6 high. Opus 4.5 was nerfed just before or after the release of 4.6.

The models don’t change.

You sure about that?

https://marginlab.ai/trackers/claude-code/

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#265

Post by Tuna-Fish »

b3ing wrote: post #50

Will open source or local llms kill the big AI providers eventually? If so when? I can see maybe basic chat, not sure about coding and images yet

Centralized inference is more economically efficient⁰, and should be cheaper for most users once competition squeezes the air out of token prices. It remains very valid for anyone who wants to maintain their privacy, ofc.

0: Because the only way to get cache locality out of a LLM is to batch invocations. A centralized system where the server handles thousands of invocations at the same time only needs a tiny fraction of the total memory throughput as having all of those invocations run locally on different machines would.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#266

Post by metalliqaz »

0xbadcafebee wrote: post #42
negativegate wrote: post #5

Am I still SOL on AMD (9070 XT) when it comes to this stuff?

No? You can run any model that fits in its VRAM, and you can run larger models with layer/MoE offloading. Ask an AI what the best models you can run on that card are, then ask it for newer models than that. Ask what tuning options to pass to llama.cpp, and what the auto-tuning options are. Use ROCm builds. It looks like your card has 16GB VRAM? Start with Qwen 3.5 9B Unsloth GGUFs (UD-Q6_K_XL) and branch out from the…

I've been running local models on my 9070XT and I have never found ROCm to be faster than Vulkan

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#268

Post by dahart »

Aerroon wrote: post #253
dahart wrote: post #241

Earlier quoted context omitted.

$200/mo is a lot, sure, but the shocking part of that comparison is your rent. I didn’t know $400/mo apartments still existed. For most people in the US and EU, $200 would be closer to 15%-20% of rent I think? My cell phone bill for my family is almost $200/mo. Last year, at first, $200 seemed crazy. Now that I’m getting addicted to coding agents, not so much. Some companies are paying API rates for AI for employees,…

It is my belief that rent price scales with the leftover income people have after they've paid for other necessities. Ie if you're from a poorer country/area then things like milk and gasoline will cost a similar amount (maybe 2x difference), but rent will cost a lot less. As people in a country get richer they start paying a larger and larger share of their income as rent of various forms. Even the US has places wit…

It’s true that average rent prices are regional and poorer areas have lower rents, but that doesn’t tend to make much difference in urban areas and large cities where the majority of people live now. Why do you feel that rent scales with disposable income? Economists generally say the opposite based on housing being a core necesessity; that people pay rent in proportion to their income, and only what’s left over the the disposable amount. That’s why we have the 30% rule, for example.

You’re technically correct, btw, rental housing is a market and is subject to market forces, meaning what people are willing to pay. I’m just not so sure about framing rent as being lower priority than other necessities. And rent prices have been increasing faster than other necessities, and faster than income, so that might be a confounding factor in your argument.

Still, my initial reaction above is due to the fact that in the US and in Europe in most large cities, the average rent is north of $1000/mo.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#269

Post by philbitt »

bloppe wrote: post #76

Generating big chunks of code is rarely what I want from an agent. They really shine for stuff like combing through logs or scanning dozens of source files to explain a test failure. Which benchmark covers that? I want the debugging benchmark that tests mastery of build systems, CLIs, etc.

[dead]

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#270

Post by BoorishBears »

XCSme wrote: post #95
BoorishBears wrote: post #93

Earlier quoted context omitted.

> It is not my fault if Claude outputs something like "*1*, *1*", adding markdown highlighting, when most other models respect the required format correctly. Yuck. At that point don't publish a benchmark, explains why their results are useless too. - Edit since I'm not able to reply to the below comment: "I want structured output from a model that supports structured output but will not enable structured output, nor…

Why not? I described this in more detail in other comments. Even when using structured output, sometimes you want to define how the data should be displayed or formatted, especially for cases like chat bots, article writing, tool usage, calling external api's, parsing documents, etc. Most models get this right. Also, this is just one failure mode of Claude.

Like I said in the edit, when people want specific formatting they ask for well known formats: Markdown, XML, JSON

I don't even need to debate if the benchmark is useful, it doesn't pass a sniff test: GPT-5.4 is not worse than Gemini 2.5 Flash in any way that matters to most users. In your benchmark it's meaningfully worse.

Post reply on HN