$500 GPU outperforms Claude Sonnet on coding benchmarks • Hacker News Board

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#291

Post by walletdrainer » Sat, Mar 28, 2026, 12:17 PM UTC

stavros wrote: post #135

walletdrainer wrote: post #129

Earlier quoted context omitted.

>I'm often flabbergasted about the number of people here that lives in a bubble with regard to the price of tech Sorry, no. You live in the bubble, the people you think are living in a bubble are actually doing the very opposite and taking advantage of the lack of bubbles in our globally connected world. Today, basically anyone can sell any bullshit to billions of people around the world. We’ve never lived in less of…

I guess all those people who live in not-SF just can't be bothered to succeed!

I’ve never been to SF, wouldn’t know anything about it.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#292

Post by walletdrainer » Sat, Mar 28, 2026, 12:22 PM UTC

stavros wrote: post #159

weird-eye-issue wrote: post #156

Earlier quoted context omitted.

The person you were replying to was not talking about SF but you specifically called out SF so you were implying that

The thread started with "$200 is a lot for most of the world", the person I was replying to said "no it's not, now anyone can sell to billions of people", and I said "company success being concentrated in SF shows that that's not true". I didn't say "only SF can afford $200/mo".

>company success being concentrated in SF shows that that's not true

You didn’t say that until now.

I think you’re wrong, SF being particularly good for big companies might indicate something if the conversation was about succeeding at a grand scale, not about being able to afford $200/mo

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#293

Post by robotresearcher » Sat, Mar 28, 2026, 4:17 PM UTC

Epholys wrote: post #287

dahart wrote: post #241

Earlier quoted context omitted.

$200/mo is a lot, sure, but the shocking part of that comparison is your rent. I didn’t know $400/mo apartments still existed. For most people in the US and EU, $200 would be closer to 15%-20% of rent I think? My cell phone bill for my family is almost $200/mo. Last year, at first, $200 seemed crazy. Now that I’m getting addicted to coding agents, not so much. Some companies are paying API rates for AI for employees,…

As you've probably guessed, I don't live in the US, so the price are drastically different. I live in the EU. And for my case, I love in really small flat for some years, so the rent couldn't go up a lot. > most people in the US and EU, $200 would be closer to 15%-20% of rent I think? > the average rent is north of $1000/mo. I really don't know where you get your number from, $1000/mo average is really wild to me. Wi…

A friend’s 2BR in Palo Alto is $6K/mo. It’s a cute little mid-century house with a small backyard, but no AC or garage.

The salaries are good in SCV, but the local economy is calibrated to absorb the money in proportion.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#294

Post by spiderfarmer » Sat, Mar 28, 2026, 4:28 PM UTC

Razengan wrote: post #284

cesarvarela wrote: post #257

Earlier quoted context omitted.

Codex desktop is Electron too. What app are you talking about?

Oh my fuck, I was led to believe that the Codex Mac app was native. Codex itself said so. Google also said so but now it says different. Codex definitely "feels" more native than Claude: Proper menus etc, like when you right-click on a session in the sidebar, Codex shows an actual context menu, whereas Claude reveals its HTML-rendered jank and highlights the word you right-clicked on as if the sidebar item is just a…

"Claude Code has been bleh or meh at best in my experience."

"Why"

"HAVE YOU SEEN THEIR CONTEXT MENU"

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#295

Post by dahart » Sat, Mar 28, 2026, 9:14 PM UTC

Epholys wrote: post #287

dahart wrote: post #241

Earlier quoted context omitted.

$200/mo is a lot, sure, but the shocking part of that comparison is your rent. I didn’t know $400/mo apartments still existed. For most people in the US and EU, $200 would be closer to 15%-20% of rent I think? My cell phone bill for my family is almost $200/mo. Last year, at first, $200 seemed crazy. Now that I’m getting addicted to coding agents, not so much. Some companies are paying API rates for AI for employees,…

As you've probably guessed, I don't live in the US, so the price are drastically different. I live in the EU. And for my case, I love in really small flat for some years, so the rent couldn't go up a lot. > most people in the US and EU, $200 would be closer to 15%-20% of rent I think? > the average rent is north of $1000/mo. I really don't know where you get your number from, $1000/mo average is really wild to me. Wi…

> I really don’t know where you get your number from

I googled it. According to Google, London’s average rent is around €2,700, around 3x higher than the average. I assume the number of people living there and paying that much balances against the number of people like you living in smaller towns and rural areas who are paying lower rents.

But yes, rents have become very high everywhere. I live in a medium sized city in the US not anywhere near a coast, and most kids attending the local university are paying over $1000/mo for a 1-bedroom place. The primary way to get cheaper rent is to have flat-mates, try to get 3 or 4 people into a place that rents for, say, $2500/mo.

I was paying $2k/mo in San Francisco 25 years ago for a place that was maybe 90m^2, and since then rents have gone way up. Google says the average now is just under $4k/mo. In some nicer neighborhoods, some people pay $8k/mo for a single bedroom. This big-city rent in SF, LA, NY, Chicago, Miami, etc. balances against the small towns in the US where you can find a room for $500/mo, which is why the average is above $1k.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#296

Post by bartread » Sat, Mar 28, 2026, 10:18 PM UTC

mlrtime wrote: post #288

bartread wrote: post #247

Earlier quoted context omitted.

This is what gives me the warm fuzzies about the HN community: people jumping to wild conclusions about your domain and systems based on a 4 sentence comment. /s

The thing you'll start to notice is this happens A LOT on every subject. HN tends to think of itself as smarter than the average for every topic. But it turns out there is a lot of bad and factually wrong information in every thread.

Yeah, I know, and I know I've been guilty of it myself at times. It's a trap that's too easy to fall into.

Something about aughts dev culture as well: I remember it being really common back then. Everybody had to appear to be smart by second guessing everything everyone else was doing. Exhausting.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#297

Post by bartread » Sat, Mar 28, 2026, 10:20 PM UTC

senbrow wrote: post #276

bartread wrote: post #246

Earlier quoted context omitted.

Yeah, I did consider moving records to shadow tables, but - because of the nature of our data - it requires moving a lot of child records as well, so it's quite a lot of additional churn in WAL, and the same for restore. And this approach has its own challenges with referential integrity. More than that, though: lots of queries for reporting, and the like, suddenly need to use JOINs. Same for admin use cases where we…

Thanks for taking the time to write such a high quality reply; this is something I've wondered about for a long time and I appreciate the thought and detail you've shared here. :)

No worries - I'm glad it's helpful. Like anything, it's incredibly context specific, and you're always weighing up trade offs that may or may not turn out to be valid over the long term based on the best information you have right now.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#298

Post by BoorishBears » Sun, Mar 29, 2026, 3:23 AM UTC

XCSme wrote: post #279

BoorishBears wrote: post #270

Earlier quoted context omitted.

Like I said in the edit, when people want specific formatting they ask for well known formats: Markdown, XML, JSON I don't even need to debate if the benchmark is useful, it doesn't pass a sniff test: GPT-5.4 is not worse than Gemini 2.5 Flash in any way that matters to most users. In your benchmark it's meaningfully worse.

The questions do ask specifically to respond with the answer only, with an example format given in many cases. Note that all reasoning models are tested with "medium" reasoning. The benchmarks are questions/data processing tasks that an average user will likely ask, not coding questions (I didn't add any coding tests yet). Gemini models also tend to be very consistent. Asking the same question will likely give the sa…

If Gemini 2.5 Flash and GPT 5.4 perform the same for you, I'm glad.

It's not a useful finding for the rest of the world, and I sure hope non-technical people aren't being taken in by a steaming pile that implies those similarly performing LLMs (and many other ridiculous findings), but c'est la vie.

Now a days anyone can vibecode a "benchmark" with 0 understanding of the domain, what more should I expect?

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#299

Post by Razengan » Sun, Mar 29, 2026, 6:25 AM UTC

spiderfarmer wrote: post #294

Razengan wrote: post #284

Earlier quoted context omitted.

Oh my fuck, I was led to believe that the Codex Mac app was native. Codex itself said so. Google also said so but now it says different. Codex definitely "feels" more native than Claude: Proper menus etc, like when you right-click on a session in the sidebar, Codex shows an actual context menu, whereas Claude reveals its HTML-rendered jank and highlights the word you right-clicked on as if the sidebar item is just a…

"Claude Code has been bleh or meh at best in my experience." "Why" "HAVE YOU SEEN THEIR CONTEXT MENU"

Are you being dumb on purpose and ignoring the rest of the shit I said or just fanboying for Anthropic?

Claude's AI. itself. is. trash.

Their UI/UX makes it worse.

The constant fanboying/paid PR around it makes it the worst.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#300

Post by nl » Sun, Mar 29, 2026, 6:42 AM UTC

withinboredom wrote: post #206

nl wrote: post #180

Earlier quoted context omitted.

So - as the charts say - no statistical difference? Isn't this link am argument against the point you are making?

The chart doesn't cover the 4.6 release which was in the end of December/early January time frame. So, it's hard to tell from existing data.

That isn't true. The whole point it to quickly pick up statistically significant variations quickly, and with the volume of tests they are doing there is plenty of data.

If you turn on the 95% CI bands you can see there is plenty of statistical significance.