Live data from Hacker News

$500 GPU outperforms Claude Sonnet on coding benchmarks

github.com

231–240 of 311 posts

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#231
post #146

Earlier quoted context omitted.

What is this 10€ per month subscription that you are talking about?

MiniMax token plan https://platform.minimax.io/docs/guides/pricing-token-plan

How is the speed and stability?

These small Chinese companies dont always have access to serious hardware.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#232

Earlier quoted context omitted.

A subscription for coding - no thanks.

If you think it's only for coding you don't have much of an imagination :)

These are the types of individuals that become so left in the dust that they don't realize what's going on anymore, and it's obvious this person is already there. Claude hasn't been a "subscription for coding" product for quite some time now. That's how it started out and while that's certainly what Claude is known for, Anthropic has been pushing for Claude to also be a general productivity tool -- Claude Code, then Claude Desktop, Claude Work, and now Claude Desktop has Chat, Work, and Code essentially built into a single desktop app that just works wonders for those who are looking for a general productivity tool.

I'd not use it over pure Claude Code because I am at heart a coder and I want the raw terminal experience and there's some features missing from the "Code" tab in Claude Desktop, but just saying "a subscription to code", just goes to show how out of touch that person already is, and that's what resistance does to you when you try to resist making use of any kind of modern tooling or technology.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#233
post #215

Earlier quoted context omitted.

[flagged]

We have the quietest on-call rotation of any company I've ever worked at. We have a high standard for code review, static verification, and tests. The fact that the code isn't hand-rolled artisanal code, and is generated by AI now, has so far turned out to have no impact on product quality or bugs reported.

What are company or tools you are working?

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#235
post #215

Earlier quoted context omitted.

[flagged]

Why? Because writing code is the only measure of quality when producing tools? What about Unit and Integration Tests, UX research, and Performance tests.

I agree that for many applications the code written by an LLM can be good enough, as proven by the many commercial applications that contain even worse code.

However, anyone who uses an LLM must remain aware of the limitations of this method.

There are many features of a program that cannot be tested exhaustively and which must be guaranteed by its design. When you do not understand very well the structure of a program it may be difficult to decide what must be tested.

With performance, the confidence in what an LLM produces is even lower, because it is unlikely to know if you have really reached a performance limited by hardware. Obtaining a performance better than a previously existing program does not prove anything, because most existing programs are likely to have a performance much lower than possible.

In many cases you just want a performance good enough, not the best attainable, so you can be content with your LLM-generated program. But you must not fool yourself by believing that this is really the best that can be done.

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#236
post #215

Earlier quoted context omitted.

Generating big chunks code is all I do, all day. I don't write code by hand any more, neither at work, nor for side projects. I work mostly in Rust and TypeScript at a developer tools company.

[flagged]

I have never read a snide comment on this site that i've been more repulsed by.

I think because it's so specifically sharpened to stab at the software developer, my compatriot, one of the foremost primary populations here, rather than just an overall shitty human insult -- and timed to do so when the person opens up in an honest dialogue about what they're doing.

But good news: every large software house i've talked to in the past two years is touching AI. As tragic as that is for a multitude of good reasons surrounding the workforce/copyright/ip/human-laziness/loss-of-skill/etc, that means imric is going to be outside of software , by their own rules, in totality in just a few short years!

Happy days!

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#237

Earlier quoted context omitted.

> I’d encourage devs to use MiniMax, Kimi, etc for real world tasks that require intelligence. I use MiniMax daily, mostly for coding tasks, using pi-coding-agent mostly. > The down sides emerge pretty fast: much higher reasoning token use, slower outputs, and degradation that is palpable. I don't care about token use, I pay per request in my cheap coding plan. I didn't notice slower outputs, it's even faster than An…

I've only been using free tokens for a year now. Gemini and they just dropped pro so I switched to minimax. Bit of a hurdle switching from Gemini-cli to kilo-cli, but now I can't really see too much difference. If I was starting new projects I'd pay for a better model, but honestly I don't really know any different. I've not ever used Claude and people seem to rave about it. Maybe its good, but I doubt its $200/month…

I've used Gemini and now claude. Both were meh until I found the superpowers skill. Will be trying chatgpt next month.

You can "feel" the llm being limited with Gemini, less so with Claude. Hopefully even less so with chatgpt

Re: $500 GPU outperforms Claude Sonnet on coding benchmarks

#239
post #76

Generating big chunks of code is rarely what I want from an agent. They really shine for stuff like combing through logs or scanning dozens of source files to explain a test failure. Which benchmark covers that? I want the debugging benchmark that tests mastery of build systems, CLIs, etc.

[flagged]
Post reply on HN