Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro beats GPT-5.5 Pro on precision

runtimewire.com

Post reply on HN
Print view View original

181–190 of 249 posts

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#181

Post by wolttam »

I rarely work on anything that demands better than DSv4 Flash, let alone pro.

If I can describe the problem and its solution well enough, Flash just does it.

If I can’t (or am feeling too lazy to) describe the problem well enough, and can only describe the desired outcome, then I’ve noticed models like GPT 5.5 being clearly better at working out a solid solution on their own.

There are some clear differences in the capabilities of the models, but it’s also clear that smaller open weight models are good enough to be a huge help for most tasks.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#183

Post by gosub100 »

monooso wrote: post #153
hypfer wrote: post #145

Earlier quoted context omitted.

I think the criticism is less about whether the lede is good at achieving its goal and more about whether that goal is honorable in the first place. So dismissing it on technicalities is for sure clever but also obvious and lame. The Letter/spirit thing eventually got boring. Please find better material

I apologise if using words correctly is obvious and lame. GP is explicitly criticising the language in the lede as being unsuitably vague, hence my reply. As to the goal of the article, I fail to see what is dishonourable about comparing LLMs. You may consider the methodology flawed, but it's a perfectly respectable goal. Sorry, was that another technicality? I'll try to find better material, just for you.

There are monied interests that do not want inexpensive Chinese successors to Scam Altman's creation.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#185

Post by andai »

SwellJoe wrote: post #7

I tried adding GPT 5.5 Pro to a vulnerability scanning benchmark I made ( https://swelljoe.com/post/will-it-mythos/ ), and it blew through the $100 budget limit halfway through. DeepSeek V4 Pro cost about a dollar for the whole benchmark. GPT Pro cost an average of $22 per case (a case could be 1-5 files with a recent known vulnerability, usually just a single file and a prompt along the lines of "does this file have…

Great article. I'm confused how Sonnet did worse than Haiku though. You mention it did find a bunch of other bugs, just not the ones you were looking for?

9 bugs is probably a bit low of a sample size to get a ranking.

That being said the ranking does end up roughly how you'd expect.

Deepseek is Pro, right? Not Flash? I've been using Flash for a lot of smaller tasks and finding it reasonably good. It's good for "interactive" use. Very fast, does small tasks nearly instantly.

It's also decent for investigating large codebases. I wonder if it could do security work too.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#186

Post by an0malous »

Stitch4223 wrote: post #38

It’s four poorly constructed arbitrary experiments which say very little about the competency of either model. The article reads like thin, auto-generated ai clickbait for nerd sniping or shilling a model. Consider the lead: > DeepSeek V4 Pro wins this head-to-head by being more exact where it matters: following instructions, matching schemas, and solving edge cases cleanly. GPT-5.5 Pro is still strong, but it gave a…

> poorly constructed arbitrary experiments which say very little about the competency of either model.

No one ever says this about the “pelican on a bicycle” metric

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#187

Post by redsocksfan45 »

an0malous wrote: post #186
Stitch4223 wrote: post #38

It’s four poorly constructed arbitrary experiments which say very little about the competency of either model. The article reads like thin, auto-generated ai clickbait for nerd sniping or shilling a model. Consider the lead: > DeepSeek V4 Pro wins this head-to-head by being more exact where it matters: following instructions, matching schemas, and solving edge cases cleanly. GPT-5.5 Pro is still strong, but it gave a…

> poorly constructed arbitrary experiments which say very little about the competency of either model. No one ever says this about the “pelican on a bicycle” metric

Simon's pelican is in fact routinely criticised for exactly that.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#188

Post by infecto »

an0malous wrote: post #186
Stitch4223 wrote: post #38

It’s four poorly constructed arbitrary experiments which say very little about the competency of either model. The article reads like thin, auto-generated ai clickbait for nerd sniping or shilling a model. Consider the lead: > DeepSeek V4 Pro wins this head-to-head by being more exact where it matters: following instructions, matching schemas, and solving edge cases cleanly. GPT-5.5 Pro is still strong, but it gave a…

> poorly constructed arbitrary experiments which say very little about the competency of either model. No one ever says this about the “pelican on a bicycle” metric

I am willing to guess it is but gets downvoted or similar. Simon is a bit of a cult of personality on HN for better or worse.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#189

Post by root-parent »

Stitch4223 wrote: post #38

It’s four poorly constructed arbitrary experiments which say very little about the competency of either model. The article reads like thin, auto-generated ai clickbait for nerd sniping or shilling a model. Consider the lead: > DeepSeek V4 Pro wins this head-to-head by being more exact where it matters: following instructions, matching schemas, and solving edge cases cleanly. GPT-5.5 Pro is still strong, but it gave a…

In the car business there is only one or two car models that are the best ideal choice, but many subpar companies and models, are still selling for many reasons.

It shows DeepSeek is competitive, if not better sometimes, than GPT 5.5. Also shows there is no moat. As such it is a highly significant signal.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#190

Post by andai »

metalspot wrote: post #120
freakynit wrote: post #48

Earlier quoted context omitted.

My personal observation (using a mix of opencode and pi harness): 1. DS4Pro: around opus 4.5 2. DS4Flash: around sonnet 4 3. Mimo v2.5 pro: between opus 4.5 and opus 4.6. 4. minimax M3: around opus 4.6 All of these are very close in terms of quality and pricing. For anything that is not specifically related to coding, DS4Flash has become ny de-factor model. It just works... super fast, tool calling is perfect, and th…

i have been using deepseek-v4-flash since it came out. i use a highly structured harness and spec/test driven workflow running through opencode, and so far there has been nothing it can't do. i have run through a bunch of tests: re-writing vvenc with assembly kernels, creating the first generation agent harness integration with opencode, porting TS npm modules to C++, porting an entire TS server app to C++, creating…

Thanks for the details. What's a second generation agent?

You mentioned the workflow is heavy on specs and tests. The smaller models seem to be really good at following instructions now. (Well, some of them!)

So that's probably part of why you're seeing good results. It has a very clear target.

Whereas with more open ended instructions they seem to struggle more. I think common sense is the main thing you get with model size.

When I'm working with the big models I feel like I don't have to spell things out so much. The gap is closing, but I'm assuming there is some fundamental limit there based on the size.

Of course the ideal would be Mythos, running for free, in my house, at 1,000 tok/s ;) Someday...

Post reply on HN