Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro beats GPT-5.5 Pro on precision

runtimewire.com

Post reply on HN
Print view View original

11–20 of 249 posts

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#11

Post by ekidd »

embedding-shape wrote: post #2

... according to grok-4-1-fast-non-reasoning who was the judge, on 4 tasks in total, score was 38 to 33 so obviously huge conclusions can be made. > We ran 4 fresh text tasks, generated on the fly for this matchup so neither model could prepare in advance, and had grok-4-1-fast-non-reasoning score each one. DeepSeek: DeepSeek V4 Pro scored 38.0 to OpenAI: GPT-5.5 Pro's 33.0.

The OP uses tons of typical AI turns of phrase, and Pangram classified it as AI with high confidence.

So it doesn't surprise me at all that the methodology is weak, too.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#13

Post by nhod »

“the matchup feels earned” is a current AI-written tell. To whom does it feel earned? To the AI that wrote this article?

I don’t know what it is specifically, but my weak human pattern-matching skills find this kind of language increasingly revolting. I don’t know why it is revolting, per se. It’s just the feeling I get.

Of course, me saying this on HN will get incorporated into GPT-5.6.175 or Claude 4.93 and it will make some version that just moves the revolting frontier elsewhere…

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#15

Post by rglover »

nhod wrote: post #13

“the matchup feels earned” is a current AI-written tell. To whom does it feel earned? To the AI that wrote this article? I don’t know what it is specifically, but my weak human pattern-matching skills find this kind of language increasingly revolting. I don’t know why it is revolting, per se. It’s just the feeling I get. Of course, me saying this on HN will get incorporated into GPT-5.6.175 or Claude 4.93 and it will…

I think it's because it's using storytelling-like language to describe reality.

"Harry finally had control of the broom. Draco was dead in his sights. The matchup feels earned."

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#16

Post by jameson »

random3 wrote: post #10
SwellJoe wrote: post #7

I tried adding GPT 5.5 Pro to a vulnerability scanning benchmark I made ( https://swelljoe.com/post/will-it-mythos/ ), and it blew through the $100 budget limit halfway through. DeepSeek V4 Pro cost about a dollar for the whole benchmark. GPT Pro cost an average of $22 per case (a case could be 1-5 files with a recent known vulnerability, usually just a single file and a prompt along the lines of "does this file have…

Where do you run DeepSeek?

Discounted pricing is available only at https://platform.deepseek.com. All of OpenRouter providers do not match their pricing at the moment.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#17

Post by jodacola »

Curious for folks who have made the switch I’m considering: if I swapped Claude Code to DeepSeek API pricing, would I get more bang for my buck compared to the $100 Max plan I’m using now?

I only hit the 5 hour limit every few days and the weekly limit a day or two before it resets at the most aggressive. I wouldn’t expect my usage to increase dramatically, other than not being stopped by limits.

I’m still apprehensive about shipping all my stuff off to a lab under an adversarial government (to the US), so not just looking at this from a pure cost basis, but my question is from the cost lens at the moment.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#18

Post by SwellJoe »

zaptrem wrote: post #8
SwellJoe wrote: post #7

I tried adding GPT 5.5 Pro to a vulnerability scanning benchmark I made ( https://swelljoe.com/post/will-it-mythos/ ), and it blew through the $100 budget limit halfway through. DeepSeek V4 Pro cost about a dollar for the whole benchmark. GPT Pro cost an average of $22 per case (a case could be 1-5 files with a recent known vulnerability, usually just a single file and a prompt along the lines of "does this file have…

Can you include GPT 5.5 non-pro (extra high thinking I guess) in your comparison? GPT Pro is the "I am willing to torch cash for a sooometimes slighty better result" option, not the one people are actually expected to use daily. That's probably part of the reason it's not in Codex

It's already there. It performed well. And, it'll be in the replication run later, as well.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#19

Post by SwellJoe »

random3 wrote: post #10
SwellJoe wrote: post #7

I tried adding GPT 5.5 Pro to a vulnerability scanning benchmark I made ( https://swelljoe.com/post/will-it-mythos/ ), and it blew through the $100 budget limit halfway through. DeepSeek V4 Pro cost about a dollar for the whole benchmark. GPT Pro cost an average of $22 per case (a case could be 1-5 files with a recent known vulnerability, usually just a single file and a prompt along the lines of "does this file have…

Where do you run DeepSeek?

I used the native DeepSeek API at deepseek.com. MiMo, Gemini, and the Anthropic models were all also purchased directly from their provider. The other models in the bench were either on OpenRouter or self-hosted.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#20

Post by willsmith72 »

jodacola wrote: post #17

Curious for folks who have made the switch I’m considering: if I swapped Claude Code to DeepSeek API pricing, would I get more bang for my buck compared to the $100 Max plan I’m using now? I only hit the 5 hour limit every few days and the weekly limit a day or two before it resets at the most aggressive. I wouldn’t expect my usage to increase dramatically, other than not being stopped by limits. I’m still apprehensi…

also curious. On the claude code $200 plan, get close to weekly limits but don't usually hit it. to me just about any small reduction in performance would not be acceptable, the cost of redirecting and getting stuck during long runs without me are too big (like when I tried gemini cli for a few days).

if it's 99.9% comparable performance for less money I'm interested, but I'm skeptical it's there

Post reply on HN