Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro beats GPT-5.5 Pro on precision

runtimewire.com

Post reply on HN
Print view View original

121–130 of 249 posts

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#121

Post by nibbleyou »

coldtea wrote: post #98
gcgbarbosa wrote: post #86

Earlier quoted context omitted.

"the intelligence is clearly there" I wonder if I am using the same models as everyone else. To me, LLMs still give good answers 80% of the time, but 20% it fails in such a miserable way that makes it obvious that the "intelligence" is not there.

It might be extra demand for rigor that's not equally applied to humans. One could argue that other coders in our teams, or even ourselves, often fail in "a miserable way", say about 20% of the time. But we block this out, or consider it "regular functioning", or just a one-off based on something we got wrong, "just a try" we redo, etc. But when an LLM does it on an area we know, we notice and suddenly it's too much.

Because a human fails in a known way. If a human does not have expertise in domain X or tech Y, they will fail there and the expectation is that they will fail.

With an LLM you never know where it can fail. There is no domain expertise for an LLM. It can fail in a miserable way in the same domain it worked spectacularly for.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#122

Post by skerit »

joystick_0x0 wrote: post #101
psadauskas wrote: post #57

I was using Claude until they banned Opencode, and now use GPT at my day job. I've been using Deepseek through Opencode Go on the $10/mo plan, and I honestly can't really tell much difference. Its just as capable, and makes the same kinds of dumb mistakes and the other two have been making since March. For the price, I'm more than happy with it.

I am not sure what I am doing wrong then. I am using claude the last 7 months and from time to time try other models like deepseek, kimi etc. Nothing can come even close to it. Claude is almost evrytime (99.99%) one shot.

Same here. Claude isn't perfect. It still makes a lot of mistakes. But whenever I try GPT-5.5 it's ten times worse, and Claude just has to clean up GPT's mess.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#123

Post by otabdeveloper4 »

20k wrote: post #105
wolvesechoes wrote: post #99

Earlier quoted context omitted.

> while some c++ exotic physics simulation developer will find it lacking Can confirm, but I always read I am holding it wrong.

I've consistently tried to apply LLMs to physics problems and they're utterly useless. They'll just confidently lie, or blatantly plagiarise source materials The issue is once you hit niche physics simulations there simply isn't any training data available, so the limitations of them become incredibly apparent. Its also problematic because a field itself will contain lots of wrong information (its research!), and AI…

> confidently lie, or blatantly plagiarise

Good enough for enterprise work tho. (Also the secret sauce to "holding LLMs right".)

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#124

Post by InsideOutSanta »

joystick_0x0 wrote: post #101
psadauskas wrote: post #57

I was using Claude until they banned Opencode, and now use GPT at my day job. I've been using Deepseek through Opencode Go on the $10/mo plan, and I honestly can't really tell much difference. Its just as capable, and makes the same kinds of dumb mistakes and the other two have been making since March. For the price, I'm more than happy with it.

I am not sure what I am doing wrong then. I am using claude the last 7 months and from time to time try other models like deepseek, kimi etc. Nothing can come even close to it. Claude is almost evrytime (99.99%) one shot.

In my experience, there is a very specific use case of one-shotting complex, long tasks with relatively vague or incomplete descriptions where Opus does substantially better than all other models I've tried, including GPT 5.5, GLM 5.1 and DS4. It seems to be better at inferring unstated requirements and creating a complete, working, reasonably well-designed solution.

However, that's probably not how most professional developers use LLMs. I tend to give well-specified, more constrained tasks, and for those, I find that Opus performs worse than other models precisely because it tends to infer unstated requirements and do things I didn't want it to do. In this situation, GPT 5.5 works better for me because it only and precisely does what I ask it to.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#125

Post by girvo »

coldtea wrote: post #98
gcgbarbosa wrote: post #86

Earlier quoted context omitted.

"the intelligence is clearly there" I wonder if I am using the same models as everyone else. To me, LLMs still give good answers 80% of the time, but 20% it fails in such a miserable way that makes it obvious that the "intelligence" is not there.

It might be extra demand for rigor that's not equally applied to humans. One could argue that other coders in our teams, or even ourselves, often fail in "a miserable way", say about 20% of the time. But we block this out, or consider it "regular functioning", or just a one-off based on something we got wrong, "just a try" we redo, etc. But when an LLM does it on an area we know, we notice and suddenly it's too much.

> But when an LLM does it on an area we know, we notice and suddenly it's too much.

Well of course. The owners of the companies building this are constantly talking about it replacing us all. Why would it be surprising that it would then be held to a higher standard?

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#126

Post by sankaritan »

psadauskas wrote: post #57

I was using Claude until they banned Opencode, and now use GPT at my day job. I've been using Deepseek through Opencode Go on the $10/mo plan, and I honestly can't really tell much difference. Its just as capable, and makes the same kinds of dumb mistakes and the other two have been making since March. For the price, I'm more than happy with it.

It's interesting. 95% of time you don't need the extra 5% rigor that frontier models provide to you compared to the 10-100x cheaper Chinese equivalents.

The remaining 5% of time you get a big boost for your high-reasoning problem solving needs and evade a lot of pain. Now, I just need to be able to predict accurately when I need this extra 5% and when not :)

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#127

Post by miroljub »

selfawareMammal wrote: post #109
psadauskas wrote: post #57

I was using Claude until they banned Opencode, and now use GPT at my day job. I've been using Deepseek through Opencode Go on the $10/mo plan, and I honestly can't really tell much difference. Its just as capable, and makes the same kinds of dumb mistakes and the other two have been making since March. For the price, I'm more than happy with it.

I have both subscriptions and I definitely feel gpt is better and more consistent, but when I run out of limits I don't miss it too much

That's the whole point. The tool you have vs. the expensive tools you don't have because they're too expensive.

I don't feel like paying 100 times the price for a 1-5% better tool.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#128

Post by monooso »

Stitch4223 wrote: post #38

It’s four poorly constructed arbitrary experiments which say very little about the competency of either model. The article reads like thin, auto-generated ai clickbait for nerd sniping or shilling a model. Consider the lead: > DeepSeek V4 Pro wins this head-to-head by being more exact where it matters: following instructions, matching schemas, and solving edge cases cleanly. GPT-5.5 Pro is still strong, but it gave a…

I think you've misunderstood the purpose of a lead (sic).

Per Merriam-Webster [^1], a lede is:

> the introductory section of a news story that is intended to entice the reader to read the full story

(Emphasis mine)

You may prefer more matter-of-fact phrasing, of course, but criticising a lede for attempting to achieve its goal is unjustified.

[^1]: https://www.merriam-webster.com/dictionary/lede

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#129

Post by coldtea »

girvo wrote: post #125
coldtea wrote: post #98

Earlier quoted context omitted.

It might be extra demand for rigor that's not equally applied to humans. One could argue that other coders in our teams, or even ourselves, often fail in "a miserable way", say about 20% of the time. But we block this out, or consider it "regular functioning", or just a one-off based on something we got wrong, "just a try" we redo, etc. But when an LLM does it on an area we know, we notice and suddenly it's too much.

> But when an LLM does it on an area we know, we notice and suddenly it's too much. Well of course. The owners of the companies building this are constantly talking about it replacing us all. Why would it be surprising that it would then be held to a higher standard?

Because it doesn't need to match a higher standard to "replace us all". It's enough that it works on the same standard, or even a lesser one, but for cheaper, with no complaints, and 24/7.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#130

Post by powerapple »

sankaritan wrote: post #126
psadauskas wrote: post #57

I was using Claude until they banned Opencode, and now use GPT at my day job. I've been using Deepseek through Opencode Go on the $10/mo plan, and I honestly can't really tell much difference. Its just as capable, and makes the same kinds of dumb mistakes and the other two have been making since March. For the price, I'm more than happy with it.

It's interesting. 95% of time you don't need the extra 5% rigor that frontier models provide to you compared to the 10-100x cheaper Chinese equivalents. The remaining 5% of time you get a big boost for your high-reasoning problem solving needs and evade a lot of pain. Now, I just need to be able to predict accurately when I need this extra 5% and when not :)

the extra 5% time you will need to help AI with multiple turns and information it needed. These 5% time reasoning rarely is enough to finish the task. i.e. 5% time AI is just not enough to complete the task without a lot help.
Post reply on HN