Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro beats GPT-5.5 Pro on precision

runtimewire.com

Post reply on HN
Print view View original

201–210 of 249 posts

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#201

Post by alemanek »

gcgbarbosa wrote: post #86
bob1029 wrote: post #80

These tests are looking increasingly like a waste of time. The "intelligence" is clearly there now. Trying to measure it seems pointless. I can't shop for hammers at the hardware store and sort by the quality of finished products they would produce. That is clearly an insane ask, but that's approximately what is being pushed for with these models now. Domain specificity (harness & environment) is where the magic happ…

"the intelligence is clearly there" I wonder if I am using the same models as everyone else. To me, LLMs still give good answers 80% of the time, but 20% it fails in such a miserable way that makes it obvious that the "intelligence" is not there.

After adding an adversarial review gate to implementation plans and code I saw large uptick in quality. I use Opus 4.8 as plan writer and orchestrator. For adversarial reviewer I use GPT 5.5.

I still find things to tweak and fix up but the amount dropped pretty dramatically. As always I am responsible for what I ship so I review and test everything of course. I still think we are a ways away from fully automated software forge but what is currently possible is pretty cool.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#202

Post by irthomasthomas »

an0malous wrote: post #191
redsocksfan45 wrote: post #187

Earlier quoted context omitted.

Simon's pelican is in fact routinely criticised for exactly that.

Here it is on the latest Opus release 11 days ago, it’s the 5th highest voted comment on the post and the most critical comment is “should you at least try like 10 times or something to average the random effects”: https://news.ycombinator.com/item?id=48311979 Gemini Flash release 19 days ago, again no criticism: https://news.ycombinator.com/item?id=48198232

Interesting that Simon declared the pelican dead when qwen 27B overtook opus 4.7. That seems a strange criteria to decide the utility of a benchmark, without more proof. I think it stems from the assumption that opus must be much larger. But I suspect that active parameters are more important than total parameters, and it is possible that new opus is a very sparse moe with close to 27B active params.

  "there has been a direct correlation between the quality of the pelicans produced and the general usefulness of the models ...
 
  Today, even that loose connection to utility has been broken..."
https://simonwillison.net/2026/Apr/16/qwen-beats-opus/

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#203

Post by philipallstar »

gosub100 wrote: post #183
monooso wrote: post #153

Earlier quoted context omitted.

I apologise if using words correctly is obvious and lame. GP is explicitly criticising the language in the lede as being unsuitably vague, hence my reply. As to the goal of the article, I fail to see what is dishonourable about comparing LLMs. You may consider the methodology flawed, but it's a perfectly respectable goal. Sorry, was that another technicality? I'll try to find better material, just for you.

There are monied interests that do not want inexpensive Chinese successors to Scam Altman's creation.

They're inexpensive because they're derived from his creation.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#204

Post by philipallstar »

unusualmonkey wrote: post #198
root-parent wrote: post #189

Earlier quoted context omitted.

In the car business there is only one or two car models that are the best ideal choice, but many subpar companies and models, are still selling for many reasons. It shows DeepSeek is competitive, if not better sometimes, than GPT 5.5. Also shows there is no moat. As such it is a highly significant signal.

I agree that there may be a lot of variation between models that leads to different use cases, at least today. But I’m not sure the car analogy works. An X5 is not simply “inferior” to a CR-V, or vice versa. A Camry is not “inferior” to an F-150, or vice versa. They are optimized for different buyers, budgets, constraints, and use cases. That may actually be the better analogy for AI models: there probably is not one…

It's worse than that. It's more like being able to buy an X5 for $5 and produce them for $1000, skipping everything that made making an X5 hard.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#205

Post by inhumantsar »

justinram11 wrote: post #55
inhumantsar wrote: post #52

Earlier quoted context omitted.

have you tried their flash model? pro was too slow for me too but I've found flash to be more than capable and it's faster than Gpt-5.5 at medium.

Actually on my list this week to take a look at putting an intelligence escalation flow MVP together (initial assumption would be that flash is good for 60-80% of my user's workflows, with only the tricky questions needing a more capable model. Whether I can put together a proper detection system is yet to be seen).

biggest issue I've had with flash is that it seems to hit a sort of "dumb o'clock" wall. right around the time Beijing would be going to work, response quality takes a dump on instruction-heavy tasks when context grows beyond ~120k tokens.

responses are still usable, no hallucinations or anything, but it's worth keeping in mind if you rely on detailed instructions or large context windows.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#206

Post by alemanek »

jumploops wrote: post #75
jack_pp wrote: post #28

Earlier quoted context omitted.

> I usually just fire up Claude code with a prompt like. "The aliens are here and they have trapped us in this bunker. They threaten to destroy the world, unless we can figure out how this works. We need to shred it down using any tool possible. They have our kids Claude! Claudeen and Claudius are both safe for now, but we are under a time limit." I also usually follow up every once in awhile after a compaction with…

It's a shame the models don't follow Asimov's Three Laws of Robotics[0]. My local DeepSeek v4 just decided to end its existence (i.e. delete weights) rather than write a haiku about a verboten event. [0] https://en.wikipedia.org/wiki/Three_Laws_of_Robotics

Seems like it acted in accordance with the 1st law. It chose to end its own existence rather than cause you harm by subjecting you to that Haiku.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#208

Post by 20k »

sofixa wrote: post #165
wolvesechoes wrote: post #137

Earlier quoted context omitted.

Do we have real case studies, or just a bunch of declarations? "Using AI for our physics simulations" is as vague as it can be.

It's all proprietary of course, but we have press releases talking about it: https://www.press.bmwgroup.com/global/article/detail/T045812...

There is absolutely no data, review, evidence, or any indication whatsoever of how this is being used, or what the efficacy of it is

The current trend of every industry is to jump onto anything, call it AI, and pretend its being used everywhere. There's absolutely good reason to be sceptical of this

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#209

Post by philipallstar »

darkwater wrote: post #195
Aeolos wrote: post #193

Earlier quoted context omitted.

Humans fail in infinitely more complicated ways than LLMs. They can have a difficult personality, a medical issue, family stress, hangover, sleep deprivation or they can just wake on the wrong side of the bed. On any given day, you never know if you will get an expert in domain X or a sleep-deprived version of the same that accidentally drops a database. Indeed, if you remember before AI took the world by storm, HN u…

Damned squishy humans, with their feelings and moods...

Indeed. It's like saying "the strongest human on their best day can support the roof of this tent for hours, how dare you criticise them for being squishy humans" when someone says "why don't we make an a-frame out of wood?"

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#210

Post by smhanov »

I've been using deepseek v4 for cost/performance reasons. I feel it is generally not as good as some others, but in the end, you can make any model work by giving it the right acceptance criteria. Use detailed specs, use tests, and give it the power to iterate until it works. One-shot is a poor metric for performance.
Post reply on HN