Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 Pro beats GPT-5.5 Pro on precision

runtimewire.com

Post reply on HN
Print view View original

61–70 of 249 posts

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#61

Post by epolanski »

manmal wrote: post #58
epolanski wrote: post #54

Earlier quoted context omitted.

I have been saying that from multiple of my tests you can use Claude Code with DS4 Pro or Flash (you just swap api keys) at more or less equivalent performance and people keep screaming "that it's not SOTA". I don't know whether models are over fitted to benchmarks and people take them at face value, but I spend less on DS4 apis than I do for Claude Code 100$ subscription and I code everyday. So far I'm quite happy w…

Are you not worried about where your data will end up? By now I‘m feeding things to Codex that I‘d rather not have in a leak.

Yes, that's exactly why I avoid OpenAI and Anthropic products.

Besides the (quite true) joke, if sending data to DeepSeek is a concern the good thing is that the models are open weight, you can self host them or use third party providers.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#63

Post by axus »

manmal wrote: post #58
epolanski wrote: post #54

Earlier quoted context omitted.

I have been saying that from multiple of my tests you can use Claude Code with DS4 Pro or Flash (you just swap api keys) at more or less equivalent performance and people keep screaming "that it's not SOTA". I don't know whether models are over fitted to benchmarks and people take them at face value, but I spend less on DS4 apis than I do for Claude Code 100$ subscription and I code everyday. So far I'm quite happy w…

Are you not worried about where your data will end up? By now I‘m feeding things to Codex that I‘d rather not have in a leak.

It might be a while before DeepSeek shows up on GovCloud

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#65

Post by trollbridge »

ipaddr wrote: post #60
natrys wrote: post #21

Earlier quoted context omitted.

It seems frontier, on the balance, would rather lose that segment of he market than lower the API price. They are getting the bag in the enterprise segment, those clients aren't ditching them for DeepSeek. As for other segments, high API pricing gets people to switch to the subscriptions instead which is stickier than the API.

I've been hearing that Anthropic want all major AI providers to stop developing front tier models for a year for safety reasons. The real reason is they need time to get there models cheaper because of the DeepSeek threat or local llms or other even cheaper providers.

Seems like a ridiculous request - how can they ensure China will stop developing frontier models?

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#66

Post by SubiculumCode »

jodacola wrote: post #17

Curious for folks who have made the switch I’m considering: if I swapped Claude Code to DeepSeek API pricing, would I get more bang for my buck compared to the $100 Max plan I’m using now? I only hit the 5 hour limit every few days and the weekly limit a day or two before it resets at the most aggressive. I wouldn’t expect my usage to increase dramatically, other than not being stopped by limits. I’m still apprehensi…

Yeah, the discounted deepseek inference is subsidized by the CCP for a reason, and it's one that might well come back to bite.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#67

Post by SwellJoe »

manmal wrote: post #58
epolanski wrote: post #54

Earlier quoted context omitted.

I have been saying that from multiple of my tests you can use Claude Code with DS4 Pro or Flash (you just swap api keys) at more or less equivalent performance and people keep screaming "that it's not SOTA". I don't know whether models are over fitted to benchmarks and people take them at face value, but I spend less on DS4 apis than I do for Claude Code 100$ subscription and I code everyday. So far I'm quite happy w…

Are you not worried about where your data will end up? By now I‘m feeding things to Codex that I‘d rather not have in a leak.

These days I'm also worried about US companies having my data. I hate that we're at that point, but with Trump talking about taking an ownership stake in AI companies, and tech companies, including the leading AI companies, lining up to participate in the war crime of the day, I don't have a lot of faith my data is any safer with US companies than those in China.

Though, I added Mistral's latest model to the mix in the hope that some European model could be a contender, but it failed completely. I don't know if it hit safety guardrails or is just not competent at security work, but it scored 0/9. No errors, it returned the empty JSON set it was supposed to return if it didn't find anything. But, there were plenty of real bugs to find, and some very small self-hosted models found at least some of them.

Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision

#70

Post by scrollop »

jodacola wrote: post #17

Curious for folks who have made the switch I’m considering: if I swapped Claude Code to DeepSeek API pricing, would I get more bang for my buck compared to the $100 Max plan I’m using now? I only hit the 5 hour limit every few days and the weekly limit a day or two before it resets at the most aggressive. I wouldn’t expect my usage to increase dramatically, other than not being stopped by limits. I’m still apprehensi…

I'd recommend carefully looking at a few benchmarks (even though generally relying on benchmarks is problematic)

https://artificialanalysis.ai/evaluations/omniscience

Esp check the Hallucination rate for Deepseek - it's not good.

Post reply on HN