Earlier quoted context omitted.
i have been using deepseek-v4-flash since it came out. i use a highly structured harness and spec/test driven workflow running through opencode, and so far there has been nothing it can't do. i have run through a bunch of tests: re-writing vvenc with assembly kernels, creating the first generation agent harness integration with opencode, porting TS npm modules to C++, porting an entire TS server app to C++, creating…
Thank you a lot for such an insightful comment. The low level stuff part, including porting entire codebases using DV4Flash came as a genuine surprise to me. I did not expected it to be this good. When you say "i use a highly structured harness" ... can you please tell me what is it exactly?
DeepSeek V4 Pro beats GPT-5.5 Pro on precision
171–180 of 249 posts
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#172Earlier quoted context omitted.
Have you even used deepseek pro/flash? Yes, it is astroturfed to the maxx. There is a reason for that. The performance/price ratio beats anything available today.
"Don't you understand? I'm on team deepseek! It doesn't matter what's written about it. Heck it doesn't even matter if it's all lies - it supports my team and here's why I love my team."
The only thing I could read from your posts is that you are team openai and completely mad that people are abandoning chatgpt
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#173These tests are looking increasingly like a waste of time. The "intelligence" is clearly there now. Trying to measure it seems pointless. I can't shop for hammers at the hardware store and sort by the quality of finished products they would produce. That is clearly an insane ask, but that's approximately what is being pushed for with these models now. Domain specificity (harness & environment) is where the magic happ…
> clearly there
Already the fact that we could have to ask "there where", the fact that we have met clearly unintelligent bots, creates a requirement about defining where it (intelligence) is and investigating what put it there, to get the warranties that intelligence will be met consistently, structurally, and not casually, apparently.
Casual use, casual tool; mission critical use, certified tool.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#174imo there are two major kinds of models
there are models you can speak with, that respond to what you say
and there are models that just make lists, that list everything, include weird formats and add asteriks everywhere.
deepseek, to me, will always be the latter, and i can't stand it, you can't ask it a coherent question and get a coherent response.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#175Curious for folks who have made the switch I’m considering: if I swapped Claude Code to DeepSeek API pricing, would I get more bang for my buck compared to the $100 Max plan I’m using now? I only hit the 5 hour limit every few days and the weekly limit a day or two before it resets at the most aggressive. I wouldn’t expect my usage to increase dramatically, other than not being stopped by limits. I’m still apprehensi…
Deepseek cost/performance is incredible. That said, I still feel like for agentic coding we haven't plateaued (I slightly prefer GPT 5.5 to Claude for complex stuff, to be honest), and so the extra price is absolutely worth it to push you over the 'impossible' to 'feasible' bar on complex tasks. Once you're in a domain that Deepseek can handle though that requires volume, I would almost always default to it now.
For evals in particular (tuning workflows that agents are using), effectively not having to worry about price is an incredible multiplier - getting statistical significant signal is not cheap otherwise.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#176i tried deepseek, while the model is good, when i use it with openrouter hosted ones the performance is poor. sometimes it takes 2x-3x the time it takes for openai or anthropic equivalent model, making it unusable. what is the performance others are seeing, which providers you use (i cant use china hosted models).
That's about what we've seen as well (even directly from deepseek themselves). We've been using it for async "heartbeat" processing and sms replies, but it's just too slow for live chat replies (which is a shame, as I'd really love to use it there). Very capable model, but also very slow.
That isn't what the charts on OpenRouter appear to show but they only seem to go back 1 week (unless I missed something). It should be less than 2 seconds to first token and anywhere from 15 to 50 tps depending on the provider. Admittedly 15 is a bit slow but most look to be closer to 30 or 40 which at least personally I think is fine.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#177Earlier quoted context omitted.
Thank you a lot for such an insightful comment. The low level stuff part, including porting entire codebases using DV4Flash came as a genuine surprise to me. I did not expected it to be this good. When you say "i use a highly structured harness" ... can you please tell me what is it exactly?
https://github.com/opensassi/opencode
Thanks..
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#178DeepSeek V4 Pro is wonderful and ridiculously cheap, but we are sleeping on MiMo V2.5 Pro, which have the same price (and lower cached price), it's multimodal and it's higher up in most benchmarks. Same thing for MiMo V2.5 vs DeepSeek V4 Flash.
> MiMo V2.5 Pro ... lower cached price At the moment of writing https://news.ycombinator.com/item?id=48343690 MiMo V2.5 Pro had a lower cache hit ratio. From the article: OSS models, depending on who you use them from, make a huge difference, mostly due to cache-hit rates. Model Cheapest effectiveInputPrice (Provider) MiMo-V2.5-Pro 0.3720 (Xiaomi) DeepSeek V4 Pro (Max) 0.0560 (DeepSeek)
Could it be that it changed recently, or am I missing something? Both prices are the same https://openrouter.ai/compare/xiaomi/mimo-v2.5-pro/deepseek/...
EDIT: okay I misread it, does this mean that DeepSeek reuses a higher percentage of tokens at cache price that MiMo, am I right?
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#179These tests are looking increasingly like a waste of time. The "intelligence" is clearly there now. Trying to measure it seems pointless. I can't shop for hammers at the hardware store and sort by the quality of finished products they would produce. That is clearly an insane ask, but that's approximately what is being pushed for with these models now. Domain specificity (harness & environment) is where the magic happ…
"the intelligence is clearly there" I wonder if I am using the same models as everyone else. To me, LLMs still give good answers 80% of the time, but 20% it fails in such a miserable way that makes it obvious that the "intelligence" is not there.
Can I ask what your task and application is? A ~20% failure rate sounds atypical. If you’re slightly hyperbolic and mean something like 2-5%, yeah that’s a property of LLMs; but also heavily affected by how you prompt and how you constrain the task.
An auditing/QA step (whether a grading checklist, verification, etc) can get you further. Likewise for a planning step.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#180These tests are looking increasingly like a waste of time. The "intelligence" is clearly there now. Trying to measure it seems pointless. I can't shop for hammers at the hardware store and sort by the quality of finished products they would produce. That is clearly an insane ask, but that's approximately what is being pushed for with these models now. Domain specificity (harness & environment) is where the magic happ…
> Domain specificity (harness & environment) is where the magic happens next.
not really. it happens in training and RL. your harness is not going to override what it has been trained to do.
sure harness is useful if you are trying to build crud websites if model is trained on stamping out crud websites. But thats just a waste of time remxing things better.