[flagged]
DeepSeek V4 Pro beats GPT-5.5 Pro on precision
131–140 of 249 posts
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#132Earlier quoted context omitted.
Are you not worried about where your data will end up? By now I‘m feeding things to Codex that I‘d rather not have in a leak.
Yes, that's exactly why I avoid OpenAI and Anthropic products. Besides the (quite true) joke, if sending data to DeepSeek is a concern the good thing is that the models are open weight, you can self host them or use third party providers.
You can theoretically self-host. DeepSeek is big. DS4 (the 2-bit quantization of DeepSeek Flash) runs on my Strix Halo with 128GB, but it's slow as hell. Completely unusable for interactive work. But, I guess a company that cared about data privacy and wanted a Good Enough local model could spend $100,000 or more on hardware to run it properly.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#133Earlier quoted context omitted.
My personal observation (using a mix of opencode and pi harness): 1. DS4Pro: around opus 4.5 2. DS4Flash: around sonnet 4 3. Mimo v2.5 pro: between opus 4.5 and opus 4.6. 4. minimax M3: around opus 4.6 All of these are very close in terms of quality and pricing. For anything that is not specifically related to coding, DS4Flash has become ny de-factor model. It just works... super fast, tool calling is perfect, and th…
i have been using deepseek-v4-flash since it came out. i use a highly structured harness and spec/test driven workflow running through opencode, and so far there has been nothing it can't do. i have run through a bunch of tests: re-writing vvenc with assembly kernels, creating the first generation agent harness integration with opencode, porting TS npm modules to C++, porting an entire TS server app to C++, creating…
Thank you a lot for such an insightful comment. The low level stuff part, including porting entire codebases using DV4Flash came as a genuine surprise to me. I did not expected it to be this good.
When you say "i use a highly structured harness" ... can you please tell me what is it exactly?
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#134Earlier quoted context omitted.
I've consistently tried to apply LLMs to physics problems and they're utterly useless. They'll just confidently lie, or blatantly plagiarise source materials The issue is once you hit niche physics simulations there simply isn't any training data available, so the limitations of them become incredibly apparent. Its also problematic because a field itself will contain lots of wrong information (its research!), and AI…
That's why there are companies specialising in AI for physics, like Emmi AI (now part of Mistral). If BMW and Airbus go on stage to talk about how they're using it for their physics simulations, it's probably at least decent.
Usage isn't really a good indicator of quality currently in the AI space, the issue is that there's inherently no way that an AI physics sim can be as good as a real physics simulation, which makes it a very low value prospect
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#135Earlier quoted context omitted.
That's why there are companies specialising in AI for physics, like Emmi AI (now part of Mistral). If BMW and Airbus go on stage to talk about how they're using it for their physics simulations, it's probably at least decent.
Usage isn't really a good indicator of quality currently in the AI space, the issue is that there's inherently no way that an AI physics sim can be as good as a real physics simulation, which makes it a very low value prospect
Usage by reputable engineering organisations with strict compliance and external testing validation (most notably Airbus, they have to prove to EASA that their tests are real and representative) is a decent indicator that there is something there.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#136Earlier quoted context omitted.
Yes, that's exactly why I avoid OpenAI and Anthropic products. Besides the (quite true) joke, if sending data to DeepSeek is a concern the good thing is that the models are open weight, you can self host them or use third party providers.
You can theoretically self-host. DeepSeek is big . DS4 (the 2-bit quantization of DeepSeek Flash) runs on my Strix Halo with 128GB, but it's slow as hell. Completely unusable for interactive work. But, I guess a company that cared about data privacy and wanted a Good Enough local model could spend $100,000 or more on hardware to run it properly.
DS4 flash runs okay on MacBook Pro though:
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#137Earlier quoted context omitted.
Usage isn't really a good indicator of quality currently in the AI space, the issue is that there's inherently no way that an AI physics sim can be as good as a real physics simulation, which makes it a very low value prospect
Usage by reputable engineering organisations with strict compliance and external testing validation (most notably Airbus, they have to prove to EASA that their tests are real and representative) is a decent indicator that there is something there.
Do we have real case studies, or just a bunch of declarations? "Using AI for our physics simulations" is as vague as it can be.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#138Earlier quoted context omitted.
Yes, that's exactly why I avoid OpenAI and Anthropic products. Besides the (quite true) joke, if sending data to DeepSeek is a concern the good thing is that the models are open weight, you can self host them or use third party providers.
You can theoretically self-host. DeepSeek is big . DS4 (the 2-bit quantization of DeepSeek Flash) runs on my Strix Halo with 128GB, but it's slow as hell. Completely unusable for interactive work. But, I guess a company that cared about data privacy and wanted a Good Enough local model could spend $100,000 or more on hardware to run it properly.
The DS4 author has demoed upcoming work on Strix Halo that makes it roughly competitive with the Apple Silicon equivalent (i.e. Pro models with similar memory bandwidth figures, not Max or Ultra). Maybe even a bit faster for prefill, and with further potential for running small batches in parallel (since the GPU clearly has some amount of compute headroom during decode).
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#139These tests are looking increasingly like a waste of time. The "intelligence" is clearly there now. Trying to measure it seems pointless. I can't shop for hammers at the hardware store and sort by the quality of finished products they would produce. That is clearly an insane ask, but that's approximately what is being pushed for with these models now. Domain specificity (harness & environment) is where the magic happ…
> I can't shop for hammers at the hardware store and sort by the quality of finished products they would produce.
What? You can and you should. That's exactly what product tests are enabling you to do. If you need a glue, you want to look at someone who tried to glue some things with few glues so you know what to roughly expect form which specific glue.
Re: DeepSeek V4 Pro beats GPT-5.5 Pro on precision
#140These tests are looking increasingly like a waste of time. The "intelligence" is clearly there now. Trying to measure it seems pointless. I can't shop for hammers at the hardware store and sort by the quality of finished products they would produce. That is clearly an insane ask, but that's approximately what is being pushed for with these models now. Domain specificity (harness & environment) is where the magic happ…
"the intelligence is clearly there" I wonder if I am using the same models as everyone else. To me, LLMs still give good answers 80% of the time, but 20% it fails in such a miserable way that makes it obvious that the "intelligence" is not there.
GPT-5.5, 100% so far for all of my problems that actually have an anwser.