Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek V4 – almost on the frontier

simonwillison.net

Post reply on HN
Print view View original

291–300 of 420 posts

Re: DeepSeek V4 – almost on the frontier

#291

Post by djmips »

segmondy wrote: post #119
cheshire_cat wrote: post #69

While the cost are lower than frontier models there are two factors that make DS4 Pro and K2.6 not as cheap as they might look. For DS4 Pro there's a discount going on for the official API, which sometimes gets overlooked and mixed up in discussions. Simon uses the full price in the comparison, so that's not an issue here. The other issue is that DS4 Pro and K2.6 often use way more reasoning tokens than the frontier…

This is very false DS4 is super cheap. I would advise to begin by reading their release paper. https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/main... They introduce very novel methods to improve long context efficiency and attention. HCA & mCH. It requires only 27% of flops for inference and 10% for KV cache than v3.2. This makes it super efficient. Think of this. For flops, we can now serve more than 3x the…

No offense but everything comments about local models without telling their GPU setup and VRAM so it's pretty useless information.

Re: DeepSeek V4 – almost on the frontier

#292

Post by rurban »

gigatexal wrote: post #283
mrbonner wrote: post #275

Earlier quoted context omitted.

Your method of combining models to strengthen the implementation reminds me of how we form stronger alloys by combining metals!

it also sounds like a lot to manage, do you have some sort of agentic framework that's treating all of these llm's you have access to as sort of inputs that it optimizes?

Unfortunately not. I'm using plain kimi, opencode (with deepseek, gpt, minmax, whatever) and claude. claude is the best, but only for some hours. The trick is to get a good AGENTS.md file, good test cases and test runner to repro, like seemless docker and qemu calls. GNU autotools would be easiest, but here I'm using plain makefiles. Also for LSP clangd being up-to-date a compile_commands.json is important. git worktrees helped developing the arm port and fixing c-testsuite cases in parallel. I wanted to keep the costs down. About $15-$30 I think.

And for low-level problems, like ARM calling-convention in asm, those models are much better than simple algorithmic python problems. Just for the hardest problem I needed the big expensive gun, but never opus. This helps in deciding what to do with my next jit project.

Re: DeepSeek V4 – almost on the frontier

#293

Post by Accacin »

I tried DeepSeek via chat, and gave it a rather simple question:

"Can you tell me who was on series 8 of Taskmaster, and what's the general opinion about the series? No spoilers!"

It told me amongst other things that Paul Sinha was diagnosed with Parkinsons, as well as who the winner was.

Then I said, "But I said no spoilers!"

And it apologised for telling me Paul Sinha was diagnosed with Parkinsons.

Re: DeepSeek V4 – almost on the frontier

#294

Post by i-blis »

grassfedgeek wrote: post #171
cedws wrote: post #113

The biggest differentiator for me: DeepSeek just does what I ask. I've tried using both GPT and Claude for reverse engineering recently, both refused. I even got a warning on my OpenAI account.

Are you kidding? Ask this question and see what answer you get: What famous photo depicts a man standing in front of a line of tanks?

[flagged]

Re: DeepSeek V4 – almost on the frontier

#295

Post by zozbot234 »

sylware wrote: post #280
zozbot234 wrote: post #63

Earlier quoted context omitted.

DeepSeek V4 Pro has about 25GB worth of active parameters, so if you can fit the whole ~870GB weights + cache in RAM your tok/s is bounded above by 25GB divided into your system memory bandwidth in GB/s. If you can't fit your whole model in RAM you'll be bottlenecked to some degree by storage bandwidth which is in the single or low double digits in GB/s. Mind you, it's an absolutely sensible setup either way if you a…

Let's say I get 32GB of RAM, with a lean elf(glibc)/linux system, for which 7GB is beyond enormous to run. Let's book 8/16 cores/threads to run a prompt. What are the timing figures I am looking at to run an "average" coding prompt?

The basic bottleneck with 32GB RAM would be your storage, so for a baseline estimate you'd be looking at anything from ~2 secs per token (if you had really high performance PCIe 5.0 SSD at ~14 GB/s max) to ~5 secs per token (for an average PCIe 4.0 SSD, ~7 GB/s max). This would then be boosted by being able to keep the shared model layers in RAM, since these are part of the 25GB active parameters. I'm not sure what fraction of the active params that makes up for DeepSeek V4 Pro, but in a typical MoE it's about half, so you could approximately halve those secs-per-token figures. That's acceptable if you care about unattended inference for testing purposes or simple Q&A (leveraging the model's vast world knowledge); it doesn't look very good for interactive use. But the flip side is that you can batch a large amount of model queries together, since the KV cache for very short prompts is quite negligible. AIUI, that's basically unique to this series of models and a huge selling point.

Re: DeepSeek V4 – almost on the frontier

#298

Post by 0xbadcafebee »

eckelhesten wrote: post #287
0xbadcafebee wrote: post #198

Earlier quoted context omitted.

If you're not Chinese, and you start a company outside of China, and your whole pitch is "We run open weights and we have nothing to do with China", 1) why would send data to China?? 2) why would you risk your business to do a thing that makes no sense?

Well, the context was running the models via open router, not hosting 800B> models yourself. Of course, if given the option I believe most people would pick ”don’t share sensitive data”. What I’m trying to say is that EVERYONE uses your data, even the sensitive type. So you might aswell use an endpoint that does what it says and treat EVERY endpoint whether that’s OpenAI or anthropic as if it’s collecting all of your…

No, not everyone uses your data. There are providers who very explicitly do not collect or use your data.

Re: DeepSeek V4 – almost on the frontier

#299

Post by nsingh2 »

cedws wrote: post #113

The biggest differentiator for me: DeepSeek just does what I ask. I've tried using both GPT and Claude for reverse engineering recently, both refused. I even got a warning on my OpenAI account.

I've been using GPT-5.4, and more recently 5.5, with Codex CLI + Ghidra MCP for reverse engineering a game without many issues. Injecting code is where it usually balks at, but I'm just trying to discover and parse structures from game memory.

I did get a refusal when trying to read in-game currency, even though modifying it would do nothing. It has some strange boundaries.

Re: DeepSeek V4 – almost on the frontier

#300

Post by SeriousM »

rgbrgb wrote: post #279
rurban wrote: post #253

Earlier quoted context omitted.

Well, I'm using all the top models extensively on the very same codebase, my new compiler. I use deepseek for it's cheap API costs, when kimi, claude and codex are in their overbudget phase. I asked deepseek V4 Pro for an estimate of a new arm64 port. It said 4 weeks, I said, ok, do it. (I knew ncc was there, and tinycc was also known to the AI's). So it took it half an hour to produce a working arm64 port. First for…

what harness do you use with all of these?

It really sounds like pi.dev
Post reply on HN