A few words on DS4

#1

Post by caust1c » Thu, May 14, 2026, 10:29 PM UTC

Re: A few words on DS4

#2

Post by bjconlan » Thu, May 14, 2026, 10:45 PM UTC

This is great! I feel the same way about the deepseek v4 architecture for commodity hardware.

Also have enjoyed playing with https://huggingface.co/HuggingFaceTB/nanowhale-100m-base (but early days for me understanding this space)

Re: A few words on DS4

#3

Post by simonw » Thu, May 14, 2026, 11:11 PM UTC

I got this running on a 128GB M5 the other day - pretty painless, model runs in about 80GB of RAM and it seemed to be very capable at writing code and tool execution.

Re: A few words on DS4

#4

Post by perfmode » Thu, May 14, 2026, 11:21 PM UTC

simonw wrote: post #3

I got this running on a 128GB M5 the other day - pretty painless, model runs in about 80GB of RAM and it seemed to be very capable at writing code and tool execution.

How’s the token throughput / response time?

Re: A few words on DS4

#5

Post by simonw » Thu, May 14, 2026, 11:24 PM UTC

perfmode wrote: post #4

simonw wrote: post #3

I got this running on a 128GB M5 the other day - pretty painless, model runs in about 80GB of RAM and it seemed to be very capable at writing code and tool execution.

How’s the token throughput / response time?

Healthy!

  prefill: 30.91 t/s, generation: 29.58 t/s

From https://gist.github.com/simonw/31127f9025845c4c9b10c3e0d8612...

Re: A few words on DS4

#6

Post by kamranjon » Thu, May 14, 2026, 11:28 PM UTC

Just want to mention that I've been pulling down and using DwarfStar locally and it's incredible. I actually have it running on my personal macbook m4 max with 128gb of ram and I am running the server to share it through tailscale with my work laptop and just have pi running there.

The long context reasoning is something I haven't even seen in frontier models - I was running at 124k tokens earlier and it was still just buzzing along with no issues or fatigue.

I am amazed at how well it works, I'm using it right now for some pretty complex frontend work, and it is much much faster than, for example running a dense 27b or 31b model (like qwen or gemma) for me (The benefits of MoE) - but the long context capabilities have been what have been absolutely flooring me.

Super excited about this project and hope Antirez can keep himself from burning out - i've been following the repo pretty closely and there are a ton of PR's flooding in and it seems like he's had to do a lot of filtering out of slop code.

Re: A few words on DS4

#7

Post by xienze » Thu, May 14, 2026, 11:34 PM UTC

simonw wrote: post #5

perfmode wrote: post #4

Earlier quoted context omitted.

How’s the token throughput / response time?

Healthy! prefill: 30.91 t/s, generation: 29.58 t/s From https://gist.github.com/simonw/31127f9025845c4c9b10c3e0d8612...

I don't want to be a jerk but 31t/s prefill is basically unusable in an agentic situation. A mere 10k in context and you're sitting there for 5+ minutes before the first token is generated.

Re: A few words on DS4

#8

Post by le-mark » Thu, May 14, 2026, 11:37 PM UTC

kamranjon wrote: post #6

Just want to mention that I've been pulling down and using DwarfStar locally and it's incredible. I actually have it running on my personal macbook m4 max with 128gb of ram and I am running the server to share it through tailscale with my work laptop and just have pi running there. The long context reasoning is something I haven't even seen in frontier models - I was running at 124k tokens earlier and it was still ju…

Is DS4 dwarf star 4 or deep seek 4?

Re: A few words on DS4

#9

Post by aiscoming » Thu, May 14, 2026, 11:37 PM UTC

xienze wrote: post #7

simonw wrote: post #5

Earlier quoted context omitted.

Healthy! prefill: 30.91 t/s, generation: 29.58 t/s From https://gist.github.com/simonw/31127f9025845c4c9b10c3e0d8612...

I don't want to be a jerk but 31t/s prefill is basically unusable in an agentic situation. A mere 10k in context and you're sitting there for 5+ minutes before the first token is generated.

if it's just the coding agent system prompt and tools, you can cache that

Re: A few words on DS4

#10

Post by wolttam » Thu, May 14, 2026, 11:38 PM UTC

le-mark wrote: post #8

kamranjon wrote: post #6

Just want to mention that I've been pulling down and using DwarfStar locally and it's incredible. I actually have it running on my personal macbook m4 max with 128gb of ram and I am running the server to share it through tailscale with my work laptop and just have pi running there. The long context reasoning is something I haven't even seen in frontier models - I was running at 124k tokens earlier and it was still ju…

Is DS4 dwarf star 4 or deep seek 4?

DwarfStar 4 is DeepSeek 4 (check the repo)