DwarfStar4 is a small LLM inference runtime that can run DeepSeek 4. The blog post implies that it currently requires 96GB of VRAM. For others who are lacking context :-)
I knew Death Stranding 3 wasn't out yet!
A few words on DS4
111–120 of 197 posts
Re: A few words on DS4
#112Earlier quoted context omitted.
It’s just per token, but burning up 100 million+ tokens is a $3 transaction with their pricing right now
Do you use the official API or another provider?
I use the official API, OpenRouter somehow didn't use caching and one short session with Qwen cost me $5.
Re: A few words on DS4
#113A relevant recent tweet from antirez: https://x.com/antirez/status/2054854124848415211 > Gentle reminder on how, in the recent DS4 fiesta, not just me but every other contributor found GPT 5.5 able to help immensely and Opus completely useless. I've noticed the same for lower level squeezing-as-much-performance-as-possible code work.
There's so much subjectivity with models. As soon as a new model comes out people act like the last model they used for 6 months was completely useless.
Re: A few words on DS4
#114Earlier quoted context omitted.
it's almost certainly not true yet but at some point there might be an equilibrium reached of speed Vs quality (and let's not forget, cost) where it's true for most of what you do. Perhaps you'd still turn to hosted models for the hardest tasks, but most tasks go local. It does seem like that would make demand go down significantly. Of course that's all predicated on model advances plateauing, or at least getting inc…
Why is this presumed to be de facto inevitable: * local compute isn’t scaling as before, so algorithmic improvements are the only ways models get meaningfully faster and smarter * all those same algorithmic improvements would also be true for larger models * hardware manufacturers have an incentive against local LLMs because cloud LLMs are so much more lucrative (+ corps would by desktop variants if they were good en…
> It may be good enough for what you want but there will always be a harder problem that you need to throw more compute and more memory at.
Sure, but if the “good enough for what you want” consumes the vast majority of cases - data-center ai becomes just for the very extreme edge cases. Like how I can render a 4k rez video game at 60fps on my home pc, but if pixar wants to render their next movie they use data-center compute.
> all those same algorithmic improvements would also be true for larger models
Smaller models run faster. If ten runs of a small model gets me the same quality result as one run of the big model, and the small model runs 10x faster, then they are functionally the same.
Re: A few words on DS4
#115Re: A few words on DS4
#116Re: A few words on DS4
#117Earlier quoted context omitted.
Why is this presumed to be de facto inevitable: * local compute isn’t scaling as before, so algorithmic improvements are the only ways models get meaningfully faster and smarter * all those same algorithmic improvements would also be true for larger models * hardware manufacturers have an incentive against local LLMs because cloud LLMs are so much more lucrative (+ corps would by desktop variants if they were good en…
> It may be good enough for what you want but there will always be a harder problem that you need to throw more compute and more memory at. Sure, but if the “good enough for what you want” consumes the vast majority of cases - data-center ai becomes just for the very extreme edge cases. Like how I can render a 4k rez video game at 60fps on my home pc, but if pixar wants to render their next movie they use data-center…
> Like how I can render a 4k rez video game at 60fps on my home pc, but if pixar wants to render their next movie they use data-center compute.
This is a very nice analogy actually and it impacts the whole story about US vs. Chinese leadership in "frontier AI".
Re: A few words on DS4
#118This project is a week old and already super popular. Guess people really were tired of lmstudio or tunning llama.cpp with settings.
Re: A few words on DS4
#119Love this, even if can't use it atm (not got the h/w - only 96gb on M2 Max). I get it the general comp/public will find it unusable or worse. Reminds me of how home computers were - mere toys - before they became personal computers (PC). On my h/w the only passable combo for me atm is pi agent + llama.cpp + nemotron cascade-2 model: to 1M context, hybrid arch doesn't crash & burn 1/N^2 with context depths of 10K-50K-100K used by code agents. Was on a plane without Internet the other day. Brought a smile to my face that I could run pi agent (with llama.cpp serving), and it was just about usable at 40-30 tok/s. Afaik the usual API speeds are double that, 60-80 tok/s. Sensors showing using 60W when running inference. So battery probably would not last more than >3h. Model only 30B in size leaves plenty of space for KV-caches, and other programs - even at generous 8-bit quant. Only 3B active params at one time (with MoE A3B) is about the most that ageing M2 Max can carry it seems.
Re: A few words on DS4
#120I've been using the Q4 version on my Mac Studio over my local network and it's been good. Indeed, I had the first ever experience where I was playing with it alongside my various other agents and forgot it was a local model as it was doing such a good job.
I do wonder, though, if another agent is really needed. I've been driving it with Pi (Claude Code's system prompt is far too heavy given the prefill speeds) and it's been great. OpenCode is another good option. Is there anything else to gain from another similar tool specific to Deepseek 4?