131–140 of 197 posts
DwarfStar4 is a small LLM inference runtime that can run DeepSeek 4. The blog post implies that it currently requires 96GB of VRAM. For others who are lacking context :-)
That's the flash version not the full model and only at Q2-3~ so while impressive it's still quite different from the full model.
Earlier quoted context omitted.
You can buy 128GB M5 MacBook Pros? Configured one just now, delivers in 2 weeks
Interesting there were news last week or so of apple removing Mac minis options.
Wink wink, nudge nudge.
I have a feeling most cybersec researchers would only be interested in negative values of "reduce" :D
Earlier quoted context omitted.
In my (obviously anecdotal) testing, DeepseekV4 Pro was better than Sonnet at coding. However, it is much slower, but also many times cheaper, especially with the promotion right now.
Do they have a coding plan or you only pay per API call?
DwarfStar4 is a small LLM inference runtime that can run DeepSeek 4. The blog post implies that it currently requires 96GB of VRAM. For others who are lacking context :-)
The code seems based on llama.cpp and GGML.
I don't fully understand why it is a standalone project. The readme discusses this: DwarfStar 4 is a small native inference engine specific for DeepSeek V4 Flash. It is intentionally narrow: ...
I think the only bigger difference in DeepSeek V4 vs other models is maybe the type of self-attention. And that leads to: KV cache is actually a first-class disk citizen.
But I still feel like those changes could have been implemented as part of some of the other local engines.
I also assume more models will come out, not just from DeepSeek but also from others, and they might share similar self-attention approaches, that would benefit from a similar KV cache implementation.
This project is a week old and already super popular. Guess people really were tired of lmstudio or tunning llama.cpp with settings.