Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A few words on DS4

antirez.com

Post reply on HN
Print view View original

141–150 of 197 posts

Re: A few words on DS4

#141

Post by theturtletalks »

driese wrote: post #104
simpaticoder wrote: post #81

Earlier quoted context omitted.

An M5 Max MBP with 128G of RAM costs ~$5k. An Nvidia RTX 5090 with 32G RAM is $4-5k, and RTX PRO 6000 with 96GB RAM $10k. Do you have any data on which is the best price/performance for local inference? Do you know what the big OpenAI/Anthropic/Google datacenters are running?

As always: it depends on your needs. Here's a very basic heuristics rundown: - More RAM: bigger models, more intelligence. - More FLOPs: higher pre-fill (reading large files and long prompts before answering, the so-called "time to first token"). - More RAM bandwidth: higher token generation (speed of output). So basically Macs (high RAM, okay bandwidth, lowish FLOPs) can run pretty intelligent models at an okay outp…

Do you think Apple will fix prefill speed with the M6 Max MacBook Ultra 128GB?

Re: A few words on DS4

#142

Post by kaoD »

zmmmmm wrote: post #92
skybrian wrote: post #78

Earlier quoted context omitted.

I imagine we'll get to "good enough" for hobbyist programmers fairly quickly, but businesses will still be willing to pay more for faster and smarter. Why make your programmers wait?

> Why make your programmers wait? That depends on where the methodology goes. But more and more it's hands off. If the trajectory continues it won't matter because nobody is sitting their waiting / watching the LLM code anyway. It is all happening in the background. We might see hybrid approaches where the weaker / cheaper agent tries to solve it and just "asks for help" from the more expensive agent when it needs it…

> nobody is sitting their waiting / watching the LLM code anyway

My personal experience is that for production-grade code you need to steer the agent more often than not... so yes, at least some of us are watching the LLM code.

Re: A few words on DS4

#143

Post by skiwithuge »

albertzeyer wrote: post #138

More information about DwarfStar 4 (DS4) in the readme: https://github.com/antirez/ds4 The code seems based on llama.cpp and GGML. I don't fully understand why it is a standalone project. The readme discusses this: DwarfStar 4 is a small native inference engine specific for DeepSeek V4 Flash. It is intentionally narrow: ... I think the only bigger difference in DeepSeek V4 vs other models is maybe the type of self-at…

because llama.cpp doesn't accept fully pr made by ai agents even if they are guided by the author

https://github.com/ggml-org/llama.cpp/blob/master/AGENTS.md

Re: A few words on DS4

#144

Post by wg0 »

DeepSeekV4 Pro is really really competent model and what makes it extremely good is the price point it is offered at.

I have been toying with a 2.5D engine in C on on top of raylib and using DeepSeek as companion in between.

It's thinking transcripts in OpenaCode are transparent and mind boggling to look at things it would consider in its thought process. Very long to read but none of them useless or meaningless.

Always happened that I discovered an assumption that I didn't think about or was wrong but DeepSeek flags it in its thought process and then in final output it would "align" to my flawed request and I'll tell it wait, I saw you thought so and so too and that's correct I made a mistake let's consider that aspect too.

Re: A few words on DS4

#145

Post by trollbridge »

DeathArrow wrote: post #93
trollbridge wrote: post #85

Earlier quoted context omitted.

It’s just per token, but burning up 100 million+ tokens is a $3 transaction with their pricing right now

Do you use the official API or another provider?

Just directly. Paid for it with PayPal. It’s quite simple to set up and use.

Re: A few words on DS4

#146

Post by embedding-shape »

ljosifov wrote: post #119

Love this, even if can't use it atm (not got the h/w - only 96gb on M2 Max). I get it the general comp/public will find it unusable or worse. Reminds me of how home computers were - mere toys - before they became personal computers (PC). On my h/w the only passable combo for me atm is pi agent + llama.cpp + nemotron cascade-2 model: to 1M context, hybrid arch doesn't crash & burn 1/N^2 with context depths of 10K-50K-…

> even if can't use it atm (not got the h/w - only 96gb on M2 Max).

Not sure if it works different on macOS, but with CUDA + DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf I can fit it within 96GB of VRAM, together with context, so theoretically I feel like you should too, unless macOS uses GB of RAM/VRAM for the OS/display by default.

Re: A few words on DS4

#147

Post by embedding-shape »

skiwithuge wrote: post #143
albertzeyer wrote: post #138

More information about DwarfStar 4 (DS4) in the readme: https://github.com/antirez/ds4 The code seems based on llama.cpp and GGML. I don't fully understand why it is a standalone project. The readme discusses this: DwarfStar 4 is a small native inference engine specific for DeepSeek V4 Flash. It is intentionally narrow: ... I think the only bigger difference in DeepSeek V4 vs other models is maybe the type of self-at…

because llama.cpp doesn't accept fully pr made by ai agents even if they are guided by the author https://github.com/ggml-org/llama.cpp/blob/master/AGENTS.md

Which makes sense, the amount of PRs llama.cpp receives from authors who have no clue what they're doing and can't even answer simple questions about "what they did" is staggering, must be very exhausting to have to figure out "is it worth replying to this author?" for every single PR.

Re: A few words on DS4

#148

Post by embedding-shape »

DeathArrow wrote: post #94
thomasm6m6 wrote: post #83

Earlier quoted context omitted.

FYI, llama.cpp (which antirez/ds4 is inspired by) supports system ram. E.g. [1] is a good guide for running a similar-sized model with 128gb ram and a 3090-sized GPU. [1] https://unsloth.ai/docs/models/tutorials/minimax-m27 (Unsloth's deepseek-v4 support is still WIP)

Thanks, I can run Qwen 3.6 27B with vllm, but I was curious about antirez tool.

Have you had it getting stuck in endless loops maybe ~10-20% of the invocations? Seems it happens for both the responses and chatcompletion APIs, and no matter what inference parameters I try it happens at least for 1/10 of the requests, I've tried every compatible vLLM version + currently using it from git (#main) yet the issue persists.

Seems to happen with various quantizations too, even the NVFP4 versions and any others, so seems like a deeper issue to me, or hardware incompatible perhaps.

Re: A few words on DS4

#149

Post by neomantra »

antirez wrote: post #126
petercooper wrote: post #120

I've been using the Q4 version on my Mac Studio over my local network and it's been good. Indeed, I had the first ever experience where I was playing with it alongside my various other agents and forgot it was a local model as it was doing such a good job. I do wonder, though, if another agent is really needed. I've been driving it with Pi (Claude Code's system prompt is far too heavy given the prefill speeds) and it…

There is no need for another agent, functionally. But if you follow the idea of DS4 itself: the API agents use forces to do odd things, like translating the DSML stanzas to JSON, with all the canonicalization / KV cache checkpointing problems resulting from that. Is it really the case? What about also providing a sane alternative? Also I'm not sure why people don't try to write more stuff in that area in C/Go/Rust to…

For Golang, I highly recommend yzma to explore this surface. I’ve used it for embedding and summarization (with small models) and just mucking around with integrated LLM BubbleTea TUI idea (with bigger models).

https://github.com/hybridgroup/yzma

And thank you antirez for using your rep and quality output to push this line of evangelism; it is even more important than the software itself.

Re: A few words on DS4

#150

Post by wolttam »

nl wrote: post #96
zmmmmm wrote: post #55

I'm very curious where we will saturate the curve on "enough" intelligence for coding. At some point, you can let a less smart model hammer at a problem for longer and get to the same result, and as long as you are not involved it comes to the same thing. I feel like DeepSeek V4 Pro is nearly there. Maybe Flash is too. Once we hit that point, I am curious how much of Anthropic's current business model falls apart? So…

Kilo (the open source coding agent) tested Deepseek v4 Pro and Flash vs Opus 4.7 and Kimi K2[1]. It did ok, but scored substantially less than Opus. It also cost nearly as much, even with the current launch promo pricing for Deepseek. That cost is interesting - I've seen similar things with Sonnet vs Opus, and in my own benchmarking there are some models that benchmark well, seem to have a good price but use so many…

Their pricing shown is without the discount.

> With DeepSeek’s 75% promo applied to current rates, the same run would have cost closer to $0.55, putting it below Kimi K2.6 in absolute cost while scoring 9 points higher.

I will be sad when the discount ends.

Post reply on HN