Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A few words on DS4

antirez.com

Post reply on HN
Print view View original

181–190 of 197 posts

Re: A few words on DS4

#181

Post by ljosifov »

antirez wrote: post #155
ljosifov wrote: post #119

Love this, even if can't use it atm (not got the h/w - only 96gb on M2 Max). I get it the general comp/public will find it unusable or worse. Reminds me of how home computers were - mere toys - before they became personal computers (PC). On my h/w the only passable combo for me atm is pi agent + llama.cpp + nemotron cascade-2 model: to 1M context, hybrid arch doesn't crash & burn 1/N^2 with context depths of 10K-50K-…

It works on your computer I believe. There are a few positive reports.

Thanks for the DS4, will give it a try. Was hoping maybe I can re-quantise shave few GB... MiniMax-M2.7 Unsloth's UD-IQ2_XXS is down to 65GB - it run albeit too slow to be usable to an agent at context depth. I'm curious DS4F with it being economical with the KV caches - if that translates into keeping up with context. Was hoping 80GB 2-bit quants maybe come down to 70GB... that would be more comfortable to run.

Re: A few words on DS4

#182

Post by tredre3 »

rurban wrote: post #165
gekoxyz wrote: post #163

Earlier quoted context omitted.

which harness are you using? pi? opencode?

That's not a harness. That's an agent cli. A harness is something completely different. Wish people could use proper terminology. A test harness is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It automates the execution of test suites, providing the necessary stubs, drivers, and runtime environments so develope…

You're free to fight the terminology if you want (I did at first too), but the zeitgeist has chosen a meaning that disagrees with you, so people will see you as being deliberately obtuse and unpleasant when you fight back.

Learning when to let go is an incredibly important skill that I have learned way too late in life.

Re: A few words on DS4

#183

Post by teknologist »

rurban wrote: post #180
Sinidir wrote: post #170

Earlier quoted context omitted.

Harness: a piece of equipment with straps and belts, used to control or hold in place a person, animal, or object. So yes the generel meaning applies to test setup and running and also to the agent cli which is the harness for the model.

No, an agent cli is no harness. You have to provide a harness for an agent by yourself, otherwise it will run free. Which is called vibe coding. Free as you wish, without any harness.

An agent cli provides a sandbox, with permission systems and auto command classifiers. That’s part of the harness.

Re: A few words on DS4

#184

Post by dolmen »

rurban wrote: post #165
gekoxyz wrote: post #163

Earlier quoted context omitted.

which harness are you using? pi? opencode?

That's not a harness. That's an agent cli. A harness is something completely different. Wish people could use proper terminology. A test harness is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It automates the execution of test suites, providing the necessary stubs, drivers, and runtime environments so develope…

May I ask about your trust issue regarding Google models?

Is it about quality issues (lack of guardrails, agent runs dangerous commands)? I have seen first-hand Gemini-cli going out of the project directory and using my home directory as a work area.

Or is it about terms of service?

Or other concerns?

Re: A few words on DS4

#185

Post by quicklywilliam »

petercooper wrote: post #175
antirez wrote: post #126

Earlier quoted context omitted.

There is no need for another agent, functionally. But if you follow the idea of DS4 itself: the API agents use forces to do odd things, like translating the DSML stanzas to JSON, with all the canonicalization / KV cache checkpointing problems resulting from that. Is it really the case? What about also providing a sane alternative? Also I'm not sure why people don't try to write more stuff in that area in C/Go/Rust to…

I agree philosophically about building more takes on ideas to flesh out ideas. I guess I was querying more the idea about an agent being part of DS4 specifically. I'm 100% up for an "agent by antirez", but I'm intrigued why it would/might be part of DS4 itself. Is there something extra to gain from a tighter coupling between inference and harness? (My gut instinct is.. maybe? I'm guessing Anthropic does stuff like ha…

I think the big idea here is that you can get a lot more performance if you take an integrated approach. This specific model made to work with this specific inference engine made to work with this specific harness/agent. When everything is done separately, developers of a given pieces have no idea what they are targeting for all the other pieces.

This is currently a huge advantage that Anthropic has over open weights models – they control the whole stack. Indeed, they train new models against Claude Code!

It's early days on this project, but just imagine it gets enough traction that future models start training against ds4. Indeed, in the post Antirez even seems to be hinting at some sort of collaboration with DeepSeek?

Re: A few words on DS4

#186

Post by rurban »

dolmen wrote: post #184
rurban wrote: post #165

Earlier quoted context omitted.

That's not a harness. That's an agent cli. A harness is something completely different. Wish people could use proper terminology. A test harness is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It automates the execution of test suites, providing the necessary stubs, drivers, and runtime environments so develope…

May I ask about your trust issue regarding Google models? Is it about quality issues (lack of guardrails, agent runs dangerous commands)? I have seen first-hand Gemini-cli going out of the project directory and using my home directory as a work area. Or is it about terms of service? Or other concerns?

Quality. They are too dumb.

And the lack of ease of use.

Re: A few words on DS4

#187

Post by etoulas »

> The last week was funny and also tiring, I worked 14 hours per day on average. My normal average is 4/6 since early Redis times, but the first few months of Redis were like that.

Is he taking about 4/6 h of coding? If he meant total working time I’d say this is a very balanced lifestyle!

Re: A few words on DS4

#188

Post by hellifino »

DeathArrow wrote: post #74
gcr wrote: post #46

DwarfStar4 is a small LLM inference runtime that can run DeepSeek 4. The blog post implies that it currently requires 96GB of VRAM. For others who are lacking context :-)

>The blog post implies that it currently requires 96GB of VRAM. From the Github page it seems it only supports Apple and DGX Spark. I have 128 GB of RAM and a 3090 but it probably won't work.

I have an AMD 3995wx and 128GB DDR4 3200 I can load the Q2 and using -t 64 can get around 4 t/s out of the box. Havent tried any other configs yet.

I do not think it can use multi-gpu or gpu/cpu offloading at this time.

Re: A few words on DS4

#189

Post by vlovich123 »

kennywinker wrote: post #114
vlovich123 wrote: post #86

Earlier quoted context omitted.

Why is this presumed to be de facto inevitable: * local compute isn’t scaling as before, so algorithmic improvements are the only ways models get meaningfully faster and smarter * all those same algorithmic improvements would also be true for larger models * hardware manufacturers have an incentive against local LLMs because cloud LLMs are so much more lucrative (+ corps would by desktop variants if they were good en…

> It may be good enough for what you want but there will always be a harder problem that you need to throw more compute and more memory at. Sure, but if the “good enough for what you want” consumes the vast majority of cases - data-center ai becomes just for the very extreme edge cases. Like how I can render a 4k rez video game at 60fps on my home pc, but if pixar wants to render their next movie they use data-center…

Even accepting the premise, it should be obviously true that 10 dumber models running 10x as fast != 1 smarter model. Otherwise engineering would just be a matter of throwing people at a problem when it’s very clear that 1 talented engineer can outperform a team of engineers or accomplish things the team would never have been able to. There’s also the assumption you’re making that a 10x smaller model is 10x dumber when it’s not - it’s a curve and some people seem to struggle with non linear effects

Re: A few words on DS4

#190

Post by zozbot234 »

hellifino wrote: post #188
DeathArrow wrote: post #74

Earlier quoted context omitted.

>The blog post implies that it currently requires 96GB of VRAM. From the Github page it seems it only supports Apple and DGX Spark. I have 128 GB of RAM and a 3090 but it probably won't work.

I have an AMD 3995wx and 128GB DDR4 3200 I can load the Q2 and using -t 64 can get around 4 t/s out of the box. Havent tried any other configs yet. I do not think it can use multi-gpu or gpu/cpu offloading at this time.

That sounds memory bandwidth limited. Does the total t/s decode throughput improve by running multiple sessions in parallel?

(Note, that's total not per-session. Tok/s figures per session will initially tank since you're using the same total mem bandwidth to load incrementally more active params.)

Post reply on HN