Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A few words on DS4

antirez.com

Post reply on HN
Print view View original

191–197 of 197 posts

Re: A few words on DS4

#191

Post by sowbug »

rjh29 wrote: post #113
minimaxir wrote: post #15

A relevant recent tweet from antirez: https://x.com/antirez/status/2054854124848415211 > Gentle reminder on how, in the recent DS4 fiesta, not just me but every other contributor found GPT 5.5 able to help immensely and Opus completely useless. I've noticed the same for lower level squeezing-as-much-performance-as-possible code work.

There's so much subjectivity with models. As soon as a new model comes out people act like the last model they used for 6 months was completely useless.

That's a corollary of the Jevons paradox, isn't it? Six months ago I was watching every line of my LLM's code generation like a hawk. Now I give it a few assignments before bed and wake up to see what's live from the CI/CD pipeline. Expectations and usage have ratcheted up.

Re: A few words on DS4

#192

Post by jaimeyap »

zimmerfrei wrote: post #97
karmakaze wrote: post #42

Great to find this narrow focused thing: > We support the following backends: Metal is our primary target. Starting from MacBooks with 96GB of RAM. NVIDIA CUDA with special care for the DGX Spark. AMD ROCm is only supported in the rocm branch. It is kept separate from main since I (antirez) don't have direct hardware access, so the community rebases the branch as needed. > This project would not exist without llama.c…

> AMD ROCm is only supported in the rocm branch. Has anybody tried it? There is a lot of emphasis on MacBook Pro in this thread, but I would like to use it with an AMD Halo Strix with 128GB of unified RAM.

I just got the rocm branch compiled and running. Starting with one of the common strix halo rocm toolboxes, just needed to install a few more dependencies to get the repo to build. So far just tried the q2-imatrix model and I'm seeing ~7.32tok/s with a locally bound claude code session. It's pretty unusably slow for agentic coding like this - with it being tens of minutes per round of thinking. But it does seem to be working. Suspiciously amdgpu_top is only showing ~16GB of memory being used. Not sure if this is somehow misreading that.

Re: A few words on DS4

#193

Post by neomantra »

neomantra wrote: post #149
antirez wrote: post #126

Earlier quoted context omitted.

There is no need for another agent, functionally. But if you follow the idea of DS4 itself: the API agents use forces to do odd things, like translating the DSML stanzas to JSON, with all the canonicalization / KV cache checkpointing problems resulting from that. Is it really the case? What about also providing a sane alternative? Also I'm not sure why people don't try to write more stuff in that area in C/Go/Rust to…

For Golang, I highly recommend yzma to explore this surface. I’ve used it for embedding and summarization (with small models) and just mucking around with integrated LLM BubbleTea TUI idea (with bigger models). https://github.com/hybridgroup/yzma And thank you antirez for using your rep and quality output to push this line of evangelism; it is even more important than the software itself.

Literal #showerthought: "why don't you just make the same thing for ds4, it will be awesome"

Toweled off and got to work:

https://github.com/NimbleMarkets/ds4-go

The concept is marrying the flexibility of Golang with a specific local high-performance inferencing engine. The clean C interface made it easy. Initial release wraps the API using purego and requires pointing to a DS4 installation.

I'm now adding some pre-built installer ergonomics and directory opinions and demos.

Re: A few words on DS4

#194

Post by kennywinker »

vlovich123 wrote: post #189
kennywinker wrote: post #114

Earlier quoted context omitted.

> It may be good enough for what you want but there will always be a harder problem that you need to throw more compute and more memory at. Sure, but if the “good enough for what you want” consumes the vast majority of cases - data-center ai becomes just for the very extreme edge cases. Like how I can render a 4k rez video game at 60fps on my home pc, but if pixar wants to render their next movie they use data-center…

Even accepting the premise, it should be obviously true that 10 dumber models running 10x as fast != 1 smarter model. Otherwise engineering would just be a matter of throwing people at a problem when it’s very clear that 1 talented engineer can outperform a team of engineers or accomplish things the team would never have been able to. There’s also the assumption you’re making that a 10x smaller model is 10x dumber wh…

> it should be obviously true that 10 dumber models running 10x as fast != 1 smarter model

If a smaller model tries ten things and comes to the same conclusion as the big model gets first try, then yeah 10x small = 1x big. Is that where we are at now? Idk probably not - but it’s not hard to imagine something like that emerging soon. There is already evidence that smaller models get some things _better_ than bigger models (e.g. https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... )

> There’s also the assumption you’re making that a 10x smaller model is 10x dumber when it’s not

That is not an assumption i am making. I said “a smaller model” not “a 10x smaller model”. Model speed and model “intelligence” are both non-linear.

Re: A few words on DS4

#195

Post by modmans2nd »

rurban wrote: post #165
gekoxyz wrote: post #163

Earlier quoted context omitted.

which harness are you using? pi? opencode?

That's not a harness. That's an agent cli. A harness is something completely different. Wish people could use proper terminology. A test harness is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It automates the execution of test suites, providing the necessary stubs, drivers, and runtime environments so develope…

It is really telling when people say that. It’s clear they think the job of harness development is done by selecting the agent environment.

Re: A few words on DS4

#196

Post by modmans2nd »

teknologist wrote: post #183
rurban wrote: post #180

Earlier quoted context omitted.

No, an agent cli is no harness. You have to provide a harness for an agent by yourself, otherwise it will run free. Which is called vibe coding. Free as you wish, without any harness.

An agent cli provides a sandbox, with permission systems and auto command classifiers. That’s part of the harness.

lol pi doesn’t

Re: A few words on DS4

#197

Post by modmans2nd »

embedding-shape wrote: post #148
DeathArrow wrote: post #94

Earlier quoted context omitted.

Thanks, I can run Qwen 3.6 27B with vllm, but I was curious about antirez tool.

Have you had it getting stuck in endless loops maybe ~10-20% of the invocations? Seems it happens for both the responses and chatcompletion APIs, and no matter what inference parameters I try it happens at least for 1/10 of the requests, I've tried every compatible vLLM version + currently using it from git (#main) yet the issue persists. Seems to happen with various quantizations too, even the NVFP4 versions and any…

There’s a fixed version out there with corrected templates.
Post reply on HN