Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A few words on DS4

antirez.com

Post reply on HN
Print view View original

71–80 of 197 posts

Re: A few words on DS4

#71

Post by dilap »

0xbadcafebee wrote: post #12

I don't see an explanation of why they would make a model-specific inference engine vs just using llamacpp. There are already lots of people working on the llamacpp integration. This is a lot of effort spent on a single model which is likely to become obsolete when a different model comes out that does better. In some discussions, people are now making PRs against both the llamacpp branches and ds4... so it's taking…

way easier to work on a focussed c codebase you own than a mature unwieldy c++ codebase you don't. but it's fine, people will take that work and port to llamacpp and everyone wins.

(the ux of ds4 is fantastic too -- it's dead-easy to get a known-good model, great quant. llamacpp you're much more hacking in the wilderness, w/ many many knobs.)

Re: A few words on DS4

#72

Post by linkregister »

calmingsolitude wrote: post #68
linkregister wrote: post #61

Earlier quoted context omitted.

It's trivial to find reviews and benchmarks of DS4 online. Also, there are benchmarks in the article. Here's one of the top hits: https://forums.developer.nvidia.com/t/fully-custom-cuda-nati... Bizarre comment; sounds like "How do you know Porsches are fast? Did you drive one?"

Parent is simply pointing out the incorrect usage of "empirically", which should typically only be mentioned when you've tested it yourself.

I'm having trouble finding dictionaries or other references that add the qualifier that it needs to be self-tested and not relying on the research of others. Can you point me to one?

Re: A few words on DS4

#73

Post by linkregister »

dakolli wrote: post #66
linkregister wrote: post #61

Earlier quoted context omitted.

It's trivial to find reviews and benchmarks of DS4 online. Also, there are benchmarks in the article. Here's one of the top hits: https://forums.developer.nvidia.com/t/fully-custom-cuda-nati... Bizarre comment; sounds like "How do you know Porsches are fast? Did you drive one?"

[flagged]

It's funny because you make these assertions without any empiricism of your own. They're just speculations.

"You LLM people". Has it occurred to you that individuals have variation within groups?

Re: A few words on DS4

#74

Post by DeathArrow »

gcr wrote: post #46

DwarfStar4 is a small LLM inference runtime that can run DeepSeek 4. The blog post implies that it currently requires 96GB of VRAM. For others who are lacking context :-)

>The blog post implies that it currently requires 96GB of VRAM.

From the Github page it seems it only supports Apple and DGX Spark. I have 128 GB of RAM and a 3090 but it probably won't work.

Re: A few words on DS4

#75

Post by DeathArrow »

stavros wrote: post #37
FuckButtons wrote: post #31

It’s shocking how close this feels to claude, obviously it's much slower, but I don’t know that it’s significantly dumber. Interestingly the imatrix quantization seems to be better than whatever quant the zdr inference backends on open router are using. It was self aware enough yesterday to realize that it’s own server process was itself without me telling it, which is not something I’ve ever observed a local model d…

In my (obviously anecdotal) testing, DeepseekV4 Pro was better than Sonnet at coding. However, it is much slower, but also many times cheaper, especially with the promotion right now.

Do they have a coding plan or you only pay per API call?

Re: A few words on DS4

#76

Post by dakolli »

linkregister wrote: post #72
calmingsolitude wrote: post #68

Earlier quoted context omitted.

Parent is simply pointing out the incorrect usage of "empirically", which should typically only be mentioned when you've tested it yourself.

I'm having trouble finding dictionaries or other references that add the qualifier that it needs to be self-tested and not relying on the research of others. Can you point me to one?

I don't think comments on the internet count as "empirical" evidence, but sure.

Re: A few words on DS4

#77

Post by famouswaffles »

somewhatrandom9 wrote: post #40

With "intelligence" (or whatever you want to call it) and speed both seeming to ramp up quickly with local models I wonder what the growth rate and ceiling(?) might be in this space. Will this kind of iq and performance work with just e.g: 16GB RAM in a couple years? Is there a new kind of Moore's law to be defined here?

Squeezing a model like this complete with 'big model smell' into 16GB...Honestly it's not even possible or feasibly possible today.

It'll require some kind of:

- breakthrough in architecture or

- breakthrough in hardware or

- some breakthrough quantisization technique

The problem is that all the parameters need to be in memory, even the ones that aren't active (say for Mixture Of Expert Models) because switching parametrs in and out of ram is far too slow.

Re: A few words on DS4

#78

Post by skybrian »

zmmmmm wrote: post #55

I'm very curious where we will saturate the curve on "enough" intelligence for coding. At some point, you can let a less smart model hammer at a problem for longer and get to the same result, and as long as you are not involved it comes to the same thing. I feel like DeepSeek V4 Pro is nearly there. Maybe Flash is too. Once we hit that point, I am curious how much of Anthropic's current business model falls apart? So…

I imagine we'll get to "good enough" for hobbyist programmers fairly quickly, but businesses will still be willing to pay more for faster and smarter. Why make your programmers wait?

Re: A few words on DS4

#79

Post by keyle »

karmakaze wrote: post #42

Great to find this narrow focused thing: > We support the following backends: Metal is our primary target. Starting from MacBooks with 96GB of RAM. NVIDIA CUDA with special care for the DGX Spark. AMD ROCm is only supported in the rocm branch. It is kept separate from main since I (antirez) don't have direct hardware access, so the community rebases the branch as needed. > This project would not exist without llama.c…

If only you could still buy Mac's with that much RAM

Re: A few words on DS4

#80

Post by wren6991 »

brcmthrowaway wrote: post #26

This guy is falling deep into Yegge-tier psychosis.

Not even close. "I made this DSP task faster by focusing on exactly one compute graph on one machine instead of a compute graph compiler that runs on every possible machine" is a real engineering approach, and the AI usage is incidental. Things like Gas Town are self-serving turboslop whose only purpose is to generate more slop.
Post reply on HN