Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark as a daily driver

daniel.lawrence.lu

Post reply on HN
Print view View original

11–20 of 85 posts

Re: Nvidia DGX Spark as a daily driver

#11

Post by trentor »

mapontosevenths wrote: post #10
trentor wrote: post #9

I'm genuinely disappointed with my Spark. I don't know how anyone can claim it performs decently with LLMs or diffusion models. Back when I worked in VFX in the early 2000s, we had a saying: "Render time is coffee time" and if you try to run this thing with a usable context size, you'll be drinking a lot of coffee. Most of the optimizations it relies on for inference simply aren't available for training, so it crawls…

If you bought it for inference you made a mistake. They aren't good at that. Use it to train models and experiment with ML. It's much better at that. If you just want local inference buy a Mac. If you bought early on, like I did, the Spark is probably worth double what you payed now. I think I paid $3,000 retail for mine and the last time I looked they were fetching close to $6k on ebay. I'm not sure if that's still…

Mhm... maybe read my full comment?

Re: Nvidia DGX Spark as a daily driver

#12

Post by dllu »

cogman10 wrote: post #4

I strongly considered it, but the one thing that scares me away from wanting to do the spark is you basically have to use nvidia's linux (from what I've read) and it doesn't appear the nvidia is interested in upstreaming their kernel changes. I'm avoiding where possible buying electronics where support is controlled by the manufacturer and not me.

Many other distros almost work out of the box (as in they boot and run without any modifications). The custom kernel patches you mentioned address mainly non-critical bugs such as a bug where the Realtek r8127 stops working after a reboot (but it works if you turn it off and on again) [1] [2]. I'd consider it in a way better state than trying to run other Linux distros on certain device tree-based devices like, say, Qualcomm Snapdragon machines. The regular NVIDIA drivers with the open source kernel modules work just fine. Talos Linux supports DGX Spark since version 1.12 [3]. I also know of people using Fedora and nixOS successfully.

[1] https://github.com/NVIDIA/NV-Kernels/compare/ea55925ab430f1e...

[2] https://forums.developer.nvidia.com/t/realtek-r8127-ethernet...

[3] https://github.com/siderolabs/talos/issues/12170

Re: Nvidia DGX Spark as a daily driver

#13

Post by mapontosevenths »

trentor wrote: post #11
mapontosevenths wrote: post #10

Earlier quoted context omitted.

If you bought it for inference you made a mistake. They aren't good at that. Use it to train models and experiment with ML. It's much better at that. If you just want local inference buy a Mac. If you bought early on, like I did, the Spark is probably worth double what you payed now. I think I paid $3,000 retail for mine and the last time I looked they were fetching close to $6k on ebay. I'm not sure if that's still…

Mhm... maybe read my full comment?

Are you saying that it's slow for training? Sorry, your comment is confusingly worded to me.

I've not had any issues in that regard, but I'm working with LLM's not training diffusion models. Are you following one of the Nvidia provided recipes or inventing something on your own? The last time I looked into it they benchmarked very well, but we both know that doesn't always mean much.

Re: Nvidia DGX Spark as a daily driver

#16

Post by dllu »

trentor wrote: post #9

I'm genuinely disappointed with my Spark. I don't know how anyone can claim it performs decently with LLMs or diffusion models. Back when I worked in VFX in the early 2000s, we had a saying: "Render time is coffee time" and if you try to run this thing with a usable context size, you'll be drinking a lot of coffee. Most of the optimizations it relies on for inference simply aren't available for training, so it crawls…

The DGX Spark hits a sweet spot for me where it can simultaneously work for general computing and run local LLM inference fast enough for some hobbyist dabbling. The RTX 6000 Pro Blackwell is more than twice the price (it has increased quite a bit recently from $8000 to $11600), not to mention the "rest of the PC" needed to get it working, so it's not really a fair comparison. Compared to other 128 GB unified memory devices like the Mac Studio and the Strix Halo, the DGX Spark fairly priced in my opinion.

Re: Nvidia DGX Spark as a daily driver

#17

Post by aftbit »

Neat blog! I was intrigued by this bullet point mentioned in passing:

>my four hard drive USB 3.2 ZFS raidz2 array with four 24 TB drives

Can you speak more about this? Which USB array did you choose? How well does it work? I've been slowly planning a transition away from my power-hungry surplus enterprise gear in the 19" rack towards a smaller, quieter, lower power setup ... but storage is the real kicker right now. I have a 12x18TB array in raidz2 built into a 1U NAS case, and I just can't quite figure out a better way to package something like that. I would need three USB arrays if I want to reuse the existing drives, which I think I do given how expensive storage is today.

Re: Nvidia DGX Spark as a daily driver

#19

Post by ARandomerDude »

willis936 wrote: post #6
cogman10 wrote: post #4

I strongly considered it, but the one thing that scares me away from wanting to do the spark is you basically have to use nvidia's linux (from what I've read) and it doesn't appear the nvidia is interested in upstreaming their kernel changes. I'm avoiding where possible buying electronics where support is controlled by the manufacturer and not me.

A good instinct. There are a lot of things a $500 AMD GPU can do in linux that a $5000 DGX cannot.

I know nothing about this topic but your comment piqued my curiosity. What would a $500 AMD GPU do better than DGX?

Re: Nvidia DGX Spark as a daily driver

#20

Post by haunter »

This is something I'd do if I've had the disposable income lol

>Non-Steam games have a lower chance of working

Wonder if it's true for GOG games because they are usually installed in a neatly packaged folder without any bloat.

Post reply on HN