Live data from Hacker News

Nvidia DGX Spark as a daily driver

daniel.lawrence.lu

81–85 of 85 posts

Re: Nvidia DGX Spark as a daily driver

#81
post #66
post #27

Earlier quoted context omitted.

Although one of the ones it cannot do is have/address 128GB of video memory, so it depends on what you want to achieve.

AMD Strix Halo was half the price at launch, same 128GB unified memory and nearly identical memory bandwidth. It is x86, so run whichever OS your heart desires. You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed..

>You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed..

I mean that's not as true as you'd think: in fact its the whole benefit IMO of the spark-alikes, and if you're not running TP=2 or above you're sort of wasting it's potential

Re: Nvidia DGX Spark as a daily driver

#82
post #80
post #12

Earlier quoted context omitted.

Many other distros almost work out of the box (as in they boot and run without any modifications). The custom kernel patches you mentioned address mainly non-critical bugs such as a bug where the Realtek r8127 stops working after a reboot (but it works if you turn it off and on again) [1] [2]. I'd consider it in a way better state than trying to run other Linux distros on certain device tree-based devices like, say,…

Why are those patches not in mainline? They appear to be generic enough not to give out any Nvidia secret sauce.

iirc the bug that the R8127 patch was addressing was fixed in the R8169 driver upstream in kernel 6.18 and later. Not super sure what other patches there are and what the status of those are...

Re: Nvidia DGX Spark as a daily driver

#83
post #54

Earlier quoted context omitted.

Is it to me or the iGPU of Ryzen 9950X is not supported by ROCm?

The igpu is a tiny RDNA2 GPU. Pretty certain the 9950x cpu will outperform it in compute tasks.

What if the software was written for ROCm?

Re: Nvidia DGX Spark as a daily driver

#84
post #63

Earlier quoted context omitted.

That ~119 GB optimization is great. Hadn't seen that yet. That pops a 4x-Spark setup to 476 GB, which gives a lot more headroom for running something like GLM 5.2 at 4-bit, which isn't completely terrible. I should have prefaced my post - I almost bought four Sparks a couple months ago, but ultimately opted to buy two more RTX 6000 Pro Max-Q's. It was a painful choice because the two 6000's were more expensive than f…

In your previous posting you said that for more than 2 Sparks you also need to buy a very expensive switch. That is not really true. Once you have 2 fast Ethernet ports, like DGX Spark has, you can interconnect any number of systems without using a switch. In the simplest case, you just daisy chain the systems and you configure in Linux the Ethernet interfaces as bridges. For better performance, you can close the cha…

On each Spark you can bridge the two interfaces but you cannot bridge those bridges at layer two and it sounds from your post that you are confusing layer two and three (ARPA).

So if you have three Sparks, you have three two port switches. Take three two port switches and connect them in a ring and you have a collision storm. You have to use STP or similar to sort that out.

The ring is now collapsed to A-B-C (with no A-C) and to get from A to C you have to go via B which halves thoughput for those two paths: A-C and C-A.

Another "option" is to define two VLANs and use MSTP and layer three and hope the protocol that shuffles data can route packets effectively. So you get two sets of paths at layer two: A1-B1-C1-A1 and A2-B2-C2-A2 and you cut the two graphs at different points, so you get two disjoint graphs, for example: A1-B1-C1 and B2-C2-A2. However they will still overlap and it all falls apart, somewhat.

You can only link two Sparks together without a switch to get the full possible throughput. That is why Nvidia only offer a two pack option. Three or more requires a switch to avoid a path being used twice. Ethernet is not Token Ring! Mind you Tring was 16Mb/s - luxury when ethernet was 10Mb/s 8)

I've also had this discussion with folk when it comes to "hyper converged" virty systems: So we have three nodes with two NICs each for Ceph and we cable them up in a ring and it goes weird ... . At least there I'm only generally specifying 10G.

Now, do you really need the full 200Mb/s for clustering SParks? No idea yet, I've only got one and I've only got 40Gb/s QSFP+ to play with on my switches.

However I'm seeing value in the one box already at the moment and if that scales then I'll be buying more Sparks and a really funky switch when I need three Sparks and perhaps another switch down the line with some really fancy VLT or whatever wankery is the stacking du jour thing!

Re: Nvidia DGX Spark as a daily driver

#85
I purchased my Spark back in March before the RAM price increase. My use case is local self-sovereign AI in Healthcare (https://hcvc.net), so I haven't done any gaming. Its been a great driver for Hermes Agent, serving local MoE and dense models up to 122B. It now handles about 90% of my inference load (more complex devops/coding tasks are still routed to cloud). LoRa works well. All my models are open source. There is no NVIDIA dependency in my build. Its also a backend server for my PoC app (https://proforta.com). Zero issues so far. Extremely reliable and gives me straight path to production for VPS deployment.
Post reply on HN