Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark as a daily driver

daniel.lawrence.lu

Post reply on HN
Print view View original

61–70 of 85 posts

Re: Nvidia DGX Spark as a daily driver

#61

Post by pjmlp »

bigyabai wrote: post #23
ARandomerDude wrote: post #19

Earlier quoted context omitted.

I know nothing about this topic but your comment piqued my curiosity. What would a $500 AMD GPU do better than DGX?

AMD has Mesa drivers for graphics, which are better-optimized than Nvidia's proprietary Linux Vulkan drivers. It can be fixed in software, but Nvidia's only barely started to catch up. The focus for Nvidia's GPU stack on Linux is getting CUDA working, which means that some traditional raster features get neglected.

NVIDIA also cares about Hollywood and CAD/CAM shops.

I learnt three times that AMD isn't necessarily better in Linux as advertised, now I don't care any longer.

Re: Nvidia DGX Spark as a daily driver

#62

Post by trentor »

mapontosevenths wrote: post #13
trentor wrote: post #11

Earlier quoted context omitted.

Mhm... maybe read my full comment?

Are you saying that it's slow for training? Sorry, your comment is confusingly worded to me. I've not had any issues in that regard, but I'm working with LLM's not training diffusion models. Are you following one of the Nvidia provided recipes or inventing something on your own? The last time I looked into it they benchmarked very well, but we both know that doesn't always mean much.

Yes it is slow for training/fine-tuning decently sized LLMs and unusable for diffusion models. What are you training were you get usable speeds?

Re: Nvidia DGX Spark as a daily driver

#63

Post by biddit »

gerdesj wrote: post #49
biddit wrote: post #33

Please don’t buy a DGX Spark unless all three of these are true: - You value simplicity more than performance or price-to-performance. - You accept that the hardware will depreciate rapidly. - You’re prepared to buy two or four of them. OR: - You want to run frontier models right now as cheaply as possible - You want to run high-parameter models on a 15a breaker/line Otherwise, get a normal, high-bandwidth GPU. A sin…

Actually, its ~119GB usable (I have one). You can shutdown a lot of unneeded services if you only use it remote which will trim a lot more fat. You can enable the RDP service if you don't want to sit in front of it but get a desktop interface. The "shitty" network is 10Gb/s and wifi7! You get a twin QSFP28DDlol+++ (I jest) that each run at 200Gb/s - not for the casual home user but handy at work, although I "only" ha…

That ~119 GB optimization is great. Hadn't seen that yet. That pops a 4x-Spark setup to 476 GB, which gives a lot more headroom for running something like GLM 5.2 at 4-bit, which isn't completely terrible.

I should have prefaced my post - I almost bought four Sparks a couple months ago, but ultimately opted to buy two more RTX 6000 Pro Max-Q's.

It was a painful choice because the two 6000's were more expensive than four Sparks, and ultimately gave me only 384 GB VRAM.

It was even more painful when GLM 5.2 was released, and a 4x Spark setup could run it at a decent quant, but 4x 6000's cannot with any headroom.

But the 6k's absolutely destroy the Sparks on prefill and inference speed. Model intelligence is compressing. The smaller VRAM pool will matter less over time than slower prefill/inference speed.

That is to say, I'm sure we'll end up with <500B parameter models that are Fable-level in the next 8 months or so. Performant quants of those will fit comfortably in 384 GB.

Re: Nvidia DGX Spark as a daily driver

#64

Post by m463 »

MrVitaliy wrote: post #24

I do appreciate how Nvidia tries to say close to vanilla with Linux and Android (nvidia shield). Instead of trying to build a shitty moat like Samsung with all their garbage software. If nvidia ever releases Android smartphone, I'd probably stand in line to get one.

I thought they did that once.

hmmm... about 2010-ish:

NVIDIA Tegra mobile phones include the Motorola Droid X2, LG Optimus 2X, Samsung Galaxy R, HTCOne X, and Xiaomi Mi 3.

https://www.androidauthority.com/nvidia-phones-1234416/

Re: Nvidia DGX Spark as a daily driver

#65

Post by abe_m »

pjmlp wrote: post #61
bigyabai wrote: post #23

Earlier quoted context omitted.

AMD has Mesa drivers for graphics, which are better-optimized than Nvidia's proprietary Linux Vulkan drivers. It can be fixed in software, but Nvidia's only barely started to catch up. The focus for Nvidia's GPU stack on Linux is getting CUDA working, which means that some traditional raster features get neglected.

NVIDIA also cares about Hollywood and CAD/CAM shops. I learnt three times that AMD isn't necessarily better in Linux as advertised, now I don't care any longer.

I'm not aware of any commercial CAD/CAM software running Linux that Nvidia would have a money interest in supporting. Which current CAD/CAM is running on Linux with graphics? The closest I'm aware of it FEA packages allowing remote calculation on Linux machines while the GUI runs on Windows. Is that what you're referring to?

Re: Nvidia DGX Spark as a daily driver

#66

Post by seemaze »

girvo wrote: post #27
willis936 wrote: post #6

Earlier quoted context omitted.

A good instinct. There are a lot of things a $500 AMD GPU can do in linux that a $5000 DGX cannot.

Although one of the ones it cannot do is have/address 128GB of video memory, so it depends on what you want to achieve.

AMD Strix Halo was half the price at launch, same 128GB unified memory and nearly identical memory bandwidth. It is x86, so run whichever OS your heart desires.

You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed..

Re: Nvidia DGX Spark as a daily driver

#67

Post by htrp »

seemaze wrote: post #66
girvo wrote: post #27

Earlier quoted context omitted.

Although one of the ones it cannot do is have/address 128GB of video memory, so it depends on what you want to achieve.

AMD Strix Halo was half the price at launch, same 128GB unified memory and nearly identical memory bandwidth. It is x86, so run whichever OS your heart desires. You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed..

I'm seeing it for 4k and up for similar specs as the spark?

Re: Nvidia DGX Spark as a daily driver

#68

Post by ColdStream »

MrVitaliy wrote: post #24

I do appreciate how Nvidia tries to say close to vanilla with Linux and Android (nvidia shield). Instead of trying to build a shitty moat like Samsung with all their garbage software. If nvidia ever releases Android smartphone, I'd probably stand in line to get one.

It is funny how they make some really decent hardware but the uptake can be fairly limited.

Tegra range is a good example. Yes there is the Nintendo Switch but beyond that it become very limited. Surface RT, Ouya, Nvidia shield and then a few small bits here and there.

Re: Nvidia DGX Spark as a daily driver

#69

Post by bigyabai »

pjmlp wrote: post #61
bigyabai wrote: post #23

Earlier quoted context omitted.

AMD has Mesa drivers for graphics, which are better-optimized than Nvidia's proprietary Linux Vulkan drivers. It can be fixed in software, but Nvidia's only barely started to catch up. The focus for Nvidia's GPU stack on Linux is getting CUDA working, which means that some traditional raster features get neglected.

NVIDIA also cares about Hollywood and CAD/CAM shops. I learnt three times that AMD isn't necessarily better in Linux as advertised, now I don't care any longer.

A lot of Hollywood studios use renderers that jump straight to CUDA as their backend. Cycles is a popular one, as well as Renderman and Octane.

Re: Nvidia DGX Spark as a daily driver

#70

Post by jauntywundrkind »

aftbit wrote: post #38
dllu wrote: post #22

Earlier quoted context omitted.

It's an Orico 9948C3 with four Seagate Barracuda 24TB drives. They were on sale last year [1]. Unfortunately, the enclosure doesn't work super well on Linux. There is a weird bug where the drives don't enumerate when I boot up my computer. This happens on both my x86_64 AMD machine running Linux, and on the DGX Spark. The solution is... simply power cycle the enclosure a couple of times by toggling the power button o…

Yeah... that's been my past experience with USB docks, at least any with more than one slot. I've never had great luck with them. "Can recover from power outage without being touched" is a key requirement for my NAS so I'll give that one a pass and stick with my "SATA drives directly attached to a SAS controller" strategy for now. Thanks for the reply. As for the 24/7 use, yeah so be it. The I in RAID stands for Inex…

It's absolutely the stupidest solution, but SwitchBot Bot is a home-automation button pusher. They are quite cheap on ebay (especially compared to the price of hard drives) but still a face-meltingly stupid problem to have. https://us.switch-bot.com/products/switchbot-bot

I have quite a mix of 2x and 4x drive enclosures, and generally I don't notice anything particularly odd about them? Some enclosures have degraded smart or power management capabilities though. I really wish I'd noticed at the time & sent those ones back! What a waste! My favorite are some dual slot HighPoint toaster-style ones that have independent USB connections per drive, which rocks, but these are so rare & I almost never take advantage of it (replug on drive into a different mini-pc).

Post reply on HN