Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark

nvidia.com

Post reply on HN
Print view View original

31–40 of 222 posts

Re: Nvidia DGX Spark

#31

Post by aurareturn »

BoorishBears wrote: post #27
dirtyhand wrote: post #14

I was considering getting an RTX 5090 to run inference on some LLM models, but now I’m wondering if it’s worth paying an extra $2K for this option instead

No. These are practically useless for AI. Their prompt processing speeds are absolutely abysmal: if you're trying to tinker from time to time, a GPU like a 5090 or renting GPUs is a much better option. If you're just trying to prep for impending mainstream AI applications, few will be targeting this form factor: it's both too strong compared to mainstream hardware, and way too weak compared to dedicated AI-focused ac…

  Their prompt processing speeds are absolutely abysmal
They are not. This is Blackwell with Tensor cores. Bandwidth is the problem here.

Re: Nvidia DGX Spark

#32

Post by rz2k »

aurareturn wrote: post #28
cherioo wrote: post #24

The mainstream options seem to be Ryzen AI Max 395+, ~120 tops (fp8?), 128GB RAM, $1999 Nvidia DGX Spark, ~1000 tops fp4, 128GB RAM, $3999 Mac Studio max spec, ~120 tflops (fp16?), 512GB RAM, 3x bandwidth, $9499 DGX Spark appears to potentially offer the most token per second, but less useful/value as everyday pc.

Mac Studio max spec, ~120 tflops (fp16?), 384GB RAM, 3x bandwidth, $9499 512GB. DGX has 256GB/s bandwidth so it wouldn't offer the most tokens/s.

Perhaps they are referring to default GPU allocation that is 75% of the unified memory, but it is trivial to increase it.

Re: Nvidia DGX Spark

#34

Post by jauntywundrkind »

cherioo wrote: post #24

The mainstream options seem to be Ryzen AI Max 395+, ~120 tops (fp8?), 128GB RAM, $1999 Nvidia DGX Spark, ~1000 tops fp4, 128GB RAM, $3999 Mac Studio max spec, ~120 tflops (fp16?), 512GB RAM, 3x bandwidth, $9499 DGX Spark appears to potentially offer the most token per second, but less useful/value as everyday pc.

NVidia Spark is $4000. Or, will be, supposedly whenever it comes out.

Also notably, Strix Halo and DGX Spark are both ~275GBps memory bandwidth. Not always but in many machine learning cases it feels like that's going to be the limiting factor.

Re: Nvidia DGX Spark

#35

Post by echelon »

aurareturn wrote: post #28
cherioo wrote: post #24

The mainstream options seem to be Ryzen AI Max 395+, ~120 tops (fp8?), 128GB RAM, $1999 Nvidia DGX Spark, ~1000 tops fp4, 128GB RAM, $3999 Mac Studio max spec, ~120 tflops (fp16?), 512GB RAM, 3x bandwidth, $9499 DGX Spark appears to potentially offer the most token per second, but less useful/value as everyday pc.

Mac Studio max spec, ~120 tflops (fp16?), 384GB RAM, 3x bandwidth, $9499 512GB. DGX has 256GB/s bandwidth so it wouldn't offer the most tokens/s.

tokens/s/$ then.

Re: Nvidia DGX Spark

#36

Post by jauntywundrkind »

rz2k wrote: post #32
aurareturn wrote: post #28

Earlier quoted context omitted.

Mac Studio max spec, ~120 tflops (fp16?), 384GB RAM, 3x bandwidth, $9499 512GB. DGX has 256GB/s bandwidth so it wouldn't offer the most tokens/s.

Perhaps they are referring to default GPU allocation that is 75% of the unified memory, but it is trivial to increase it.

The GPU memory allocation refers to how capacity is alloted, not bandwidth. Sounds like the same 256-bit/quad-channel 8000MHz lpddr5 you can get today with Strix Halo.

Re: Nvidia DGX Spark

#37

Post by BoorishBears »

aurareturn wrote: post #31
BoorishBears wrote: post #27

Earlier quoted context omitted.

No. These are practically useless for AI. Their prompt processing speeds are absolutely abysmal: if you're trying to tinker from time to time, a GPU like a 5090 or renting GPUs is a much better option. If you're just trying to prep for impending mainstream AI applications, few will be targeting this form factor: it's both too strong compared to mainstream hardware, and way too weak compared to dedicated AI-focused ac…

Their prompt processing speeds are absolutely abysmal They are not. This is Blackwell with Tensor cores. Bandwidth is the problem here.

They're abysmal compared to anything dedicated at any reasonable batch size because of both bandwidth and compute, not sure why you're wording this like it disagrees with what I said.

I've run inference workloads on a GH200 which is an entire H100 attached to an ARM processor and the moment offloading is involved speeds tank to Mac Mini-like speeds, which is similarly mostly a toy when it comes to AI.

Re: Nvidia DGX Spark

#39

Post by jrgifford »

x2tyfi wrote: post #30
ls612 wrote: post #13

Is this the $3500 one?

That was their Digits box.

Digits is no more, it’s DGX. Source: signup link in the press release (https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...) now goes to DGX Spark preorder (https://www.nvidia.com/en-us/products/workstations/dgx-spark...)
Post reply on HN