Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark: great hardware, early days for the ecosystem

simonwillison.net

Post reply on HN
Print view View original

81–90 of 116 posts

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#81

Post by rcarmo »

3abiton wrote: post #68
rcarmo wrote: post #32

About what I expected. The Jetson series had the same issues, mostly, at a smaller scale: Deviate from the anointed versions of YOLO, and nothing runs without a lot of hacking. Being beholden to CUDA is both a blessing and a curse, but what I really fear is how long it will take for this to become an unsupported golden brick. Also, the other reviews I’ve seen point out that inference speed is slower than a 5090 (or o…

> Also, the other reviews I’ve seen point out that inference speed is slower than a 5090 (or on par with a 4090 with some tailwind), so the big difference here (other than core counts) is the large chunk of “unified” memory. It's not comparable to 4090 inference speed. It's significantly slower, because of the lack of MXFP4 models out there. Even compared to Ryzen AI 395 (ROCm / Vulkan), on gpt-oss-120B mxfp4, someho…

Yeah, good point on the FP4. I'm seeing people complain about INT8 as well, which ought to "just work", but everyone who has one (not many) is wary of wandering off the happy path.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#83

Post by storus »

3abiton wrote: post #59
simonw wrote: post #6

It's notable how much easier it is to get things working now that the embargo has lifted and other projects have shared their integrations. I'm running VLLM on it now and it was as simple as: docker run --gpus all -it --rm \ --ipc=host --ulimit memlock=-1 \ --ulimit stack=67108864 \ nvcr.io/nvidia/vllm:25.09-py3 (That recipe from https://catalog.ngc.nvidia.com/orgs/nvidia/containers/vllm?v... ) And then in the Docker…

As someone who hot on early on the Ryzen AI 395+, are there any added value for the DGX Spark beside having cuda (compared to ROCm/vulkan)? I feel Nvidia fumbled the marketing, either making it sound like an inference miracle, or a dev toolkit (then again not enough to differentiate it from the superior AGX Thor). I am curious about where you find its main value, and how would it fit within your tooling, and use case…

M3 Ultra has slow GPU and no HW FP4 support so its initial token decoding is going to be slow, practically unusable for 100k+ context sizes. For token generation that is memory bound M3 Ultra would be much faster, but who wants to wait 15 minutes to read the context? Spark will be much faster for initial token processing, giving you a much better time to first token, but then 3x slower (273 vs 800GB/s) in token generation throughput. You need to decide what is more important for you. Strix Halo is IMO the worst of both worlds at the moment due to having the worst specs in both dimensions and the least mature software stack.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#84

Post by storus »

_ache_ wrote: post #77
rvz wrote: post #74

TLDR: Just buy a RTX 5090. The DGX Spark is completely overpriced for its performance compared to a single RTX 5090.

I get the idea. But isn't 128G of "VRAM" (unified actually) could train a usefull ViT model ? I don't think the 5090 could do that with only 32G of VRAM, couldn't it ?

DGX Spark is not for training, only for inference (FP4).

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#85

Post by pjmlp »

p_l wrote: post #42
cylemons wrote: post #27

Earlier quoted context omitted.

What was the AMD GPGPU called?

Which one? We first had the flurry of third party work (Brook, Lib Sh, etc), then we had AMD "Close to Metal" which was IIRC based on Brook, soon followed with dedicated cards, year later we got CUDA (also derived partially from Brook!) and AMD Stream SDK, later renamed APP SDK. Then we got HIP / HSA stuff which unfortunately has its biggest legacy (outside of availability of HIP as way to target ROCm and CUDA simult…

You forgot the C++AMP collaboration with Microsoft.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#86

Post by pjmlp »

bradfa wrote: post #61
ChocolateGod wrote: post #48

Earlier quoted context omitted.

People may prefer running in environments that match their target production environment, so macOS is out of the question.

The Ubuntu that NVIDIA ship is not stock. They seem to be moving towards using stock Ubuntu but it’s not there yet. Running some other distro on this device is likely to require quite some effort.

It still is more of a Linux distribution than macOS will ever be, UNIX != Linux.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#87

Post by ZiiS »

TiredOfLife wrote: post #57
rcarmo wrote: post #32

About what I expected. The Jetson series had the same issues, mostly, at a smaller scale: Deviate from the anointed versions of YOLO, and nothing runs without a lot of hacking. Being beholden to CUDA is both a blessing and a curse, but what I really fear is how long it will take for this to become an unsupported golden brick. Also, the other reviews I’ve seen point out that inference speed is slower than a 5090 (or o…

No need to put unified in scare quotes.

Given the likelihood you are bound by the 4x lower memory bandwidth this implies; at least for decode, I think they are warranted.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#88

Post by ZiiS »

bradfa wrote: post #61
ChocolateGod wrote: post #48

Earlier quoted context omitted.

People may prefer running in environments that match their target production environment, so macOS is out of the question.

The Ubuntu that NVIDIA ship is not stock. They seem to be moving towards using stock Ubuntu but it’s not there yet. Running some other distro on this device is likely to require quite some effort.

I think the 'environment' here is CUDA; the OS running on the small co-processor you use to buffer some IO is irrelevant.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#89

Post by simonw »

simonw wrote: post #13
matt3210 wrote: post #9

> even in a Docker container I should be allowed to do stupid things when I want. Give me an override!

A couple of people have since tipped me off that this works around that: IS_SANDBOX=0 claude --dangerously-skip-permissions You can run that as root and Claude won't complain.

Correction: it's IS_SANDBOX=1

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#90

Post by pantalaimon »

AndroTux wrote: post #51
dialogbox wrote: post #41

Earlier quoted context omitted.

Why Macbook Pro? Isn't Mac Studio is a lot cheaper and the right one to compare with DGX Spark?

I think the idea is that instead of spending an additional $4000 on external hardware, you can just buy one thing (your main work machine) and call it a day. Also, the Mac Studio isn’t that much cheaper at that price point.

How are you spending $4000 on a screen and a keyboard?
Post reply on HN