Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark: great hardware, early days for the ecosystem

simonwillison.net

Post reply on HN
Print view View original

101–110 of 116 posts

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#101

Post by p_l »

pjmlp wrote: post #85
p_l wrote: post #42

Earlier quoted context omitted.

Which one? We first had the flurry of third party work (Brook, Lib Sh, etc), then we had AMD "Close to Metal" which was IIRC based on Brook, soon followed with dedicated cards, year later we got CUDA (also derived partially from Brook!) and AMD Stream SDK, later renamed APP SDK. Then we got HIP / HSA stuff which unfortunately has its biggest legacy (outside of availability of HIP as way to target ROCm and CUDA simult…

You forgot the C++AMP collaboration with Microsoft.

Is it the OpenMP related one or another thing?

I kinda lost track, this whole thread reminded me how hopeful I was to play with GPGPU with my then new X1600

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#102

Post by blurbleblurble »

justincormack wrote: post #91
blurbleblurble wrote: post #79

Earlier quoted context omitted.

But on the ryzen the vram allocation can be entirely dynamically allocated. I saw a review showing excellent full GPU usage during inference with the bios vram allocation set to the minimum level, using a very large model. So it's not so simple as you describe (I used to think this was the case too). Beyond that, seems like the 395 in practice smashes the dgx spark in inference speeds for most models. I haven't seen…

Yes you can set it but in the BIOS, not dynamically as you need it. I dont think there are any models supporting nvfp4 yet but we shall probably start seeing them.

That's what I'm saying, in the review video I saw they allocated as little memory as possible to the GPU in the bios, then used some kind of kernel level dynamic control.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#103

Post by sailingparrot »

rvz wrote: post #74

TLDR: Just buy a RTX 5090. The DGX Spark is completely overpriced for its performance compared to a single RTX 5090.

Its a DGX dev box, for those (not consumers) that will ultimately need to run their code on large DGX clusters where a failure or a ~3% slowdown of training ends up costing tens of thousands of dollars.

That's the use case, not running LLM efficiently, and you can't do that with a RTX5090.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#104

Post by pjmlp »

p_l wrote: post #101
pjmlp wrote: post #85

Earlier quoted context omitted.

You forgot the C++AMP collaboration with Microsoft.

Is it the OpenMP related one or another thing? I kinda lost track , this whole thread reminded me how hopeful I was to play with GPGPU with my then new X1600

Other thing,

https://learn.microsoft.com/en-us/cpp/parallel/amp/cpp-amp-c...

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#105

Post by wtallis »

TiredOfLife wrote: post #66
rubatuga wrote: post #29

Earlier quoted context omitted.

When the networking is 25GB/s and the memory bandwidth is 210GB/s you know something is seriously wrong.

It has connectx 200GB/s

No, the NIC runs at 200Gb/s, not 200GB/s.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#107

Post by teleforce »

physicsguy wrote: post #44

A few years ago I worked on an ARM supercomputer, as well as a POWER9 one. x86 is so assumed for anything other than trivial things that it is painful. What I found was a good solution was using Spack: https://spack.io/ That allows you to download/build the full toolchain of stuff you need for whatever architecture you are on - all dependencies, compilers (GCC, CUDA, MPI, etc.), compiled Python packages, etc. and if…

It's good that you've mentioned Spack but not for HPC work, and that's very interesting.

This a high level overview by one of the Spack authors from the HN post back in 2023 (top comment from 100 comments), including the Spack original paper link [1]:

At a very high level, Spack has:

* Nix's installation model and configuration hashing

* Homebrew-like packages, but in a more expressive Python DSL, and with more versions/options

* A very powerful dependency resolver that doesn't just pick from a set of available configurations -- it configures your build according to possible configurations.

You could think of it like Nix with dependency resolution, but with a nice Python DSL. There is more on the "concretizer" (resolver) and how we've used ASP for it here:

* "Using Answer Set Programming for HPC Dependency Solving", https://arxiv.org/abs/2210.08404

[1] Spack – scientific software package manager for supercomputers, Linux, and macOS (100 comments):

https://news.ycombinator.com/item?id=35237269

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#108

Post by monster_truck »

_ache_ wrote: post #78
monster_truck wrote: post #24

Whole thing feels like a paper launch being held up by people looking for blog traffic missing the point. I'd be pissed if I paid this much for hardware and the performance was this lacklustre while also being kneecapped for training

What do you mean by "kneecapped for training"? Isn't it 128GB of VRAM enougth for small model training, that a current GC can't do? Obviously, even with connectx, it's only 240Gi of VRAM, so no big models can be trained.

Spend some time looking at the real benchmarks before writing nonsense

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#109

Post by physicsguy »

teleforce wrote: post #107
physicsguy wrote: post #44

A few years ago I worked on an ARM supercomputer, as well as a POWER9 one. x86 is so assumed for anything other than trivial things that it is painful. What I found was a good solution was using Spack: https://spack.io/ That allows you to download/build the full toolchain of stuff you need for whatever architecture you are on - all dependencies, compilers (GCC, CUDA, MPI, etc.), compiled Python packages, etc. and if…

It's good that you've mentioned Spack but not for HPC work, and that's very interesting. This a high level overview by one of the Spack authors from the HN post back in 2023 (top comment from 100 comments), including the Spack original paper link [1]: At a very high level, Spack has: * Nix's installation model and configuration hashing * Homebrew-like packages, but in a more expressive Python DSL, and with more versi…

Well to be fair, I'd consider this to be semi-HPC work - obviously it's not multi-node but because of the hardware it's not the same as using it on an ordinary desktop machine either and has many of the challenges of HPC too in getting stuff compiled for it, particularly with it being ARM based. What you learn when you work on this stuff is that you need very specific combinations of packages that your distro just isn't going to be able to do, and Homebrew doesn't give you enough flexibility in that.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#110

Post by BoredPositron »

pantalaimon wrote: post #96
AndroTux wrote: post #95

Earlier quoted context omitted.

You're not going to use the DGX as your main machine, so you'll need another computer. Sure, not a $4000 one, but you'll want at least some performance, so it'll be another $1000-$2000.

> You're not going to use the DGX as your main machine Why not?

Because Nvidia is incredibly slow with kernel updates and you are lucky if you get them at all after just two years. I am curious if they will update these machines for longer than their older dgx like hardware.
Post reply on HN