Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark: great hardware, early days for the ecosystem

simonwillison.net

Post reply on HN
Print view View original

51–60 of 116 posts

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#51

Post by AndroTux »

dialogbox wrote: post #41
smallnamespace wrote: post #39

An 14-inch M4 Max Macbook Pro with 128GB of RAM has a list price of $4700 or so and twice the memory bandwidth. For inference decode the bandwidth is the main limitation so if running LLMs is your use case you should probably get a Mac instead.

Why Macbook Pro? Isn't Mac Studio is a lot cheaper and the right one to compare with DGX Spark?

I think the idea is that instead of spending an additional $4000 on external hardware, you can just buy one thing (your main work machine) and call it a day. Also, the Mac Studio isn’t that much cheaper at that price point.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#52

Post by Grimburger »

killingtime74 wrote: post #46
two_handfuls wrote: post #12

I wonder how this compares financially with renting something on the cloud.

For me as an employee in Australia, I could buy this and write it off my tax as a work expense myself. To rent, it would be much more cumbersome, involving the company. That's 45% off (our top marginal tax rate).

> That's 45% off (our top marginal tax rate)

Can people please not listen to this terrible advice that gets repeated so oft, especially in Australian IT circles somehow by young naive folks.

You really need to talk to your accountant here.

It's probably under 25% in deduction at double the median wage, little bit over @ triple, and that's *only* if you are using the device entirely for work, as in it sits in an office and nowhere else, if you are using it personally you open yourself up to all sorts of drama if and when the ATO ever decides to audit you for making a $6k AUD claim for a computing device beyond what you normally to use to do your job.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#53

Post by KeplerBoy »

_joel wrote: post #45

How would this fare alongside the new Ryzen chips, ooi? From memory is seems to be getting the same amount of tok/s but would the Ryzen box be more useful for other computing, not just AI?

If you need x86 or windows for anything it's not even a question.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#54

Post by _joel »

KeplerBoy wrote: post #53
_joel wrote: post #45

How would this fare alongside the new Ryzen chips, ooi? From memory is seems to be getting the same amount of tok/s but would the Ryzen box be more useful for other computing, not just AI?

If you need x86 or windows for anything it's not even a question.

Sure, Mac's are also arm based, my question was about general performance, not architecture

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#55

Post by lukeh »

Grimburger wrote: post #52
killingtime74 wrote: post #46

Earlier quoted context omitted.

For me as an employee in Australia, I could buy this and write it off my tax as a work expense myself. To rent, it would be much more cumbersome, involving the company. That's 45% off (our top marginal tax rate).

> That's 45% off (our top marginal tax rate) Can people please not listen to this terrible advice that gets repeated so oft, especially in Australian IT circles somehow by young naive folks. You really need to talk to your accountant here. It's probably under 25% in deduction at double the median wage, little bit over @ triple, and that's *only* if you are using the device entirely for work, as in it sits in an offic…

Also, you can only deduct it in a single financial year if you are eligible for the Instant asset write-off program.

I'm sure I'll get downvoted for this, but this common misunderstanding about tax deductions does remind me of a certain Seinfeld episode :)

Kramer: It's just a write off for them

Jerry: How is it a write off?

Kramer: They just write it off

Jerry: Write it off what?

Kramer: Jerry all these big companies they write off everything

Jerry: You don't even know what a write off is

Kramer: Do you?

Jerry: No. I don't

Kramer: But they do and they are the ones writing it off

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#56

Post by physicsguy »

donw wrote: post #49
physicsguy wrote: post #44

A few years ago I worked on an ARM supercomputer, as well as a POWER9 one. x86 is so assumed for anything other than trivial things that it is painful. What I found was a good solution was using Spack: https://spack.io/ That allows you to download/build the full toolchain of stuff you need for whatever architecture you are on - all dependencies, compilers (GCC, CUDA, MPI, etc.), compiled Python packages, etc. and if…

Who says we don’t have a sense of humor.

It's that it's an offensive term here, not a funny one.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#57

Post by TiredOfLife »

rcarmo wrote: post #32

About what I expected. The Jetson series had the same issues, mostly, at a smaller scale: Deviate from the anointed versions of YOLO, and nothing runs without a lot of hacking. Being beholden to CUDA is both a blessing and a curse, but what I really fear is how long it will take for this to become an unsupported golden brick. Also, the other reviews I’ve seen point out that inference speed is slower than a 5090 (or o…

No need to put unified in scare quotes.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#59

Post by 3abiton »

simonw wrote: post #6

It's notable how much easier it is to get things working now that the embargo has lifted and other projects have shared their integrations. I'm running VLLM on it now and it was as simple as: docker run --gpus all -it --rm \ --ipc=host --ulimit memlock=-1 \ --ulimit stack=67108864 \ nvcr.io/nvidia/vllm:25.09-py3 (That recipe from https://catalog.ngc.nvidia.com/orgs/nvidia/containers/vllm?v... ) And then in the Docker…

As someone who hot on early on the Ryzen AI 395+, are there any added value for the DGX Spark beside having cuda (compared to ROCm/vulkan)? I feel Nvidia fumbled the marketing, either making it sound like an inference miracle, or a dev toolkit (then again not enough to differentiate it from the superior AGX Thor).

I am curious about where you find its main value, and how would it fit within your tooling, and use cases compared to other hardware?

From the inference benchmarks I've seen, a M3 Ultra always come on top.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#60

Post by deviation »

ChocolateGod wrote: post #48
smallnamespace wrote: post #39

An 14-inch M4 Max Macbook Pro with 128GB of RAM has a list price of $4700 or so and twice the memory bandwidth. For inference decode the bandwidth is the main limitation so if running LLMs is your use case you should probably get a Mac instead.

People may prefer running in environments that match their target production environment, so macOS is out of the question.

It's a hoop to jump through, but I'd recommend checking out Apple's container/containerization services which help accomplish just that.

https://github.com/apple/containerization/

Post reply on HN