Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark: great hardware, early days for the ecosystem

simonwillison.net

Post reply on HN
Print view View original

61–70 of 116 posts

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#61

Post by bradfa »

ChocolateGod wrote: post #48
smallnamespace wrote: post #39

An 14-inch M4 Max Macbook Pro with 128GB of RAM has a list price of $4700 or so and twice the memory bandwidth. For inference decode the bandwidth is the main limitation so if running LLMs is your use case you should probably get a Mac instead.

People may prefer running in environments that match their target production environment, so macOS is out of the question.

The Ubuntu that NVIDIA ship is not stock. They seem to be moving towards using stock Ubuntu but it’s not there yet.

Running some other distro on this device is likely to require quite some effort.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#62

Post by MomsAVoxell »

AndroTux wrote: post #51
dialogbox wrote: post #41

Earlier quoted context omitted.

Why Macbook Pro? Isn't Mac Studio is a lot cheaper and the right one to compare with DGX Spark?

I think the idea is that instead of spending an additional $4000 on external hardware, you can just buy one thing (your main work machine) and call it a day. Also, the Mac Studio isn’t that much cheaper at that price point.

Being able to leave the thing at home and access it anywhere is a feature, not a bug.

The Mac Studio is a more appropriate comparison. There is not yet a DGX laptop, though.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#65

Post by justincormack »

_joel wrote: post #45

How would this fare alongside the new Ryzen chips, ooi? From memory is seems to be getting the same amount of tok/s but would the Ryzen box be more useful for other computing, not just AI?

From reading reviews, dont have either yet: the nvidia actually has unified memory, AMD you have to specify the allocation split. Nvidia maybe has some form of gpu partitioning so you can run multiple smaller models but no one got it working yet. The Ryzen is very different from the pro gpus and the software support wont benefit from work done there, while nvidia is same. You can play games on Ryzen.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#66

Post by TiredOfLife »

rubatuga wrote: post #29
monster_truck wrote: post #24

Whole thing feels like a paper launch being held up by people looking for blog traffic missing the point. I'd be pissed if I paid this much for hardware and the performance was this lacklustre while also being kneecapped for training

When the networking is 25GB/s and the memory bandwidth is 210GB/s you know something is seriously wrong.

It has connectx 200GB/s

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#67

Post by killingtime74 »

lukeh wrote: post #55
Grimburger wrote: post #52

Earlier quoted context omitted.

> That's 45% off (our top marginal tax rate) Can people please not listen to this terrible advice that gets repeated so oft, especially in Australian IT circles somehow by young naive folks. You really need to talk to your accountant here. It's probably under 25% in deduction at double the median wage, little bit over @ triple, and that's *only* if you are using the device entirely for work, as in it sits in an offic…

Also, you can only deduct it in a single financial year if you are eligible for the Instant asset write-off program. I'm sure I'll get downvoted for this, but this common misunderstanding about tax deductions does remind me of a certain Seinfeld episode :) Kramer: It's just a write off for them Jerry: How is it a write off? Kramer: They just write it off Jerry: Write it off what? Kramer: Jerry all these big companies…

Correct. You can deduct over multiple years, so you do get the same amount back.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#68

Post by 3abiton »

rcarmo wrote: post #32

About what I expected. The Jetson series had the same issues, mostly, at a smaller scale: Deviate from the anointed versions of YOLO, and nothing runs without a lot of hacking. Being beholden to CUDA is both a blessing and a curse, but what I really fear is how long it will take for this to become an unsupported golden brick. Also, the other reviews I’ve seen point out that inference speed is slower than a 5090 (or o…

> Also, the other reviews I’ve seen point out that inference speed is slower than a 5090 (or on par with a 4090 with some tailwind), so the big difference here (other than core counts) is the large chunk of “unified” memory.

It's not comparable to 4090 inference speed. It's significantly slower, because of the lack of MXFP4 models out there. Even compared to Ryzen AI 395 (ROCm / Vulkan), on gpt-oss-120B mxfp4, somehow DGX manages to lose on token generation (pp is faster though.

> Still seems like a tricky investment in an age where a Mac will outlive everything else you care to put on a desk and AMD has semi-viable APUs with equivalent memory architectures (even if RoCm is… well… not all there yet).

ROCm (v7) for APUs came a long way actually, mostly thanks to the community effort, it's quite competitive and more mature. It's still not totally user friendly, but it doesn't break between updates (I know the bar is low, but that was the status a year ago). So in comparison, the strix halo offers lots of value for your money if you need a cheap compact inference box.

Havn't tested finetuning / training yet, but in theory it's supported, not to forget that APU is extremely performany for "normal" tasks (threadripper level) compared to the CPU of the DGX Spark.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#69

Post by speedgoose »

two_handfuls wrote: post #12

I wonder how this compares financially with renting something on the cloud.

Depending on the kind of project and data agreements, it’s sometimes much easier to run computations on premise than in the cloud. Even though the cloud is somewhat more secure.

I for example have some healthcare research projects with personally identifiable data, and in these times it’s simpler for the users to trust my company, than my company and some overseas company and it’s associated government.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#70

Post by killingtime74 »

Grimburger wrote: post #52
killingtime74 wrote: post #46

Earlier quoted context omitted.

For me as an employee in Australia, I could buy this and write it off my tax as a work expense myself. To rent, it would be much more cumbersome, involving the company. That's 45% off (our top marginal tax rate).

> That's 45% off (our top marginal tax rate) Can people please not listen to this terrible advice that gets repeated so oft, especially in Australian IT circles somehow by young naive folks. You really need to talk to your accountant here. It's probably under 25% in deduction at double the median wage, little bit over @ triple, and that's *only* if you are using the device entirely for work, as in it sits in an offic…

My work is entirely from home. I happen to also be an ex lawyer, quite familiar with deduction rules and not altogether young. Can you explain why you think it's not 45% off? Ive deducted thousands in AI related work expenses over the years.

Even if what you are saying is correct, the discount is just lower. This is compared to no discount on compute/GPU rental unless your company purchases it.

Post reply on HN