Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark: great hardware, early days for the ecosystem

simonwillison.net

Post reply on HN
Print view View original

91–100 of 116 posts

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#91

Post by justincormack »

blurbleblurble wrote: post #79
justincormack wrote: post #65

Earlier quoted context omitted.

From reading reviews, dont have either yet: the nvidia actually has unified memory, AMD you have to specify the allocation split. Nvidia maybe has some form of gpu partitioning so you can run multiple smaller models but no one got it working yet. The Ryzen is very different from the pro gpus and the software support wont benefit from work done there, while nvidia is same. You can play games on Ryzen.

But on the ryzen the vram allocation can be entirely dynamically allocated. I saw a review showing excellent full GPU usage during inference with the bios vram allocation set to the minimum level, using a very large model. So it's not so simple as you describe (I used to think this was the case too). Beyond that, seems like the 395 in practice smashes the dgx spark in inference speeds for most models. I haven't seen…

Yes you can set it but in the BIOS, not dynamically as you need it.

I dont think there are any models supporting nvfp4 yet but we shall probably start seeing them.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#92

Post by smallnamespace »

dialogbox wrote: post #41
smallnamespace wrote: post #39

An 14-inch M4 Max Macbook Pro with 128GB of RAM has a list price of $4700 or so and twice the memory bandwidth. For inference decode the bandwidth is the main limitation so if running LLMs is your use case you should probably get a Mac instead.

Why Macbook Pro? Isn't Mac Studio is a lot cheaper and the right one to compare with DGX Spark?

I didn't think of it ;)

Now that you bring it up, the M3 ultra Mac Studio goes up to 512GB for about a $10k config with around 850 GB/s bandwidth, for those who "need" a near frontier large model. I think 4x the RAM is not quite worth more than doubling the price, especially if MoE support gets better, but it's interesting that you can get a Deepseek R1 quant running on prosumer hardware.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#93

Post by AndroTux »

dialogbox wrote: post #76
AndroTux wrote: post #51

Earlier quoted context omitted.

I think the idea is that instead of spending an additional $4000 on external hardware, you can just buy one thing (your main work machine) and call it a day. Also, the Mac Studio isn’t that much cheaper at that price point.

> Also, the Mac Studio isn’t that much cheaper at that price point. In the list price, it's 1000 USD cheaper. 3,699 vs 4,699 I know a lot can be relative but that's a lot for me for sure.

Fair. I looked it up just yesterday so I though I knew the prices from memory, but apparently I mixed something up.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#94

Post by AndroTux »

MomsAVoxell wrote: post #62
AndroTux wrote: post #51

Earlier quoted context omitted.

I think the idea is that instead of spending an additional $4000 on external hardware, you can just buy one thing (your main work machine) and call it a day. Also, the Mac Studio isn’t that much cheaper at that price point.

Being able to leave the thing at home and access it anywhere is a feature, not a bug. The Mac Studio is a more appropriate comparison. There is not yet a DGX laptop, though.

> Being able to leave the thing at home and access it anywhere is a feature, not a bug.

I can do that with a laptop too. And with a dedicated GPU. Or a blade in a data center. I though the feature of the DGX was that you can throw it in a backpack.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#95

Post by AndroTux »

pantalaimon wrote: post #90
AndroTux wrote: post #51

Earlier quoted context omitted.

I think the idea is that instead of spending an additional $4000 on external hardware, you can just buy one thing (your main work machine) and call it a day. Also, the Mac Studio isn’t that much cheaper at that price point.

How are you spending $4000 on a screen and a keyboard?

You're not going to use the DGX as your main machine, so you'll need another computer. Sure, not a $4000 one, but you'll want at least some performance, so it'll be another $1000-$2000.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#96

Post by pantalaimon »

AndroTux wrote: post #95
pantalaimon wrote: post #90

Earlier quoted context omitted.

How are you spending $4000 on a screen and a keyboard?

You're not going to use the DGX as your main machine, so you'll need another computer. Sure, not a $4000 one, but you'll want at least some performance, so it'll be another $1000-$2000.

> You're not going to use the DGX as your main machine

Why not?

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#98

Post by wtallis »

simonw wrote: post #21
wmf wrote: post #17

Earlier quoted context omitted.

That can't be right because RAM has always been reported in binary units. Only storage and networking use lame decimal units.

Looks like Claude reported it based on this: ● Bash(free -h) ⎿ total used free shared buff/cache available Mem: 119Gi 7.5Gi 100Gi 17Mi 12Gi 112Gi Swap: 0B 0B 0B That 119Gi is indeed gibibytes, and 119Gi in GB is 128GB.

You're barking up the wrong tree. Nobody's manufacturing power-of-ten sized DRAM chips for NVIDIA; the amount of memory physically present has to be 128GiB. If `free` isn't reporting that much usable capacity, you need to dig into the kernel logs to see how much is being reserved by the firmware and kernel and drivers. (If there was more memory missing, it could plausibly be due to in-band ECC, but that doesn't seem to be an option for DGX Spark.)

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#99

Post by ChocolateGod »

deviation wrote: post #60
ChocolateGod wrote: post #48

Earlier quoted context omitted.

People may prefer running in environments that match their target production environment, so macOS is out of the question.

It's a hoop to jump through, but I'd recommend checking out Apple's container/containerization services which help accomplish just that. https://github.com/apple/containerization/

You're likely still targeting Nvidia's stack for LLMs and Linux's containers on MacOS won't help you there.

Re: Nvidia DGX Spark: great hardware, early days for the ecosystem

#100

Post by MomsAVoxell »

AndroTux wrote: post #94
MomsAVoxell wrote: post #62

Earlier quoted context omitted.

Being able to leave the thing at home and access it anywhere is a feature, not a bug. The Mac Studio is a more appropriate comparison. There is not yet a DGX laptop, though.

> Being able to leave the thing at home and access it anywhere is a feature, not a bug. I can do that with a laptop too. And with a dedicated GPU. Or a blade in a data center. I though the feature of the DGX was that you can throw it in a backpack.

The DGX is clearly a desktop system. Sure, it's luggable. But the point is, it's not a laptop.
Post reply on HN