Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia DGX Spark

nvidia.com

Post reply on HN
Print view View original

171–180 of 222 posts

Re: Nvidia DGX Spark

#171

Post by throwaway48476 »

_zoltan_ wrote: post #107
wtallis wrote: post #96

Earlier quoted context omitted.

That's eerily similar to what happened to Qualcomm's failed Snapdragon X Elite dev kit. That one eventually shipped in small quantities with a Type-C to HDMI dongle in the box to make up for the built-in HDMI port going missing. Then Qualcomm cancelled the whole project and refunded everyone, including people who had already received their hardware.

because they realized it sucked.

Because the dev units shipped after retail units. Total incompetence from QC just as I predicted.

Re: Nvidia DGX Spark

#173

Post by throwaway48476 »

aurareturn wrote: post #159
jacquesm wrote: post #150

Earlier quoted context omitted.

It's great at one thing: memory. And that's interesting because memory is a commodity, but they still make bank on just being able to access it.

Memory is a commodity but access to high bandwidth memory is expensive whether it's HBM, or LPDDR/DDR connected to many memory channels.

Its not priced in a linear way wrt to bom cost.

Re: Nvidia DGX Spark

#174

Post by bri3d »

qwertox wrote: post #145

According to Wendell from Level1Techs, the now-launched Jetson Thor uses a Linux Kernel built by Nvidia, on Ubuntu 20.04 [0]. So I assume getting upgrades will have the same feel as those Chinese SBC's like from Radxa or cheap Android devices. I wonder if this also applies to this DGX Spark. I hope not. [0] https://www.youtube.com/watch?v=cgnKUUcCKcs&t=669s

In the case of Jetson, NVidia also have a fairly generic BSP which you can use to customize almost any distribution, and the Jetson-customized Ubuntu is standard enough that you can upgrade it using the normal Ubuntu upgrade path without major issue.

For most of the Tegra boards there’s also upstream support. Overall the situation with NVidia BSP is about 10000x better than weird Chinese stuff. In the case of Tegra/Jetson, there’s even detailed first-party documentation about reconstructing the BSP components from source:

https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-327...

I’d assume the decent software support will carry over to DGX.

Re: Nvidia DGX Spark

#175

Post by imtringued »

Y_Y wrote: post #52
godelski wrote: post #21

Earlier quoted context omitted.

If you scroll down a little and see the chip icon, where it says "NVIDIA GB10 Superchip " it also says "Experience up to 1 petaFLOP of AI performance at FP4 precision with the NVIDIA Grace Blackwell architecture." Further down, in the exploded view it says "Blackwell GPU 1PetaFLOP FP4 AI Compute" Then further down in the spec chart they get less specific again with "Tensor Performance^1 1 PFLOP" and "^1" says "1 Theo…

Even if that "sparsity feature" is that two or of every four adjacent values in your areay be zeros, and that performance halves if not doing this? I think lots of children are going to be very disappointed running their blas benchmarks on Christmas morning and seeing barely tens of teraflops. (For reference see how the still optimistic numbers are for the H200 when you use realistic datatypes. https://nvdam.widen.ne…

Using sparsity in advertising is incredibly misleading to the point of lying. The entire point of sparsity is that you avoid doing calculations. Sparsity support means you need fewer FLOPs for a matrix of the same size. It doesn't magically increase the number of FLOPs you have.

Even AMD got that memo and is mostly advertising their 8bit/block fp16 performance on their GPUs and NPUs, even though the NPUs support 4 bit INT with sparsity, which would 4x the quoted numbers if they used Nvidia's marketing FLOPs.

Re: Nvidia DGX Spark

#176

Post by NewsaHackO »

artemisart wrote: post #163
ilirium wrote: post #160

Earlier quoted context omitted.

@nabla9 have tried to tell you that for DGX Spark, you can also use optimized models; therefore, this means that Spark can also be used for inference with bigger models, such as those exceeding 200B. Please compare the same things: carrots VS carrots, not apples VS eggs.

I don't understand what's not optimized on 5090. If we're comparing with Apple chips or AMD Strix Halo yes you will have very different hardware + software support, no FP4 etc. but here everything is CUDA, Blackwell vs Blackwell, same FP4 structured sparsity, so I don't get how it would be honest to compare a quantized FP4 model on Spark with an unoptimized FP16 model on a 5090 ?

To me, what I think they are saying is that the Spark can use a FP16 unoptimized model with 200B parameters. However I don't really know.

Re: Nvidia DGX Spark

#177

Post by reissbaker »

NewsaHackO wrote: post #176
artemisart wrote: post #163

Earlier quoted context omitted.

I don't understand what's not optimized on 5090. If we're comparing with Apple chips or AMD Strix Halo yes you will have very different hardware + software support, no FP4 etc. but here everything is CUDA, Blackwell vs Blackwell, same FP4 structured sparsity, so I don't get how it would be honest to compare a quantized FP4 model on Spark with an unoptimized FP16 model on a 5090 ?

To me, what I think they are saying is that the Spark can use a FP16 unoptimized model with 200B parameters. However I don't really know.

You can't. The Spark has 128GB VRAM; the highest you can go in FP16 is 64B — and that's with no space for context.

200B is probably a rough estimate of Q4 + some space for context.

The Spark has 4x the VRAM of a 5090. That's all you need to know from a "how big can it go" perspective.

Re: Nvidia DGX Spark

#178

Post by manishsharan »

zxexz wrote: post #10
lvl155 wrote: post #4

Is this worth getting vs AMD?

What are you trying to do?

Not the above poster. I would like to run agents with local multimodal LLMs to process huge numbers of sensitive files for an org( summarization, knowledge extraction, answer user questions ,etc). Any ideas?

Re: Nvidia DGX Spark

#179

Post by alkonaut »

bjackman wrote: post #119
hereme888 wrote: post #49

FP4-sparse (TFLOPS) | Price | $/TF4s 5090: 3352 | 1999 | 0.60 Thor: 2070 | 3499 | 1.69 Spark: 1000 | 3999 | 4.00 ____________ FP8-dense (TFLOPS) | Price | $/TF8d (4090s have no FP4) 4090 : 661 | 1599 | 2.42 4090 Laptop: 343 | vary | - ____________ Geekbench 6 (compute score) | Price | $/100k 4090: 317800 | 1599 | 503 5090: 387800 | 1999 | 516 M4 Max: 180700 | 1999 | 1106 M3 Ultra: 259700 | 3999 | 1540 ____________ Ap…

Note you cannot actually get a 5090 for $1999 that's just the RRP. I believe they actually cost $4k

So long as that's true, it's also likely you'll see the same markup for the spark, so they should compare similarly.

Re: Nvidia DGX Spark

#180

Post by aurareturn »

throwaway48476 wrote: post #173
aurareturn wrote: post #159

Earlier quoted context omitted.

Memory is a commodity but access to high bandwidth memory is expensive whether it's HBM, or LPDDR/DDR connected to many memory channels.

Its not priced in a linear way wrt to bom cost.

That's because memory channels cost money. Memory controllers are more complex. Lastly, chips that can make use of high bandwidth VRAM are both of the above.

Memory chips are a commodity, that I agree. Though HBM is trending towards not being a commodity.

Post reply on HN