Live data from Hacker News

Nvidia DGX Spark

nvidia.com

211–220 of 222 posts

Re: Nvidia DGX Spark

#211

Earlier quoted context omitted.

Note you cannot actually get a 5090 for $1999 that's just the RRP. I believe they actually cost $4k

I just googled it and the first result was one in stock for £2200. That's including tax. I assume $1999 is excluding tax. Without tax and converted to dollars it's $2470. From other less reliable sources like eBay they are more like £1800.

Huh, you are right. I googled it yesterday too but I guess I had confirmation bias and happened to stumble across an old price and go "yep still 4k".

Well, I'm glad to be wrong on his!

Re: Nvidia DGX Spark

#212

Earlier quoted context omitted.

What are they examining your bags for purchased items? That’s gestapoish.

1. Yes they will if they suspect you (age group, clothes, newest phone, certain flights like LAX, LGA) as all custom officers all over the world do. As my bags have been searched every time I've entered the US. 2. You should read up on the Gestapo

US customs won't care if they find new electronics, so they're no problem (they are annoying, with the suitcase thing at port-of-entry, but no problem). As for German customs, I don't know, but: do the initial leave out of Germany over the road where there's checks only in theory and leave from an airport not too far over the border. You can probably get a cheaper flight in the process (e.g. Fly from Basel)

Re: Nvidia DGX Spark

#213

Earlier quoted context omitted.

Matrix vector multiplication for feed forward layers is most of the bandwidth as I understand things, there's not really a way to do it "better", its just a bunch of memory-bound dot products. (Posting this comment in hopes of being corrected and learning something).

The problem is different parts of the SoC (CPU, GPU, NPU) may not actually be able to consume all of the bandwidth available to the system as a whole. This is why you'd need to benchmark - different chips may be able to feed the cores better than others.

Ah, yeah. I guess as we venture further into SoCs that will be more common, I was just thinking "it's whatever the memory controller can do".

Re: Nvidia DGX Spark

#214

Earlier quoted context omitted.

I'm pretty new to this, so if I wanted to benchmark my current hardware and compare to your results what would be the best way to do that? I'm looking at going for a Framework Desktop and would like to know what kind of performance gain I'd get over the current hardware I have, which so far I have a "feel" for the performance of from running Ollama and OpenWebUI, but no hard numbers.

What nobody seems to ever share is the context and TTFT (time to first token). You can get a very good TPS by using small prompts, even if the output tokens are very large. If you try to do any kind of agentic coding locally, where contexts are 7k+, local hardware completely falls over. qwen-code (cli) gives like 2k requests per day for free (and is fantastic), so unless you have a very specific use case, buying a sy…

My use cases are mostly for automation, and local-only is a must.

I currently use the GPU in my server for n8n and Home Assistant with small-ish tooling models that fit in my 8GB VRAM.

TTFT is pretty poor right now, I get 10+ seconds for the longer inputs from HA, n8n isn't too bad unless I'm asking it to handle a largish input, but that one is less time sensitive as it's running things on schedules rather than when I need output.

Ideally I'd like to get Assistant responses in HA to under about 2s if possible.

Looking also for a new desktop at some point but I don't want to use the same hardware, the inference GPU is in a server that's always on running "infrastructure" (Kubernetes, various pieces of software, NAS functionality, etc), but I've always build desktops from components since I was a wee child when a 1.44MB floppy was an upgrade, so a part of me is reluctant to switch to a mini-PC for that;

I might be convinced to get a Framework Desktop though if it'll do for Steam gaming on Linux knowing that when I eventually need to upgrade it, it could supplement my server rack and be replaced entirely with a new model on the desktop, given there's very little upgrade path than to replace the entire mainboard.

No real interest in coding assistants, and running within my home network is an absolute must, which limits capability to "what's the best hardware I can afford?".

Re: Nvidia DGX Spark

#215

Earlier quoted context omitted.

from the NVidia DGX Spark datasheet: With 128 GB of unified system memory, developers can experiment, fine-tune, or inference models of up to 200B parameters. Plus, NVIDIA ConnectX™ networking can connect two NVIDIA DGX Spark supercomputers to enable inference on models up to 405B parameters.

The datasheet isn't telling you the quantization (intentionally). Model weights at FP16 are roughly 2GB per billion params. A 200B model at FP16 would take 400GB just to load the weights; a single DGX Spark has 128GB. Even two networked together couldn't do it at FP16. You can do it, if you quantize to FP4 — and Nvidia's special variant of FP4, NVFP4, isn't too bad (and it's optimized on Blackwell). Some models are e…

I never claimed the 200B model was FP16.

If the 200B model was at FP16, marketing could've turned around and claimed the DGX Spark could handle a 400B model (with an 8-bit quant) or a 800B model at some 4-bit quant.

Why would marketing leave such low-hanging fruit on the tree?

They wouldn't.

Re: Nvidia DGX Spark

#216

Are there any news about power consumption? I didn’t even see a tdp or so mentioned.

One of the first things I looked at too...

from my comment elsewhere in this thread, https://news.ycombinator.com/item?id=45048078, "up to 170W." was the quote from March.

Re: Nvidia DGX Spark

#217

Earlier quoted context omitted.

Memory controllers are die area. mm2 die space is linear bom cost. Ignoring the design and platform support that comes with higher bandwidth memory controllers.

And this is not also linear or fixed cost, why? I don't think you know how industry pricing works. Wafers have a price, double mm2, double the price of chip in bom.

Yes. Bigger die size. More complex memory controller designs. More memory lanes. More software support for higher memory lanes. All adds up.

I'm explaining why despite memory being a commodity, high memory bandwidth VRAM cost is not cheap.

Re: Nvidia DGX Spark

#218

Earlier quoted context omitted.

And this is not also linear or fixed cost, why? I don't think you know how industry pricing works. Wafers have a price, double mm2, double the price of chip in bom.

Yes. Bigger die size. More complex memory controller designs. More memory lanes. More software support for higher memory lanes. All adds up. I'm explaining why despite memory being a commodity, high memory bandwidth VRAM cost is not cheap.

Memory controllers are copy paste. It's not more complex.

It's more expensive in a linear way wrt bom.

Re: Nvidia DGX Spark

#220

FP4-sparse (TFLOPS) | Price | $/TF4s 5090: 3352 | 1999 | 0.60 Thor: 2070 | 3499 | 1.69 Spark: 1000 | 3999 | 4.00 ____________ FP8-dense (TFLOPS) | Price | $/TF8d (4090s have no FP4) 4090 : 661 | 1599 | 2.42 4090 Laptop: 343 | vary | - ____________ Geekbench 6 (compute score) | Price | $/100k 4090: 317800 | 1599 | 503 5090: 387800 | 1999 | 516 M4 Max: 180700 | 1999 | 1106 M3 Ultra: 259700 | 3999 | 1540 ____________ Ap…

[deleted]
Post reply on HN