Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

lmsys.org

Post reply on HN
Print view View original

1–10 of 100 posts

Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

#3

Post by CamperBob2 »

SethTro wrote: post #2

Article doesn't seem to mention price which is $4,000 which makes it comparable to a 5090 but with 128GB of unified LPDDR5x vs the 5090's 32GB DDR7.

And about 1/4 the memory bandwidth, which is what matters for inference.

Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

#4

Post by nialse »

SethTro wrote: post #2

Article doesn't seem to mention price which is $4,000 which makes it comparable to a 5090 but with 128GB of unified LPDDR5x vs the 5090's 32GB DDR7.

Well, that’s disappointing since the Mac Studio 128GB is $3,499. If Apple happens to launch a Mac Mini with 128GB RAM it would eat Nvidia Sparks’ lunch every day.

Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

#6

Post by newman314 »

nialse wrote: post #4
SethTro wrote: post #2

Article doesn't seem to mention price which is $4,000 which makes it comparable to a 5090 but with 128GB of unified LPDDR5x vs the 5090's 32GB DDR7.

Well, that’s disappointing since the Mac Studio 128GB is $3,499. If Apple happens to launch a Mac Mini with 128GB RAM it would eat Nvidia Sparks’ lunch every day.

Agreed. I also wonder why they chose to test against a Mac Studio with only 64GB instead of 128GB.

Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

#8

Post by yvbbrjdr »

newman314 wrote: post #6
nialse wrote: post #4

Earlier quoted context omitted.

Well, that’s disappointing since the Mac Studio 128GB is $3,499. If Apple happens to launch a Mac Mini with 128GB RAM it would eat Nvidia Sparks’ lunch every day.

Agreed. I also wonder why they chose to test against a Mac Studio with only 64GB instead of 128GB.

Hi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that one of my friend has this device.

Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

#9

Post by yvbbrjdr »

pixelpoet wrote: post #5

I wonder why they didn't test against the broadly available Strix Halo with 128GB of 256 GB/s memory bandwidth, 16 core full-fat Zen5 with AVX512 at $2k... it is a mystery...

Hi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that none of my friend has this device.

Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference

#10

Post by ggerganov »

yvbbrjdr wrote: post #8
newman314 wrote: post #6

Earlier quoted context omitted.

Agreed. I also wonder why they chose to test against a Mac Studio with only 64GB instead of 128GB.

Hi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that one of my friend has this device.

FYI you should have used llama.cpp to do the benchmarks. It performs almost 20x faster than ollama for the gpt-oss-120b model. Here are some samples results on my spark:

  ggml_cuda_init: found 1 CUDA devices:
    Device 0: NVIDIA GB10, compute capability 12.1, VMM: yes
  | model                          |       size |     params | backend    | ngl | n_ubatch | fa |            test |                  t/s |
  | ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
  | gpt-oss 20B MXFP4 MoE          |  11.27 GiB |    20.91 B | CUDA       |  99 |     2048 |  1 |          pp4096 |       3564.31 ± 9.91 |
  | gpt-oss 20B MXFP4 MoE          |  11.27 GiB |    20.91 B | CUDA       |  99 |     2048 |  1 |            tg32 |         53.93 ± 1.71 |
  | gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       |  99 |     2048 |  1 |          pp4096 |      1792.32 ± 34.74 |
  | gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       |  99 |     2048 |  1 |            tg32 |         38.54 ± 3.10 |
Post reply on HN