"Metal foam" sounds cool but it just looks like a steel wool pad you would use for cleaning dishes.
Helps with the cooling is my guess. Increased surface area
NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
71–80 of 100 posts
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#72Earlier quoted context omitted.
More precisely, the RTX 5090 has a memory bandwidth of 1792 GB/s, while the DGX Spark only has 273 GB/s, which is about 1/6.5. For inference, the DGX Spark does not look like a good choice, as there are cheaper alternatives with better performance.
My understanding is that the Jetson Thor is just as good a platform, and likely more readily available. Then there's the Mac Studio, which outdoes them in all respects except FP8 and FP4 support. As someone on Reddit put it: https://old.reddit.com/r/LocalLLaMA/comments/1n0xoji/why_can...
I’ve been thinking the same… I have jetson Thor and only difference I can imagine is the capability to connect two DGX sparks together… but then I’d rather go for RTX pro 6000 instead of buying two DGX spark units, because I prefer the higher memory bandwidth, more Cuda cores, tensor cores and RT cores over 256 GB memory for my use case.
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#73"Metal foam" sounds cool but it just looks like a steel wool pad you would use for cleaning dishes.
Helps with the cooling is my guess. Increased surface area
Possibly if the case is being used as a heat sink, but in that case it would be unsafe to touch. I think it's just being used instead of a traditional mesh panel.
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#74"Metal foam" sounds cool but it just looks like a steel wool pad you would use for cleaning dishes.
Helps with the cooling is my guess. Increased surface area
I'm pretty sure they just want to be coherent with the which has that "steel scrubber finish" on the hardware
(photo for reference: https://www.wwt.com/api-new/attachments/5f033e355091b0008017...)
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#75You guys that continue to compare DGX Spark to the Mac Studios, please remember two things: 1. Virtually every model that you'd run was developed on Nvidia gear and will run on Spark. 2. Spark has fast-as-hell interconnects. The sort of interconnects that one would want to use in an actual AI DC, so you can use more than one Spark at the same time, and RDMA, and actually start to figure out how things work the way th…
Also remember that the Mx Ultras have 2-3x the memory bandwidth. Looking at the benchmarks even Strix Halo seems to beat the Spark. Buying a 200 Gbps switch is $10k-$100k+ so don't imagine anyone actually will use the interconnect. The logical thing for Nvidia would be to sell a kit with three machines and cabling, and make it a ring with the dual ports per machine. Helps for some scenarios but not others with the 10…
On another note to remember, you can also ring topology mac studios using TB5 for 120Gbps per link with four such ports, all using cheaply available cable
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#76How representative is this platform of the bigger GB200 and GB300 chips? Could I write code that runs on Spark and effortlessly run it on a big GB300 system with no code changes?
All three (GB10, GB200 and GB300) are part of the Blackwell family, which means they have Compute Capability >= 10.X. You could potentially develop kernels to optimize MoE inference (given the large available unified memory, 128Gb, it makes the most sense to me) with CUDA >= 12.9 then ship the fatbins to the "big boys". As many people have pointed out across the thread, the spark doesn't really has the best perf/$, it's rather a small portable platform for experimentation and development
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#77You guys that continue to compare DGX Spark to the Mac Studios, please remember two things: 1. Virtually every model that you'd run was developed on Nvidia gear and will run on Spark. 2. Spark has fast-as-hell interconnects. The sort of interconnects that one would want to use in an actual AI DC, so you can use more than one Spark at the same time, and RDMA, and actually start to figure out how things work the way th…
It would be very interesting to read a tutorial on case 2.
@pavlov here's the tutorial that you wanted. https://youtu.be/rKOoOmIpK3I?si=WgLTee3Kc1SnUbDZ
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#78You guys that continue to compare DGX Spark to the Mac Studios, please remember two things: 1. Virtually every model that you'd run was developed on Nvidia gear and will run on Spark. 2. Spark has fast-as-hell interconnects. The sort of interconnects that one would want to use in an actual AI DC, so you can use more than one Spark at the same time, and RDMA, and actually start to figure out how things work the way th…
Also remember that the Mx Ultras have 2-3x the memory bandwidth. Looking at the benchmarks even Strix Halo seems to beat the Spark. Buying a 200 Gbps switch is $10k-$100k+ so don't imagine anyone actually will use the interconnect. The logical thing for Nvidia would be to sell a kit with three machines and cabling, and make it a ring with the dual ports per machine. Helps for some scenarios but not others with the 10…
| Buying a 200 Gbps switch is $10k-$100k+
$1,295.00
https://www.balticnetworks.com/products/mikrotik-crs812-ddq-...
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#79Any views on how Isaac Lab / Isaac Sim would perform on DGX Spark?
Re: NVIDIA DGX Spark In-Depth Review: A New Standard for Local AI Inference
#80That memory bandwidth choked out their performance. How can you claim 1000 tflops if it's not capable of delivering it. Seems they chose to sandbag the spark in favour of the rtx pro 6000. I guess my next one I'm looking out for is the Orange Pi AI studio pro. Should have 192gb of ram, so able to run qwen3 235b, even though it's ddr4, it's nearly double the bandwidth of the spark.
Good luck with any kind of coherent ecosystem and support. Also, if you're in the U.S., there is a good chance you'll get hit with tariffs which would wipe out any potential value. I'd much rather stick with nVidia that has an ecosystem (even Apple for that matter), than touch a system like this off of Alibaba.