Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

501–510 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#501

Post by aparashk »

sliken wrote: post #116
Topfi wrote: post #84

Earlier quoted context omitted.

I see the most direct competitor in the Mac Studio, though of course we will have to wait for reviews to gauge how fair that comparison is. The Studio does have a fairly large niche as a solid workstation, though, so I could see this being successful. For general desktop use, as you described, nearly any piece of modern hardware, from a RasPI, to most modern smartphones with a dock, could realistically serve most peo…

I did want a M2 max studio, ended up with a 12 core Zen 4 + radeon 7800 XT for about half the money. A Nvidia Project Digit/GB10 for $3k with 128GB ram does sound tempting. Especially since it's very likely to have standard NVMe storage that I can expand or replace as needed, unlike the Apple solution. Decent linux support is welcome as well. Here's hoping, if not I can fall back to a 128GB ram AMD Strix Halo/395 AI…

"Starting from $3K" likely means that the maxed out memory configuration will cost more than $3K.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#502

Post by sliken »

aparashk wrote: post #501
sliken wrote: post #116

Earlier quoted context omitted.

I did want a M2 max studio, ended up with a 12 core Zen 4 + radeon 7800 XT for about half the money. A Nvidia Project Digit/GB10 for $3k with 128GB ram does sound tempting. Especially since it's very likely to have standard NVMe storage that I can expand or replace as needed, unlike the Apple solution. Decent linux support is welcome as well. Here's hoping, if not I can fall back to a 128GB ram AMD Strix Halo/395 AI…

"Starting from $3K" likely means that the maxed out memory configuration will cost more than $3K.

As several other threads of mentioned, Nvidia has statements in various places along the lines of "All Nvidia Digit systems will have 128GB". Looks like you might be able to select different storage or something or maybe the number of cores (CPU and/or GPU).

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#503

Post by menaerus »

trhway wrote: post #486
menaerus wrote: post #480

Earlier quoted context omitted.

> So, typical database operations, massively parallel in nature like join or filter, would run about that faster. Given workload A how much of the total runtime JOIN or FILTER would take in contrast to the storage engine layer for example? My gut feeling tells me not much since to see the actual gain you'd need to be able to parallelize everything including the storage engine challenges. IIRC all the startups buildin…

With cheap large RAMs and the SSD the storage has already became much less of an issue, especially when the database is primarily in-memory one. How about attaching SSD based storage to NVLink? :) Nvidia does have the direct to memory tech and uses wide buses, so i don't see any issue for them to direct attach arrays of SSD if they feel like it. >IIRC all the startups building databases around GPUs failed to deliver…

I mean you wouldn't run a database on a GB10 device or cluster of them thereof. GH200 is another story, however, the potential improvement wrt the databases-in-GPUs still falls short of the question if there are enough workloads that are compute-bound in the substantial part of total wall-clock time for given workload.

In other words, and hypothetically, if you can improve logical plan execution to run 2x faster by rewriting the algorithms to make use of GPU resources but physical plan execution remains to be bottlenecked by the storage engine, then the total sum of gains is negligible.

But I guess there could perhaps be some use-case where this could be proved as a win.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#504

Post by immibis »

tokai wrote: post #177
immibis wrote: post #34

Earlier quoted context omitted.

Never underestimate the open source world's power to create a crappy desktop experience.

You're like 15 years out of date.

No, it's still relevant in current year. Progress has gone backwards on some projects/distros.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#505

Post by rbanffy »

seanmcdirmid wrote: post #499
rbanffy wrote: post #435

Earlier quoted context omitted.

This has 128GB of unified memory. A similarly configured Mac Studio costs almost twice as much, and I'm not sure the GPU is on the same league (software support wise, it isn't, but that's fixable). A real shame it's not running mainline Linux - I don't like their distro based on Ubuntu LTS.

$4,799 for an M2 Ultra with 128GB of RAM, so not quite twice as much. I'm not sure what the benchmark comparison would be. $5,799 if you want an extra 16 GPU cores (60 vs 76).

We'll need to look into benchmarks when the numbers come out. Software support is also important, and a Mac will not help you that much if you are targeting CUDA.

I have to agree the desktop experience of the Mac is great, on par with the best Linuxes out there.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#506

Post by seanmcdirmid »

rbanffy wrote: post #505
seanmcdirmid wrote: post #499

Earlier quoted context omitted.

$4,799 for an M2 Ultra with 128GB of RAM, so not quite twice as much. I'm not sure what the benchmark comparison would be. $5,799 if you want an extra 16 GPU cores (60 vs 76).

We'll need to look into benchmarks when the numbers come out. Software support is also important, and a Mac will not help you that much if you are targeting CUDA. I have to agree the desktop experience of the Mac is great, on par with the best Linuxes out there.

A lot of models are optimized for metal already, especially lamma, deepseek, and qwen. You are still taking a hit but there wasn't an alternative solution for getting that much vram in a less than $5k before this NVIDIA project came out. Will definitely look at it closely if it isn't just vaporware.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#507

Post by bmicraft »

adrian_b wrote: post #477
bmicraft wrote: post #460

Earlier quoted context omitted.

> The thermal design power of a laptop CPU-dGPU combo is normally much higher than 120 W Normally? Much higher than 120W? Those are some pretty abnormal (and dare I say niche?) laptops you're talking about there. Remember, that's not peak power - thermal design power is what the laptop should be able to power and cool pretty much continuously. At those power levels, they're usually called DTR: desktop replacement. Yo…

Any laptop that in marketed as "gaming laptop" or "mobile workstation" belongs to this category. I do not know which is the proportion of gaming laptops and mobile workstations vs. thin and light laptops. While obviously there must be much more light laptops, the gaming laptops cannot be a niche product, because there are too many models offered by a lot of vendors. My own laptop is a Dell Precision, so it belongs to…

You can't usually just add up the TDPs of CPU and GPU, because neither cooling nor the power circuitry supports that kind of load. That's why AMDs SmartShift is a thing.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#508

Post by rbanffy »

bmicraft wrote: post #458
rbanffy wrote: post #331

Earlier quoted context omitted.

People tend to limit their usage when it's time-billed. You need some sort of desktop computer anyway, so, if you spend the 3K this one costs, you have unlimited time of Nvidia cloud software. When you need to run on bigger metal, then you pay $2/hour.

3k is still very steep for anyone not on a silicon valley like salary.

Yes. Most people make do with a generic desktop and an Nvidia GPU. What makes this machine attractive is the beefy GPU and the full Nvidia support for the whole AI stack.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#509

Post by rbanffy »

seanmcdirmid wrote: post #506
rbanffy wrote: post #505

Earlier quoted context omitted.

We'll need to look into benchmarks when the numbers come out. Software support is also important, and a Mac will not help you that much if you are targeting CUDA. I have to agree the desktop experience of the Mac is great, on par with the best Linuxes out there.

A lot of models are optimized for metal already, especially lamma, deepseek, and qwen. You are still taking a hit but there wasn't an alternative solution for getting that much vram in a less than $5k before this NVIDIA project came out. Will definitely look at it closely if it isn't just vaporware.

They cant walk back now without some major backlash.

The one thing I wonder is noise. That box is awfully small for the amount of compute it packs, and high-end Mac Studios are 50% heatsink. There isn’t much space in this box for a silent fan.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#510

Post by Tepix »

nomel wrote: post #454
bloomingkales wrote: post #194

Earlier quoted context omitted.

Jensen did say in recent interview, paraphrasing, “they are trying to kill my company”. Those Macs with unified memory is a threat he is immediately addressing. Jensen is a wartime ceo from the looks of it, he’s not joking. No wonder AMD is staying out of the high end space, since NVIDIA is going head on with Apple (and AMD is not in the business of competing with Apple).

> since NVIDIA is going head on with Apple I think this is a race that Apple doesn't know it's part of. Apple has something that happens to work well for AI, as a side effect of having a nice GPU with lots of fast shared memory. It's not marketed for inference.

Apple is both well aware and marketing it, as seen at https://www.apple.com/my/newsroom/2024/10/apples-new-macbook...

Quote:

"It also supports up to 128GB of unified memory, so developers can easily interact with LLMs that have nearly 200 billion parameters."

Post reply on HN