Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

241–250 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#241

Not much was unveiled but it showed a Blackwell GPU with 1PFLOP of FP4 compute, 128GB unified DDR5X memory, 20 ARM cores, and ConnectX powering two QSFP slots so one can stack multiple of them. edit: While the title says "personal", Jensen did say this was aimed at startups and similar, so not your living room necessarily.

From the size and pricing ($3000) alone, it's safe to conclude it has less raw FLOPs than a 5090. Since it uses LPDDR5X, almost certainly less memory bandwidth too (5090 @ 1.8 TB/s, M4 Max w/ 128GB LPDDR5X @ 546 GB/s). Basically the only advantage is how much VRAM it packs in a small form factor, and presumably greater power efficiency at its smaller scale. The only thing it really competes with is the Mac Studio for…

The product isn't even finalized. It might never come to fruition, and I cannot fathom how they will make the power profile fit. I am skeptical that a $3000 device with 128GB of RAM and a 4TB SSD with the specs provided will even see reality any time within the next year, but let's pretend it will.

However we do know that it offers 1/4 the TOPS of the new 5090. It will be less powerful than the $600 5070. Which, of course it will given power limitations.

The only real compelling value is that nvidia memory starves their desktop cards so severely. It's the small opening that Apple found, even though Apple's FP4/FP8 performance is a world below what nvidia is offering. So purely from that perspective this is a winning product, as 128GB opens up a lot of possibilities. But from a raw performance perspective, it's actually going to pale compared to other nvidia products.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#242

Earlier quoted context omitted.

There's a lot more gamers than people wanting to play with LLms at home.

There's a titanic market with people wanting some uncensored local LLM/image/video generation model. This market extremely overlaps with gamers today, but will grow exponentially every year.

I'm sure a lot of people see "uncensored" and think "porn" but there's a lot of stuff that e.g. Dall-E won't let you do.

Suppose you're a content creator and you need an image of a real person or something copyrighted like a lot of sports logos for your latest YouTube video's thumbnail. That kind of thing.

I'm not getting into how good or bad that is; I'm just saying I think it's a pretty common use case.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#244

I'm a bit surprised by the amount of comments comparing the cost to (often cheap) cloud solutions. Nvidia's value proposition is completely different in my opinion. Say I have a startup in the EU that handles personal data or some company secrets and wants to use an LLM to analyse it (like using RAG). Having that data never leave your basement sure can be worth more than $3000 if performance is not a bottleneck.

Heck, I'm willing to pay $3000 for one of these to get a good model that runs my requests locally. It's probably just my stupid ape brain trying to do finance, but I'm infinitely more likely to run dumb experiments with LLMs on hardware I own than I am while paying per token (to the point where I currently spend way more time with small local llamas than with Claude), and even though I don't do anything sensitive I'm still leery of shipping all my data to one of these companies.

This isn't competing with cloud, it's competing with Mac Minis and beefy GPUs. And $3000 is a very attractive price point in that market.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#245

There's a market not described here: bioinformatics. The owner of the market, Illumina, already ships their own bespoke hardware chips in servers called DRAGEN for faster analysis of thousands of genomes. Their main market for this product is in personalised medicine, as genome sequencing in humans is becoming common. Other companies like Oxford Nanopore use on-board GPUs to call bases (i.e., from raw electric signal…

The bigger picture is that OpenAI o3/o4.. plus specialized models will blow open the doors to genome tagging and discovery, but that is still 1 to 3 years away for ASI to kick in.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#246
post #155
post #29

Earlier quoted context omitted.

I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs. Surely a smaller market than gamers or datacenters for sure.

It's annoying I do LLMs for work and have a bit of an interest in them and doing stuff with GANS etc. I have a bit of an interest in games too. If I could get one platform for both, I could justify 2k maybe a bit more. I can't justify that for just one half: running games on Mac, right now via Linux: no thanks. And on the PC side, nvidia consumer cards only go to 24gb which is a bit limiting for LLMs, while being ver…

32gb as of last night :)

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#247
One thing I didn't see mentioned: this would be a good motivation for Nvidia to release "open weights" models.

Just like Mac OS is free when you buy a Mac, having the latest high-quality LLM for free that just happens to run well on this box is a very interesting value-prop. And Nvidia definitely has the compute to make it happen.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#248

I'm a bit surprised by the amount of comments comparing the cost to (often cheap) cloud solutions. Nvidia's value proposition is completely different in my opinion. Say I have a startup in the EU that handles personal data or some company secrets and wants to use an LLM to analyse it (like using RAG). Having that data never leave your basement sure can be worth more than $3000 if performance is not a bottleneck.

The price seems relatively competitive even compared to other local alternatives like "build your own PC". I'd definitely buy one of this (or even two if it works really well) for developing/training/using models that currently run on cobbled together hardware I got left after upgrading my desktop.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#249
post #247

One thing I didn't see mentioned: this would be a good motivation for Nvidia to release "open weights" models. Just like Mac OS is free when you buy a Mac, having the latest high-quality LLM for free that just happens to run well on this box is a very interesting value-prop. And Nvidia definitely has the compute to make it happen.

[deleted]

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#250

I'm a bit surprised by the amount of comments comparing the cost to (often cheap) cloud solutions. Nvidia's value proposition is completely different in my opinion. Say I have a startup in the EU that handles personal data or some company secrets and wants to use an LLM to analyse it (like using RAG). Having that data never leave your basement sure can be worth more than $3000 if performance is not a bottleneck.

Yeah that's cheaper than many prosumer GPUs on the market right now
Post reply on HN