Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

331–340 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#331

Post by rbanffy »

theptip wrote: post #311
nimish wrote: post #264

Earlier quoted context omitted.

1000% all these ai hardware companies will fail if they don't have this. You must have a cheap way to experiment and develop. Even if you want to only sell a $30000 datacenter card you still need a very low cost way to play. Sad to see big companies like intel and amd don't understand this but they've never come to terms with the fact that software killed the hardware star

Isn’t the cloud GPU market covering this? I can run a model for $2/hr, or get a 8xH100 if I need to play with something bigger.

People tend to limit their usage when it's time-billed. You need some sort of desktop computer anyway, so, if you spend the 3K this one costs, you have unlimited time of Nvidia cloud software. When you need to run on bigger metal, then you pay $2/hour.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#332

Post by adrian_b »

kllrnohj wrote: post #323
adrian_b wrote: post #273

Earlier quoted context omitted.

That Altra may be a good choice for certain server applications, like a Web server, but when used as a workstation it will be sluggish, because it uses weak cores, with much lower single-threaded performance than the Arm cores used in NVIDIA Digits. For certain applications, e.g. for those with many array operations, the 20 cores of Digits might match 40 cores of Altra at equal clock frequency, but the cores of Digit…

I mean I can get a Snapdragon X Elite laptop for $1200 that'll have a faster CPU than the one in the Digits, too...

Their speeds should be very similar, but which is faster is uncertain.

There have not been any published benchmarks demonstrating the speed of Cortex-X925 in a laptop/mini-PC environment.

In smartphones, Cortex-X925 and Snapdragon Elite have very similar speeds in single thread.

For multithreaded applications, 10 big + 10 medium Arm cores should be somewhat faster than 12 Snapdragon Elite.

The fact that NVIDIA Digits has a wider memory interface should give it even more advantages in some applications.

The Blackwell GPU should have much better software support in graphics applications, not only in ML/AI, in comparison with the Qualcomm GPU.

So NVIDIA Digits should be faster than a Qualcomm laptop, but unless one is interested in ML/AI applications the speed difference should not be worth the more than double price of NVIDIA.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#333

Post by otabdeveloper4 »

ceejayoz wrote: post #236
otabdeveloper4 wrote: post #180

Earlier quoted context omitted.

> There's a titanic market with people wanting some uncensored local LLM/image/video generation model. No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast.

AI porn is currently cringe, just like Eliza for conversations was cringe. The cutting edge will advance, and convincing bespoke porn of people's crushes/coworkers/bosses/enemies/toddlers will become a thing. With all the mayhem that results.

It will always be cringe due to how so-called "AI" works. Since it's fundamentally just log-likelihood optimization under the hood, it will always be a statistically most average image. Which means it will always have that characteristic "plastic" and overdone look.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#334

Post by mysteria »

egorfine wrote: post #314
theptip wrote: post #310

$3k for a 128GB standalone is quite favorable pricing considering the next best option at home is going to be a 32GB 5090 at $2k for the card alone, so probably $3k when you’re done building a rig around it.

The press-release says "up to 128GB" while the price is a single figure of $3,000. So it won't be out of the real of possibility that the 128GB version would cost quite a bit more.

From what I've seen the general consensus is that the 128GB of memory is standard across all models, and that the price would vary for different storage and networking configurations. Their marketing materials say that "Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage."

https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#335

Post by moffkalast »

pella wrote: post #282
moffkalast wrote: post #178

Earlier quoted context omitted.

I've seen some claims that it can do 512 GB/s on Reddit (not sure where they got that from), which would imply a ~300 bit bus with LPDDR5X depending on the frequency.

probably: "According to the Grace Blackwell's datasheet- Up to 480 gigabytes (GB) of LPDDR5X memory with up to 512GB/s of memory bandwidth. It also says it comes in a 120 gb config that does have the full fat 512 GB/s." via https://www.reddit.com/r/LocalLLaMA/comments/1hvj1f4/comment... "up to 512GB/s of memory bandwidth per Grace CPU" https://resources.nvidia.com/en-us-data-center-overview/hpc-...

Yep I think that's it. So it's referencing the GB200, it could have absolutely nothing in common with this low power version.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#336

Post by busterarm »

vladslav wrote: post #303
Abishek_Muthian wrote: post #283

I'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years. Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch et…

I've had a similar experience, my Xavier NX stopped working after the last update and now it's just collecting dust. To be honest, I've found the Nvidia SBC to be more of a hassle than it's worth.

Xavier AGX owner here to report the same.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#337

Post by logankeenan »

lolinder wrote: post #244
derbaum wrote: post #140

I'm a bit surprised by the amount of comments comparing the cost to (often cheap) cloud solutions. Nvidia's value proposition is completely different in my opinion. Say I have a startup in the EU that handles personal data or some company secrets and wants to use an LLM to analyse it (like using RAG). Having that data never leave your basement sure can be worth more than $3000 if performance is not a bottleneck.

Heck, I'm willing to pay $3000 for one of these to get a good model that runs my requests locally. It's probably just my stupid ape brain trying to do finance, but I'm infinitely more likely to run dumb experiments with LLMs on hardware I own than I am while paying per token (to the point where I currently spend way more time with small local llamas than with Claude), and even though I don't do anything sensitive I'm…

Have you been to the localLlama subreddit? It’s a great resource for running models locally. It’s what got me started.

https://www.reddit.com/r/LocalLLaMA/

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#338

Post by loudmax »

nilstycho wrote: post #326
loudmax wrote: post #319

Earlier quoted context omitted.

Going by the specs, this pretty much blows Tinybox out of the water. For $40,000, a Tinybox pro is advertised as offering 1.36 petaflops processing and 192 GB VRAM. For about $6,000 a pair of Nvidia Project Digits offer about a combined 2 petaflops processing and 256 GB VRAM. The market segment for Tinybox always seemed to be people that were somewhat price-insensitive, but unless Nvidia completely fumbles on executi…

> For about $6,000 a pair of Nvidia Project Digits offer about a combined 2 petaflops processing and 256 GB VRAM. 2 PFLOPS at FP4 . 256 GB RAM , not VRAM. I think they haven't specified the memory bandwidth.

You're right. Tinybox's 1.36 petaflops is FP16 so that is a significant difference.

Also, the Tinybox's memory bandwidth is 8064 GB/s, while the Digits seems to be around 512 GB/s, according to speculation on Reddit.

Moreover, Nvidia's announced their RTX 5090s priced at $2k, which could put downward pressure on the price of Tinybox's 4090s. So the Tinybox green or pro models might get cheaper, or they might come out with a 5090-based model.

If you're the kind of person that's ready to spend $40k on a beastly ML workstation, there's still some upside to Tinybox.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#339

Post by nzach »

mycall wrote: post #245
a_bonobo wrote: post #192

There's a market not described here: bioinformatics. The owner of the market, Illumina, already ships their own bespoke hardware chips in servers called DRAGEN for faster analysis of thousands of genomes. Their main market for this product is in personalised medicine, as genome sequencing in humans is becoming common. Other companies like Oxford Nanopore use on-board GPUs to call bases (i.e., from raw electric signal…

The bigger picture is that OpenAI o3/o4.. plus specialized models will blow open the doors to genome tagging and discovery, but that is still 1 to 3 years away for ASI to kick in.

While I kinda agree with you, I don't think we will ever find a meaningful way to throw genome sequencing data at LLMs. It's simple too much data.

I've worked in a project some years ago where we were using data from genome sequencing of a bacteria. Every sequenced sample was around 3GB of data and sample size was pretty small with only about 100 samples to study.

I think the real revolution will happen because code generation through LLMs will allow biologists to write 'good enough' code to transform, process and analyze data. Today to do any meaningful work with genome data you need a pretty competent bioinformatician, and they are a rare breed. Removing this bottleneck is what will allow us to move faster in this field.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#340

Post by egorfine »

mysteria wrote: post #334
egorfine wrote: post #314

Earlier quoted context omitted.

The press-release says "up to 128GB" while the price is a single figure of $3,000. So it won't be out of the real of possibility that the 128GB version would cost quite a bit more.

From what I've seen the general consensus is that the 128GB of memory is standard across all models, and that the price would vary for different storage and networking configurations. Their marketing materials say that "Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage." https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...

Indeed!
Post reply on HN