Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

391–400 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#391

Post by aleden »

justin66 wrote: post #345
busterarm wrote: post #336

Earlier quoted context omitted.

Xavier AGX owner here to report the same.

My Jetson TX2 developer kit didn't stop working, but it's on a very out of date Linux distribution. Maybe if Nvidia makes it to four trillion in market cap they'll have enough spare change to keep these older boards properly supported, or at least upstream all the needed support.

Are you aware that mainline linux runs on these Jetson devices? It's a bit of annoying work, but you can be running ArchLinuxARM.

https://github.com/archlinuxarm/PKGBUILDs/pull/1580

Edit: It's been a while since I did this, but I had to manually build the kernel, overwrite a dtb file maybe (and Linux_for_Tegra/bootloader/l4t_initrd.img) and run something like this (for xavier)

  sudo ./flash.sh -N 128.30.84.100:/srv/arch -K /home/aeden/out/Image -d /home/aeden/out/tegra194-p2972-0000.dtb jetson-xavier eth0

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#392

Post by dotancohen »

DannyBee wrote: post #387
gnatman wrote: post #374

>> The IBM Roadrunner was the first supercomputer to reach one petaflop (1 quadrillion floating point operations per second, or FLOPS) on May 25, 2008. $100M, 2.35MW, 6000 ft^2 >>Designed for AI researchers, data scientists, and students, Project Digits packs Nvidia’s new GB10 Grace Blackwell Superchip, which delivers up to a petaflop of computing performance for prototyping, fine-tuning, and running AI models. $3000…

Digits is petaflops of FP4, roadrunner is petaflops of FP32. So at least a factor of 8 difference, but in practice much more. (IE I strongly doubt digits can do 1/8th petaflop of FP32) Beyond that, the factors seem reasonable for 2 decades?

Why even use a floating point if you have only 4 bits? Models with INT8 features are not unheard of.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#393

Post by numba888 »

sheepscreek wrote: post #214
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

The developers they are referring to aren’t just enthusiasts; they are also developers who were purchasing SuperMicro and Lambda PCs to develop models for their employers. Many enterprises will buy these for local development because it frees up the highly expensive enterprise-level chip for commercial use. This is a genius move. I am more baffled by the insane form factor that can pack this much power inside a Mac M…

This looks like a bigger brother of Orin AGX, which has 64GB of RAM and runs smaller LLMs. The question will be power and performance vs 5090. We know price is 1.5x

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#394

Post by lolinder »

rsanek wrote: post #362
ynniv wrote: post #309

Earlier quoted context omitted.

I'm pretty frugal, but my first thought is to get two to run 405B models. Building out 128GB of VRAM isn't easy, and will likely cost twice this.

You can get a M4 Max MBP with 128GB for $1k less than two of these single-use devices.

Don't these devices provide 128GB each? So you'd need to price in two Macs to be a fair comparison to two Digits.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#395

Post by ynniv »

rsanek wrote: post #362
ynniv wrote: post #309

Earlier quoted context omitted.

I'm pretty frugal, but my first thought is to get two to run 405B models. Building out 128GB of VRAM isn't easy, and will likely cost twice this.

You can get a M4 Max MBP with 128GB for $1k less than two of these single-use devices.

These are 128GB each. Also, Nvidias inference speed is much higher than Apple's.

I do appreciate that my MBP can run models though!

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#396

Post by elorant »

lhl wrote: post #315
theptip wrote: post #310

$3k for a 128GB standalone is quite favorable pricing considering the next best option at home is going to be a 32GB 5090 at $2k for the card alone, so probably $3k when you’re done building a rig around it.

The memory bandwidth has not been announced for this device. It's probably going to be more appropriate to compare vs a 128GB M4 Max (410-546GB/s MBW) or an AMD Ryzen AI Max+ 395 (yes, that's its real name) at 256GB/s of MBW. The 5090 has 1.8TB/s of MBW and is in a whole different class performance-wise. The real question is how big of a model will you actually want to run based on how slowly tokens generate.

Well obviously it has to be low otherwise they would cannibalize their high end GPUs.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#397

Post by elorant »

loudmax wrote: post #319
quick_brown_fox wrote: post #145

How about “We sell a computer called the tinybox. It comes in two colors + pro. tinybox red and green are for people looking for a quiet home/office machine. tinybox pro is for people looking for a loud compact rack machine.” [0] [0] https://tinygrad.org/#tinybox

Going by the specs, this pretty much blows Tinybox out of the water. For $40,000, a Tinybox pro is advertised as offering 1.36 petaflops processing and 192 GB VRAM. For about $6,000 a pair of Nvidia Project Digits offer about a combined 2 petaflops processing and 256 GB VRAM. The market segment for Tinybox always seemed to be people that were somewhat price-insensitive, but unless Nvidia completely fumbles on executi…

You're missing the most critical part though. Memory bandwidth. It hasn't been announced yet for Digits and it probably won't be comparable to that of dedicated GPUs.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#398

Post by godelski »

lolinder wrote: post #307
stabbles wrote: post #305

Earlier quoted context omitted.

Now that the majority of their revenue is from data centers instead of Windows gaming PCs, you'd think their relationship with Linux should improve or already has.

It's possible. I haven't had a system completely destroyed by Nvidia in the last few years, but I've been assuming that's because I've gotten in the habit of just not touching it once I get it working...

I update drivers regularly. I've only had one display failure and was solved by a simple rollback. To be a bit fair (:/) it was specifically a combination of new beta driver and a newer kernel. It's definitely improved a ton since 10 years ago I just would not update them except very carefully.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#399

Post by ceejayoz »

otabdeveloper4 wrote: post #333
ceejayoz wrote: post #236

Earlier quoted context omitted.

AI porn is currently cringe, just like Eliza for conversations was cringe. The cutting edge will advance, and convincing bespoke porn of people's crushes/coworkers/bosses/enemies/toddlers will become a thing. With all the mayhem that results.

It will always be cringe due to how so-called "AI" works. Since it's fundamentally just log-likelihood optimization under the hood, it will always be a statistically most average image. Which means it will always have that characteristic "plastic" and overdone look.

The current state of the art in AI image generation was unimaginable a few years back. The idea that it'll stay as-is for the next century seems... silly.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#400

Post by robhlt »

stabbles wrote: post #305
lolinder wrote: post #301

Earlier quoted context omitted.

This is a very important point. In general, Nvidia's relationship with Linux has been... complicated. On the one hand, at least they offer drivers for it. On the other, I have found few more reliable ways to irreparably break a Linux installation than trying to install or upgrade those drivers. They don't seem to prioritize it as a first class citizen, more just tolerate it the bare minimum required to claim it works…

Now that the majority of their revenue is from data centers instead of Windows gaming PCs, you'd think their relationship with Linux should improve or already has.

They have been making real improvements the last few years. Most of their proprietary driver code is in firmware now, and the kernel driver is open-source[1] (the userland-side is still closed though).

They've also significantly improved support for wayland and stopped trying to force eglstreams on the community. Wayland+nvidia works quite well now, especially after they added explicit sync support.

1. https://github.com/NVIDIA/open-gpu-kernel-modules/

Post reply on HN