Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

321–330 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#321

Post by halJordan »

Abishek_Muthian wrote: post #283

I'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years. Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch et…

It runs their dgx os and Jensen specifically said it would be a full part if their hw stack

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#322

Post by Paradigma11 »

otabdeveloper4 wrote: post #180
anonylizard wrote: post #91

Earlier quoted context omitted.

There's a titanic market with people wanting some uncensored local LLM/image/video generation model. This market extremely overlaps with gamers today, but will grow exponentially every year.

> There's a titanic market with people wanting some uncensored local LLM/image/video generation model. No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast.

I think scams will create a far more demand. Spear Phishing targets by creating persistent elaborate online environments is going to be big.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#323

Post by kllrnohj »

adrian_b wrote: post #273
kllrnohj wrote: post #203

Earlier quoted context omitted.

> At $3,000, it will be considerably cheaper than alternatives available today $3000 gets me a 64-core Altra Q64-22 from a major-enough SI today: https://system76.com/desktops/thelio-astra-a1-n1/configure And of course if you don't care about the SI part, then you can just buy that motherboard & CPU directly for $1400 https://www.newegg.com/asrock-rack-altrad8ud-1l2t-q64-22-amp... with the 128-core variant being $240…

That Altra may be a good choice for certain server applications, like a Web server, but when used as a workstation it will be sluggish, because it uses weak cores, with much lower single-threaded performance than the Arm cores used in NVIDIA Digits. For certain applications, e.g. for those with many array operations, the 20 cores of Digits might match 40 cores of Altra at equal clock frequency, but the cores of Digit…

I mean I can get a Snapdragon X Elite laptop for $1200 that'll have a faster CPU than the one in the Digits, too...

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#324

Post by otabdeveloper4 »

Filligree wrote: post #298
otabdeveloper4 wrote: post #180

Earlier quoted context omitted.

> There's a titanic market with people wanting some uncensored local LLM/image/video generation model. No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast.

> No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast. I needed an uncensored model in order to, guess what, make an AI draw my niece snowboarding down a waterfall. All the online services refuse on basis that the picture contains -- oh horrors -- a child. "Uncensored" absolutely does not imply NSFW.

Yeah, and there's that story about "private window" mode in browsers because you were shopping for birthday gifts that one time. You know what I mean though.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#325

Post by moralestapia »

kelsey98765431 wrote: post #269
moralestapia wrote: post #222

Earlier quoted context omitted.

Yes, but people already had their Macs for others reasons. No one goes to an Apple store thinking "I'll get a laptop to do AI inference".

my $5k m3 max 128gb disagrees

Doubt it, a year ago useful local LLMs on a Mac (via something like ollama) was barely taking off.

If what you say it's true you were among the first 100 people on the planet who were doing this; which btw, further supports my argument on how extremely rare is that use case for Mac users.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#326

Post by nilstycho »

loudmax wrote: post #319
quick_brown_fox wrote: post #145

How about “We sell a computer called the tinybox. It comes in two colors + pro. tinybox red and green are for people looking for a quiet home/office machine. tinybox pro is for people looking for a loud compact rack machine.” [0] [0] https://tinygrad.org/#tinybox

Going by the specs, this pretty much blows Tinybox out of the water. For $40,000, a Tinybox pro is advertised as offering 1.36 petaflops processing and 192 GB VRAM. For about $6,000 a pair of Nvidia Project Digits offer about a combined 2 petaflops processing and 256 GB VRAM. The market segment for Tinybox always seemed to be people that were somewhat price-insensitive, but unless Nvidia completely fumbles on executi…

> For about $6,000 a pair of Nvidia Project Digits offer about a combined 2 petaflops processing and 256 GB VRAM.

2 PFLOPS at FP4.

256 GB RAM, not VRAM. I think they haven't specified the memory bandwidth.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#327

Post by moralestapia »

JohnBooty wrote: post #239
moralestapia wrote: post #222

Earlier quoted context omitted.

Yes, but people already had their Macs for others reasons. No one goes to an Apple store thinking "I'll get a laptop to do AI inference".

They have, because until now Apple Silicon was the only practical way for many to work with larger models at home because they can be configured with 64-192GB of unified memory. Even the laptops can be configured with up to 128GB of unified memory. Performance is not amazing (roughly 4060 level, I think?) but in many ways it was the only game in town unless you were willing and able to build a multi-3090/4090 rig.

I would bet that people running LLMs on their Macs, today, is <0.1% of their user base.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#328

Post by AtlasBarfed »

neom wrote: post #276

In case you're curious, I googled. It runs this thing called "DGX OS": "DGX OS 6 Features The following are the key features of DGX OS Release 6: Based on Ubuntu 22.04 with the latest long-term Linux kernel version 5.15 for the recent hardware and security updates and updates to software packages, such as Python and GCC. Includes the NVIDIA-optimized Linux kernel, which supports GPU Direct Storage (GDS) without addit…

Nvidia optimize meaning non-public patches, a non-upgradable operating system like what happens if you upgrade with a binary blob Nvidia driver?

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#329

Post by adrian_b »

T-A wrote: post #275
bloomingkales wrote: post #194

Earlier quoted context omitted.

Jensen did say in recent interview, paraphrasing, “they are trying to kill my company”. Those Macs with unified memory is a threat he is immediately addressing. Jensen is a wartime ceo from the looks of it, he’s not joking. No wonder AMD is staying out of the high end space, since NVIDIA is going head on with Apple (and AMD is not in the business of competing with Apple).

From https://www.tomshardware.com/pc-components/cpus/amds-beastly... The fire-breathing 120W Zen 5-powered flagship Ryzen AI Max+ 395 comes packing 16 CPU cores and 32 threads paired with 40 RDNA 3.5 (Radeon 8060S) integrated graphics cores (CUs), but perhaps more importantly, it supports up to 128GB of memory that is shared among the CPU, GPU, and XDNA 2 NPU AI engines. The memory can also be carved up to a distinct…

"Fire breathing" is completely inappropriate.

Strix Halo is a replacement for the high-power laptop CPUs from the HX series of Intel and AMD, together with a discrete GPU.

The thermal design power of a laptop CPU-dGPU combo is normally much higher than 120 W, which is the maximum TDP recommended for Strix Halo. The faster laptop dGPUs want more than 120 W only for themselves, not counting the CPU.

So any claims of being surprised that the TDP range for Strix Halo is 45 W to 120 W are weird, like the commenter has never seen a gaming laptop or a mobile workstation laptop.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#330

Post by AtlasBarfed »

rbanffy wrote: post #233
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

This is something every company should make sure they have: an onboarding path. Xeon Phi failed for a number of reasons, but one where it didn't need to fail was availability of software optimised for it. Now we have Xeons and EPYCs, and MI300C's with lots of efficient cores, but we could have been writing software tailored for those for 10 years now. Extracting performance from them would be a solved problem at this…

It really mystifies me that Intel AMD and other hardware companies obviously Nvidia in this case Don't either have a consortium or each have their own in-house Linux distribution with excellent support.

Windows has always been a barrier to hardware feature adoption to Intel. You had to wait 2 to 3 years, sometimes longer, for Windows to get around us providing hardware support.

Any OS optimizations in Windows you had to go through Microsoft. So say you added some instructions custom silicon or whatever to speed up Enterprise databases, provide high-speed networking that needed some special kernel features, etc, there was always Microsoft being in the way.

Not just in the drag the feet communication. Getting the tech people a line problem.

Microsoft will look at every single change. It did as to whether or not it would challenge their Monopoly whether or not it was in their business interest whether or not it kept you as the hardware and a subservient role.

Post reply on HN