Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

231–240 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#231

Post by the_other »

moralestapia wrote: post #222
Karupan wrote: post #29

Earlier quoted context omitted.

I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs. Surely a smaller market than gamers or datacenters for sure.

Yes, but people already had their Macs for others reasons. No one goes to an Apple store thinking "I'll get a laptop to do AI inference".

I'm currently wondering how likely it is I'll get into deeper LLM usage, and therefore how much Apple Silicon I need (because I'm addicted to macOS). So I'm some way closer to your steel man than you'd expect. But I'm probably a niche within a niche.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#232

Post by delegate »

I think this is version 1 of what's going to become the new 'PC'.

Future versions will get more capable and smaller, portable.

Can be used to train new types models (not just LLMs).

I assume the GPU can do 3D graphics.

Several of these in a cluster could run multiple powerful models in real time (vision, llm, OCR, 3D navigation, etc).

If successful, millions of such units will be distributed around the world within 1-2 years.

A p2p network of millions of such devices would be a very powerful thing indeed.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#233

Post by rbanffy »

Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

This is something every company should make sure they have: an onboarding path.

Xeon Phi failed for a number of reasons, but one where it didn't need to fail was availability of software optimised for it. Now we have Xeons and EPYCs, and MI300C's with lots of efficient cores, but we could have been writing software tailored for those for 10 years now. Extracting performance from them would be a solved problem at this point. The same applies for Itanium - the very first thing Intel should have made sure it had was good Linux support. They could have it before the first silicon was released. Itaium was well supported for a while, but it's long dead by now.

Similarly, Sun has failed with SPARC, which also didn't have an easy onboarding path after they gave up on workstations. They did some things right: OpenSolaris ensured the OS remained relevant (still is, even if a bit niche), and looking the other way for x86 Solaris helps people to learn and train on it. Oracle cloud could, at least, offer it on cloud instances. Would be nice.

Now we see IBM doing the same - there is no reasonable entry level POWER machine that can compete in performance with a workstation-class x86. There is a small half-rack machine that can be mounted on a deskside case, and that's it. I don't know of any company that's planning to deploy new systems on AIX (much less IBMi, which is also POWER), or even for Linux on POWER, because it's just too easy to build it on other, competing platforms. You can get AIX, IBMi and even IBMz cloud instances from IBM cloud, but it's not easy (and I never found a "from-zero-to-ssh-or-5250-or-3270" tutorial for them). I wonder if it's even possible. You can get Linux on Z instances, but there doesn't seem to be a way to get Linux on POWER. At least not from them (several HPC research labs still offer those).

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#234

Post by AnonymousPlanet »

swat535 wrote: post #217
nostromo wrote: post #128

Earlier quoted context omitted.

Fun fact: Her was set in the year 2025.

Boring fact: The underlying theme of the movie Her is actually divorce and the destructive impact it has on people, the futuristic AI stuff is just for stuffing!

The overall theme of Her was human relationships. It was not about AI and not just about divorce in particular.The AI was just a plot device to include a bodyless person into the equation. Watch it again with this in mind and you will see what I mean.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#235

Post by mycall »

delegate wrote: post #232

I think this is version 1 of what's going to become the new 'PC'. Future versions will get more capable and smaller, portable. Can be used to train new types models (not just LLMs). I assume the GPU can do 3D graphics. Several of these in a cluster could run multiple powerful models in real time (vision, llm, OCR, 3D navigation, etc). If successful, millions of such units will be distributed around the world within 1…

> A p2p network of millions of such devices would be a very powerful thing indeed.

If you think RAM speeds are slow for the transformer or inference, imagine what 100Mbs would be like.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#236

Post by ceejayoz »

otabdeveloper4 wrote: post #180
anonylizard wrote: post #91

Earlier quoted context omitted.

There's a titanic market with people wanting some uncensored local LLM/image/video generation model. This market extremely overlaps with gamers today, but will grow exponentially every year.

> There's a titanic market with people wanting some uncensored local LLM/image/video generation model. No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast.

AI porn is currently cringe, just like Eliza for conversations was cringe.

The cutting edge will advance, and convincing bespoke porn of people's crushes/coworkers/bosses/enemies/toddlers will become a thing. With all the mayhem that results.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#237

Post by mycall »

axegon_ wrote: post #230
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

> they seem to be doing everything right in the last few years About that... Not like there isn't a lot to be desired from the linux drivers: I'm running a K80 and M40 in a workstation at home and the thought of having to ever touch the drivers, now that the system is operational, terrifies me. It is by far the biggest "don't fix it if it ain't broke" thing in my life.

Buy a second system which you can touch?

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#238

Post by ben_w »

mycall wrote: post #235
delegate wrote: post #232

I think this is version 1 of what's going to become the new 'PC'. Future versions will get more capable and smaller, portable. Can be used to train new types models (not just LLMs). I assume the GPU can do 3D graphics. Several of these in a cluster could run multiple powerful models in real time (vision, llm, OCR, 3D navigation, etc). If successful, millions of such units will be distributed around the world within 1…

> A p2p network of millions of such devices would be a very powerful thing indeed. If you think RAM speeds are slow for the transformer or inference, imagine what 100Mbs would be like.

Depends on the details, as always.

If this hypothetical future is one where mixtures of experts is predominant, where each expert fits on a node, then the nodes only need the bandwidth to accept inputs and give responses — they won't need the much higher bandwidth required to spread a single model over the planet.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#239

Post by JohnBooty »

moralestapia wrote: post #222
Karupan wrote: post #29

Earlier quoted context omitted.

I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs. Surely a smaller market than gamers or datacenters for sure.

Yes, but people already had their Macs for others reasons. No one goes to an Apple store thinking "I'll get a laptop to do AI inference".

They have, because until now Apple Silicon was the only practical way for many to work with larger models at home because they can be configured with 64-192GB of unified memory. Even the laptops can be configured with up to 128GB of unified memory.

Performance is not amazing (roughly 4060 level, I think?) but in many ways it was the only game in town unless you were willing and able to build a multi-3090/4090 rig.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#240

Post by adrian_b »

skavi wrote: post #57
sliken wrote: post #36

Earlier quoted context omitted.

Not sure how to judge better price/perf. I wouldn't expect 20 Neoverse N2 cores to do particularly well vs 16 zen5 cores. The GPU side looks promising, but they aren't mentioning memory bandwidth, configuration, spec, or performance. Did see vague claims of "starting at $3k", max 4TB nvme, and max 128GB ram. I'd expect AMD Strix Halo (AI Max plus 395) to be reasonably competitive.

It’s actually “10 Arm Cortex-X925 and 10 Cortex-A725” [0]. These are much newer cores and have a reasonable chance of being competitive. [0]: https://newsroom.arm.com/blog/arm-nvidia-project-digits-high...

For programs dominated by iterations over arrays, these 10 Arm Cortex-X925 + 10 Cortex-A725, all 20 together, should have a throughput similar with only 10 of the 16 cores of Strix Halo (assuming that Strix Halo has full Zen 5 cores, which has not been confirmed yet).

For programs dominated by irregular integer and pointer operations, like software project compilation, 10 Arm Cortex-X925 + 10 Cortex-A725 should have a similar throughput with a 16-core Strix Halo, but which is faster would depend on cooling (i.e. a Strix Halo configured for a high power consumption will be faster).

There is not enough information to compare the performance of the GPUs from this NVIDIA Digits and from Strix Halo. However, it can be assumed that NVIDIA Digits will be better for ML/AI inference. Whether it can also be competitive for training or for graphics remains to be seen.

Post reply on HN