Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

361–370 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#361

Post by tempoponet »

tcdent wrote: post #350
Abishek_Muthian wrote: post #283

I'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years. Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch et…

If you're expecting this device to stay relevant for 4 years you are not the target demographic. Compute is evolving way too rapidly to be setting-and-forgetting anything at the moment.

Today I'm using 2x 3090's which are over 4 years old at this point and still very usable. To get 48gb vram I would need 3x 5070ti - still over $2k.

In 4 years, you'll be able to combine 2 of these to get 256gb unified memory. I expect that to have many uses and still be in a favorable form factor and price.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#362

Post by rsanek »

ynniv wrote: post #309
lolinder wrote: post #244

Earlier quoted context omitted.

Heck, I'm willing to pay $3000 for one of these to get a good model that runs my requests locally. It's probably just my stupid ape brain trying to do finance, but I'm infinitely more likely to run dumb experiments with LLMs on hardware I own than I am while paying per token (to the point where I currently spend way more time with small local llamas than with Claude), and even though I don't do anything sensitive I'm…

I'm pretty frugal, but my first thought is to get two to run 405B models. Building out 128GB of VRAM isn't easy, and will likely cost twice this.

You can get a M4 Max MBP with 128GB for $1k less than two of these single-use devices.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#363

Post by p_ing »

AtlasBarfed wrote: post #330
rbanffy wrote: post #233

Earlier quoted context omitted.

This is something every company should make sure they have: an onboarding path. Xeon Phi failed for a number of reasons, but one where it didn't need to fail was availability of software optimised for it. Now we have Xeons and EPYCs, and MI300C's with lots of efficient cores, but we could have been writing software tailored for those for 10 years now. Extracting performance from them would be a solved problem at this…

It really mystifies me that Intel AMD and other hardware companies obviously Nvidia in this case Don't either have a consortium or each have their own in-house Linux distribution with excellent support. Windows has always been a barrier to hardware feature adoption to Intel. You had to wait 2 to 3 years, sometimes longer, for Windows to get around us providing hardware support. Any OS optimizations in Windows you had…

From the consumer perspective, it seems that MSFT has provided scheduler changes fairly rapidly for CPU changes, like X3D, P/e cores, etc. At least within a couple of months, if not at release.

Amd/Intel work directly with Microsoft for shipping new silicon that would otherwise require it.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#364

Post by p_ing »

rbanffy wrote: post #233
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

This is something every company should make sure they have: an onboarding path. Xeon Phi failed for a number of reasons, but one where it didn't need to fail was availability of software optimised for it. Now we have Xeons and EPYCs, and MI300C's with lots of efficient cores, but we could have been writing software tailored for those for 10 years now. Extracting performance from them would be a solved problem at this…

Raptor Computing provides POWER9 workstations. They're not cheap, still use last-gen hardware (DDR4/PCIe 4 ... and POWER9 itself) but they're out there.

https://www.raptorcs.com/content/base/products.html

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#365

Post by nimish »

rbanffy wrote: post #300
nimish wrote: post #264

Earlier quoted context omitted.

1000% all these ai hardware companies will fail if they don't have this. You must have a cheap way to experiment and develop. Even if you want to only sell a $30000 datacenter card you still need a very low cost way to play. Sad to see big companies like intel and amd don't understand this but they've never come to terms with the fact that software killed the hardware star

> Sad to see big companies like intel and amd don't understand this And it's not like they were never bitten (Intel has) by this before.

Well, Intel management is very good at snatching defeat from the jaws of victory

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#366

Post by aseipp »

Abishek_Muthian wrote: post #283

I'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years. Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch et…

The Orin series and later use UEFI and you can apparently run upstream, non-GPU enabled kernels on them. There's a user guide page documenting it. So I think it's gotten a lot better, but it's sort of moot because the non-GPU thing is because the JetPack Linux fork has a specific 'nvgpu' driver used for Tegra devices that hasn't been unforked from that tree. So, you can buy better alternatives unless you're explicitly doing the robotics+AI inference edge stuff.

But the impression I get from this device is that it's closer in spirit to the Grace Hopper/datacenter designs than it is the Tegra designs, due to both the naming, design (DGX style) and the software (DGX OS?) which goes on their workstation/server designs. They are also UEFI, and in those scenarios, you can (I believe?) use the upstream Linux kernel with the open source nvidia driver using whatever distro you like. In that case, this would be a much more "familiar" machine with a much more ordinary Linux experience. But who knows. Maybe GH200/GB200 need custom patches, too.

Time will tell, but if this is a good GPU paired with a good ARM Cortex design, and it works more like a traditional Linux box than the Jeton series, it may be a great local AI inference machine.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#367

Post by tcdent »

mrybczyn wrote: post #355
tcdent wrote: post #350

Earlier quoted context omitted.

If you're expecting this device to stay relevant for 4 years you are not the target demographic. Compute is evolving way too rapidly to be setting-and-forgetting anything at the moment.

Eh? By all indications compute is now evolving SLOWER than ever. Moore's Law is dead, Dennard scaling is over, the latest fab nodes are evolutionary rather than revolutionary. This isn't the 80s when compute doubled every 9 months, mostly on clock scaling.

Fab node size is not the only factor in performance. Physical limits were reached, and we're pulling back from the extremely small stuff for the time being. That is the evolutionary part.

Revolutionary developments are: multi-layer wafer bonding, chiplets (collections of interconnected wafers) and backside power delivery. We don't need the transistors to keep getting physically smaller, we need more of them, and at increased efficiency, and that's exactly what's happening.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#369

Post by adolph »

AnonymousPlanet wrote: post #234
swat535 wrote: post #217

Earlier quoted context omitted.

Boring fact: The underlying theme of the movie Her is actually divorce and the destructive impact it has on people, the futuristic AI stuff is just for stuffing!

The overall theme of Her was human relationships. It was not about AI and not just about divorce in particular.The AI was just a plot device to include a bodyless person into the equation. Watch it again with this in mind and you will see what I mean.

The universal theme of Her was the set of harmonics that define what is something and the thresholds, boundaries, windows onto what is not thatthing but someotherthing, even if the thing perceived is a mirror, not just about human relationships in particular. The relationship was just a plot device to make a work of deep philosophy into a marketable romantic comedy.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#370

Post by sroussey »

moralestapia wrote: post #327
JohnBooty wrote: post #239

Earlier quoted context omitted.

They have, because until now Apple Silicon was the only practical way for many to work with larger models at home because they can be configured with 64-192GB of unified memory. Even the laptops can be configured with up to 128GB of unified memory. Performance is not amazing (roughly 4060 level, I think?) but in many ways it was the only game in town unless you were willing and able to build a multi-3090/4090 rig.

I would bet that people running LLMs on their Macs, today, is <0.1% of their user base.

People buying Macs for LLMs—sure I agree.

Since the current MacOS comes built in with small LLMs, that number might be closer to 50% not 0.1%.

Post reply on HN