I am pretty sure memory bandwidth will be low it doesn't eat up their enterprise lineup. If we are luck we might get 512GB/S this is still half of 4090
Nvidia's Project Digits is a 'personal AI supercomputer'
401–410 of 510 posts
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#402Earlier quoted context omitted.
I'm pretty frugal, but my first thought is to get two to run 405B models. Building out 128GB of VRAM isn't easy, and will likely cost twice this.
You can get a M4 Max MBP with 128GB for $1k less than two of these single-use devices.
But then you have to use macOS.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#403The monitor is AI generated in the product photo.... Nvidia please
https://s3.amazonaws.com/cms.ipressroom.com/219/files/20250/...
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#404Earlier quoted context omitted.
Now that the majority of their revenue is from data centers instead of Windows gaming PCs, you'd think their relationship with Linux should improve or already has.
It's possible. I haven't had a system completely destroyed by Nvidia in the last few years, but I've been assuming that's because I've gotten in the habit of just not touching it once I get it working...
I have been having a fine time with a 3080 on recent Arch, FWIW.
HDR support is still painful, but that seems to be a Linux problem, not specific to Nvidia.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#405Earlier quoted context omitted.
My Jetson TX2 developer kit didn't stop working, but it's on a very out of date Linux distribution. Maybe if Nvidia makes it to four trillion in market cap they'll have enough spare change to keep these older boards properly supported, or at least upstream all the needed support.
Are you aware that mainline linux runs on these Jetson devices? It's a bit of annoying work, but you can be running ArchLinuxARM. https://github.com/archlinuxarm/PKGBUILDs/pull/1580 Edit: It's been a while since I did this, but I had to manually build the kernel, overwrite a dtb file maybe (and Linux_for_Tegra/bootloader/l4t_initrd.img) and run something like this (for xavier) sudo ./flash.sh -N 128.30.84.100:/srv/ar…
How close does any of that get a person to having Ubuntu 24.04 running on their board?
(I guess we can put aside the issue of Nvidia's closed source graphics drivers for the moment)
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#406I'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years. Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch et…
This is a very important point. In general, Nvidia's relationship with Linux has been... complicated. On the one hand, at least they offer drivers for it. On the other, I have found few more reliable ways to irreparably break a Linux installation than trying to install or upgrade those drivers. They don't seem to prioritize it as a first class citizen, more just tolerate it the bare minimum required to claim it works…
The Digits device runs the same nVidia DGX OS (nVidia custom Ubuntu distro) that they run on their cloud infra.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#407Earlier quoted context omitted.
1000% all these ai hardware companies will fail if they don't have this. You must have a cheap way to experiment and develop. Even if you want to only sell a $30000 datacenter card you still need a very low cost way to play. Sad to see big companies like intel and amd don't understand this but they've never come to terms with the fact that software killed the hardware star
> Sad to see big companies like intel and amd don't understand this And it's not like they were never bitten (Intel has) by this before.
Intel does have https://www.clearlinux.org/
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#408Earlier quoted context omitted.
The developers they are referring to aren’t just enthusiasts; they are also developers who were purchasing SuperMicro and Lambda PCs to develop models for their employers. Many enterprises will buy these for local development because it frees up the highly expensive enterprise-level chip for commercial use. This is a genius move. I am more baffled by the insane form factor that can pack this much power inside a Mac M…
How does it run 400B models across two? I didn’t see that in the article
Point to point ConnectX connection (RDMA with GPUDirect)
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#409Earlier quoted context omitted.
It's annoying I do LLMs for work and have a bit of an interest in them and doing stuff with GANS etc. I have a bit of an interest in games too. If I could get one platform for both, I could justify 2k maybe a bit more. I can't justify that for just one half: running games on Mac, right now via Linux: no thanks. And on the PC side, nvidia consumer cards only go to 24gb which is a bit limiting for LLMs, while being ver…
The new $2k card from Nvidia will be 32GB but your point stands. AMD is planning a unified chiplet based GPU architecture (AI/data center/workstation/gaming) called UDNA, which might alleviate some of these issues. It's been delayed and delayed though - hence the lackluster GPU offerings from team Red this cycle - so I haven't been getting my hopes up. Maybe (LP)CAMM2 memory will make model usage just cheap enough th…
Unified architecture is still on track for 2026-ish.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#410Earlier quoted context omitted.
They have, because until now Apple Silicon was the only practical way for many to work with larger models at home because they can be configured with 64-192GB of unified memory. Even the laptops can be configured with up to 128GB of unified memory. Performance is not amazing (roughly 4060 level, I think?) but in many ways it was the only game in town unless you were willing and able to build a multi-3090/4090 rig.
I would bet that people running LLMs on their Macs, today, is <0.1% of their user base.
Higher than that buying the top end machines though, which are very high margin