Would love to see something like this with an ATX form factor, socketed GPU, socketed GPU and non-soldered RAM.
Nvidia's Project Digits is a 'personal AI supercomputer'
71–80 of 510 posts
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#72Okay, so this is not a peripheral that you connect to your computer to run specialized tasks, this is a full computer running Linux. It's a garden hermit. Imagine a future where everyone has one of those(not exactly this version but some future version), it lives with you it learns with you and unlike the cloud based SaaS AI you can teach it things immediately and diverge from the average to your advantage.
"garden hermit" is a very interesting and evocative phrase. Where is that from?
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#73Earlier quoted context omitted.
I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs. Surely a smaller market than gamers or datacenters for sure.
I mean negligible to their bottom line. There may be tons of units bought or not, but the margin on a single datacenter system would buy tens of these. It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.
>It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.
100%
The people who prototype on a 3k workstation will also be the people who decide how to architect for a 3k GPU buildout for model training.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#74Okay, so this is not a peripheral that you connect to your computer to run specialized tasks, this is a full computer running Linux. It's a garden hermit. Imagine a future where everyone has one of those(not exactly this version but some future version), it lives with you it learns with you and unlike the cloud based SaaS AI you can teach it things immediately and diverge from the average to your advantage.
"garden hermit" is a very interesting and evocative phrase. Where is that from?
It's a real thing: https://en.wikipedia.org/wiki/Garden_hermit
In the past, in Europe, some wealthy people used to look after of a scholar living on their premises so they can ask them questions etc.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#75With more and more personal AI, i think having a truly private device that can run large LLMs (remember: larger is better) is fantastic!
Ideally we can configure things like Apple Intelligence to use this instead of OpenAI and Apple's cloud.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#76I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
I think the enthusiast side of things is a negligible part of the market. That said, enthusiasts do help drive a lot of the improvements to the tech stack so if they start using this, it’ll entrench NVIDIA even more.
AMD thought the enthusiast side of things was a negligible side of the market.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#77Earlier quoted context omitted.
From the size and pricing ($3000) alone, it's safe to conclude it has less raw FLOPs than a 5090. Since it uses LPDDR5X, almost certainly less memory bandwidth too (5090 @ 1.8 TB/s, M4 Max w/ 128GB LPDDR5X @ 546 GB/s). Basically the only advantage is how much VRAM it packs in a small form factor, and presumably greater power efficiency at its smaller scale. The only thing it really competes with is the Mac Studio for…
I believe $3,000 is for the unmentioned minimum config, no idea on the mentioned 4TB storage and 128GB ram version. Running a 96GB ram model isn't cheap (often with unified memory 25% is reserved for CPUs), so maybe it will win there.
The NVIDIA press release [0] says "Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage", which makes it sound like the RAM is fixed size.
[0] https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#78Earlier quoted context omitted.
AI work happens predominantly on the GPU, not the CPU. This GPU with CUDA will run rings around M4 with MLX. And with much more RAM than you can get in a Mac for $3k. A lot of people have been justifying their Mac Studio or Mac Pro purchases by the potential for running large AI models locally. Project Digits will be much better at that for cheaper. Maybe it won't run compile Chromium as fast, but that's not what it'…
The quotes I've seen mention the maximum config (128GB ram and 4TB of storage) and the minimum price. Nothing saying $3k for 128GB ram and 4TB of storage. I hope I'm wrong, but I'm betting the max price is at least twice the minimum price.
NVidia says 128GB ram at $3k[0], it looks like the 4TB storage might be variable (and possibly CPU or GPU cores?). This article says 128GB too.. but used up to twice in a row with different meanings which doesn't help.
[0]: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#79Little by little, we're getting an answer to the question: "What kind of investment does an outrageous influx of capitalization spur?" One might think it would be an AI silicon-moat, and it might yet be some of that.
But it's clear that everyone's favorite goal is keretsuification. If you're looking for abnormal profits, you can't do better than to add a letter to FAANG. Nvidia already got into the cloud business, and now it's making workstations.
The era of specialists doing specialist things is not really behind us. They're just not making automatic money, nor most of it. Nvidia excelled in that pool, but it too can't wait to leave it. It knows it can always fail as a specialist, but not as a kereitsu.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#80$3000? The GB10 inside it seems to be a half of GB200 which is like $60K. One can wonder about availability at those $3K.
No way is the GPU half a GB200. I'd expect something much lower end and power conscious.
They mention 1 PFLOP for FP4, GB200 is 40 PFLOP.