[deleted]
Nvidia's Project Digits is a 'personal AI supercomputer'
221–230 of 510 posts
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#222Earlier quoted context omitted.
I think the enthusiast side of things is a negligible part of the market. That said, enthusiasts do help drive a lot of the improvements to the tech stack so if they start using this, it’ll entrench NVIDIA even more.
I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs. Surely a smaller market than gamers or datacenters for sure.
Yes, but people already had their Macs for others reasons.
No one goes to an Apple store thinking "I'll get a laptop to do AI inference".
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#223Anyone know if it can run training/fine-tuning and not just 4-bit inference? Does it support mixed precision training with either BF16 or FP16?
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#224I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
I can confirm this is the case (for me).
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#225I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
I would like to have Mac as my personal computer and digits as service to run llm.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#226$3000 seems incredibly good value
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#227I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
The developers they are referring to aren’t just enthusiasts; they are also developers who were purchasing SuperMicro and Lambda PCs to develop models for their employers. Many enterprises will buy these for local development because it frees up the highly expensive enterprise-level chip for commercial use. This is a genius move. I am more baffled by the insane form factor that can pack this much power inside a Mac M…
How does it run 400B models across two? I didn’t see that in the article
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#228Earlier quoted context omitted.
The developers they are referring to aren’t just enthusiasts; they are also developers who were purchasing SuperMicro and Lambda PCs to develop models for their employers. Many enterprises will buy these for local development because it frees up the highly expensive enterprise-level chip for commercial use. This is a genius move. I am more baffled by the insane form factor that can pack this much power inside a Mac M…
How does it run 400B models across two? I didn’t see that in the article
> Nvidia says that two Project Digits machines can be linked together to run up to 405-billion-parameter models, if a job calls for it. Project Digits can deliver a standalone experience, as alluded to earlier, or connect to a primary Windows or Mac PC.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#229I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
From the people I talk to the enthusiast market is nvidia 4090/3090 saturated because people want to do their fine tunes also porn on their off time. The Venn diagram of users who post about diffusion models and llms running at home is pretty much a circle.
Yeah, I really don't think the overlap is as much as you imagine. At least in /r/localllama and the discord servers I frequent, the vast majority of users are interested in one or the other primarily, and may just dabble with other things. Obviously this is just my observations...I could be totally misreading things.
Re: Nvidia's Project Digits is a 'personal AI supercomputer'
#230I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
> they seem to be doing everything right in the last few years
About that... Not like there isn't a lot to be desired from the linux drivers: I'm running a K80 and M40 in a workstation at home and the thought of having to ever touch the drivers, now that the system is operational, terrifies me. It is by far the biggest "don't fix it if it ain't broke" thing in my life.