Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

181–190 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#181

Post by moffkalast »

qwertox wrote: post #148
Karupan wrote: post #53

Earlier quoted context omitted.

Yep that's what I'm thinking as well. I was going to buy a 5090 mainly to play around with LLM code generation, but this is a worthy option for roughly the same price as building a new PC with a 5090.

It has 128 GB of unified RAM. It will not be as fast as the 32 GB VRAM of the 5090, but what gamer cards have always lacked was memory. Plus you have fast interconnects, if you want to stack them. I was somewhat attracted by the Jetson AGX Orin with 64 GB RAM, but this one is a no-brainer for me, as long as idle power is reasonable.

Having your main pc as an LLM rig also really sucks for multitasking, since if you want to keep a model loaded to use it when needed, it means you have zero resources left to do anything else. GPU memory maxed out, most of the RAM used. Having a dedicated machine even if it's slower is a lot more practical imo, since you can actually do other things while it generates instead of having to sit there and wait, not being able to do anything else.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#182

Post by [deleted] »

qwertox wrote: post #148
Karupan wrote: post #53

Earlier quoted context omitted.

Yep that's what I'm thinking as well. I was going to buy a 5090 mainly to play around with LLM code generation, but this is a worthy option for roughly the same price as building a new PC with a 5090.

It has 128 GB of unified RAM. It will not be as fast as the 32 GB VRAM of the 5090, but what gamer cards have always lacked was memory. Plus you have fast interconnects, if you want to stack them. I was somewhat attracted by the Jetson AGX Orin with 64 GB RAM, but this one is a no-brainer for me, as long as idle power is reasonable.

[deleted]

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#184

Post by moffkalast »

acchow wrote: post #98
Karupan wrote: post #65

Earlier quoted context omitted.

They are perfectly fine for certain people. I can run Qwen-2.5-coder 14B on my M2 Max MacBook Pro with 32gb at ~16 tok/sec. At least in my circle, people are budget conscious and would prefer using existing devices rather than pay for subscriptions where possible. And we know why they won't ship NVLink anymore on prosumer GPUs: they control almost the entire segment and why give more away for free? Good for the compa…

> I can run Qwen-2.5-coder 14B on my M2 Max MacBook Pro with 32gb at ~16 tok/sec. At least in my circle, people are budget conscious Qwen 2.5 32B on openrouter is $0.16/million output tokens. At your 16 tokens per second, 1 million tokens is 17 continuous hours of output. Openrouter will charge you 16 cents for that. I think you may want to reevaluate which is the real budget choice here Edit: elaborating, that extra…

It's a great option if you want to leak your entire internal codebase to 3rd parties.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#185

Post by smt88 »

dagmx wrote: post #20
VikingCoder wrote: post #15

Earlier quoted context omitted.

If I were NVidia, I would be throwing everything I could at making entertainment experiences that need one of these to run... I mean, this is awfully close to being "Her" in a box, right?

I feel like a lot of people miss that Her was a dystopian future, not an ideal to hit. Also, it’s $3000. For that you could buy subscriptions to OpenAI etc and have the dystopian partner everywhere you go.

If Silicon Valley could tell the difference between utopias and dystopias, we wouldn't have companies named Soylent or iRobot, and the recently announced Anduril/Palantir/OpenAI partnership to hasten the creation of either SkyNet or Big Brother wouldn't have happened at all.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#186

Post by motoxpro »

spaceman_2020 wrote: post #163
qwertox wrote: post #23

Earlier quoted context omitted.

You could have said the same about gamers buying expensive hardware in the 00's. It's what made Nvidia big.

I keep thinking about stocks that have 100xd, and most seemed like obscure names to me as a layman. But man, Nvidia was a household name to anyone that ever played any game. And still so many of us never bothered buying the stock Incredible fumble for me personally as an investor

Unless you predicted AI and Crypto then it was just really good, not 100x. It 20x from 2005-2020 but ~500x from 2005-2025

And if you truly did predict that Nvidia would own those markets and those markets would be massive, you could have also bought Amazon, Google or heck even Bitcoin. Anything you touched in tech really would have made you a millionaire really.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#187

Post by mrlongroots »

dagmx wrote: post #44
Karupan wrote: post #29

Earlier quoted context omitted.

I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs. Surely a smaller market than gamers or datacenters for sure.

I mean negligible to their bottom line. There may be tons of units bought or not, but the margin on a single datacenter system would buy tens of these. It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

> It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

It will be massive for research labs. Most academics have to jump through a lot of hoops to get to play with not just CUDA, but also GPUDirect/RDMA/Infiniband etc. If you get older/donated hardware, you may have a large cluster but not newer features.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#190

Post by tatersolid »

trhway wrote: post #97
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

>enthusiast AI dev segment i think it isn't about enthusiast. To me it looks like Huang/NVDA is pushing further a small revolution using the opening provided by the AI wave - up until now the GPU was add-on to the general computing core onto which that computing core offloaded some computing. With AI that offloaded computing becomes de-facto the main computing and Huang/NVDA is turning tables by making the CPU is jus…

> these databases can be executed on GPU with a significant performance gain vs. CPU

No, they can’t. GPU databases are niche products with severe limitations.

GPUs are fast at massively parallel math problems, they anren’t useful for all tasks.

Post reply on HN