Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

111–120 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#111

Post by Karupan »

acchow wrote: post #98
Karupan wrote: post #65

Earlier quoted context omitted.

They are perfectly fine for certain people. I can run Qwen-2.5-coder 14B on my M2 Max MacBook Pro with 32gb at ~16 tok/sec. At least in my circle, people are budget conscious and would prefer using existing devices rather than pay for subscriptions where possible. And we know why they won't ship NVLink anymore on prosumer GPUs: they control almost the entire segment and why give more away for free? Good for the compa…

> I can run Qwen-2.5-coder 14B on my M2 Max MacBook Pro with 32gb at ~16 tok/sec. At least in my circle, people are budget conscious Qwen 2.5 32B on openrouter is $0.16/million output tokens. At your 16 tokens per second, 1 million tokens is 17 continuous hours of output. Openrouter will charge you 16 cents for that. I think you may want to reevaluate which is the real budget choice here Edit: elaborating, that extra…

It's a no brainer for me cause I already own the MacBook and I don't mind waiting a few extra seconds. Also, I didn't buy the mac for this purpose, it's just my daily device. So yes, I'm sure OpenRouter is cheaper, but I just don't have to think about using it as long as the open models are reasonable good for my use. Of course your needs may be quite different.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#112

Post by int_19h »

dagmx wrote: post #20
VikingCoder wrote: post #15

Earlier quoted context omitted.

If I were NVidia, I would be throwing everything I could at making entertainment experiences that need one of these to run... I mean, this is awfully close to being "Her" in a box, right?

I feel like a lot of people miss that Her was a dystopian future, not an ideal to hit. Also, it’s $3000. For that you could buy subscriptions to OpenAI etc and have the dystopian partner everywhere you go.

This is exactly the scenario where you don't want "the cloud" anywhere.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#113

Post by int_19h »

VikingCoder wrote: post #15
dagmx wrote: post #7

Earlier quoted context omitted.

I think the enthusiast side of things is a negligible part of the market. That said, enthusiasts do help drive a lot of the improvements to the tech stack so if they start using this, it’ll entrench NVIDIA even more.

If I were NVidia, I would be throwing everything I could at making entertainment experiences that need one of these to run... I mean, this is awfully close to being "Her" in a box, right?

The real interesting stuff will happen when we get multimodal LMs that can do VR output.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#114

Post by gardnr »

z4y5f3 wrote: post #50
sliken wrote: post #36

Earlier quoted context omitted.

Not sure how to judge better price/perf. I wouldn't expect 20 Neoverse N2 cores to do particularly well vs 16 zen5 cores. The GPU side looks promising, but they aren't mentioning memory bandwidth, configuration, spec, or performance. Did see vague claims of "starting at $3k", max 4TB nvme, and max 128GB ram. I'd expect AMD Strix Halo (AI Max plus 395) to be reasonably competitive.

NVIDIA is likely citing 1 PFlops at FP 4 sparse (they did this for GB200), so that is 128 TFlops BF16 dense, or 2/3 of what RTX 4090 is capable of. I would put the memory bandwidth at 546 GBps, using the same 512 bit LPDDR5X 8533 Mbps as Apple M4 max.

Based on your evaluation, it sounds like it will run inference at speed similar to an M4 Max and also allow "startups" to experiment with fine tuning larger models or larger context windows.

It's the best "dev board" setup I've seen so far. It might be part of their larger commercial plan but it definitely hits the sweet spot for the home enthusiast who have been pleading for more VRAM.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#115

Post by blurbleblurble »

csomar wrote: post #109
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

Am I the only one disappointed by these? They cost roughly half the price of a macbook pro and offer hmm.. half the capacity in RAM. Sure speed matters in AI, but what do I do with speed when I can't load a 70b model. On the other hand, with a $5000 macbook pro, I can easily load a 70b model and have a "full" macbook pro as a plus. I am not sure I fully understand the value of these cards for someone that want to run…

Then buy two and stack them!

Also I'm unfamiliar with macs is there really a MacBook pro with 256GB of RAM?

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#116

Post by sliken »

Topfi wrote: post #84
rvz wrote: post #18

Earlier quoted context omitted.

> Wow, it may actually be the year of Linux on the Desktop. ? Yeah starting at $3,000. Surely a cheap desktop computer to buy for someone who just wants to surf the web and send email /s. There is a reason why it is for "enthusiasts" and not for the general wider consumer or typical PC buyer.

I see the most direct competitor in the Mac Studio, though of course we will have to wait for reviews to gauge how fair that comparison is. The Studio does have a fairly large niche as a solid workstation, though, so I could see this being successful. For general desktop use, as you described, nearly any piece of modern hardware, from a RasPI, to most modern smartphones with a dock, could realistically serve most peo…

I did want a M2 max studio, ended up with a 12 core Zen 4 + radeon 7800 XT for about half the money.

A Nvidia Project Digit/GB10 for $3k with 128GB ram does sound tempting. Especially since it's very likely to have standard NVMe storage that I can expand or replace as needed, unlike the Apple solution. Decent linux support is welcome as well.

Here's hoping, if not I can fall back to a 128GB ram AMD Strix Halo/395 AI Max plus. CPU perf should be in the same ballpark, but not likely to come anywhere close on GPU performance, but still likely to have decent tokens/sec for casual home tinkering.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#117

Post by dagmx »

p_l wrote: post #107
sliken wrote: post #63

Earlier quoted context omitted.

The quotes I've seen mention the maximum config (128GB ram and 4TB of storage) and the minimum price. Nothing saying $3k for 128GB ram and 4TB of storage. I hope I'm wrong, but I'm betting the max price is at least twice the minimum price.

It's GB200 in desktop compatible enclosure, the RAM is fixed, the SSDs are not, the network ports are fixed too.

It’s most definitely not a GB200 in a desktop enclosure.

The processor is using completely different cores, and the GPU is somewhere around a 5070 for TOPs.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#118

Post by rictic »

csomar wrote: post #109
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

Am I the only one disappointed by these? They cost roughly half the price of a macbook pro and offer hmm.. half the capacity in RAM. Sure speed matters in AI, but what do I do with speed when I can't load a 70b model. On the other hand, with a $5000 macbook pro, I can easily load a 70b model and have a "full" macbook pro as a plus. I am not sure I fully understand the value of these cards for someone that want to run…

Hm? They have 128GB of RAM. Macbook Pros cap out at 128GB as well. Will be interesting to see how a Project Digits machine performs in terms of inference speed.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#119

Post by saagarjha »

YetAnotherNick wrote: post #70
sabareesh wrote: post #24

Earlier quoted context omitted.

what are current h100 price ? lowest i have seen is only 0.99 per hour

Where can you find 99c/hour? Cheapest I can find is double that.

Lambda has on-demand GH200 right now for $1.49. There might be a cheaper deal elsewhere for a contract.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#120

Post by sliken »

snerbles wrote: post #103
kcb wrote: post #89

Earlier quoted context omitted.

Doesn't really help though if it requires an nvidia kernel.

The official Linux kernel driver for Blackwell is GPL/MIT licensed: https://developer.nvidia.com/blog/nvidia-transitions-fully-t...

Keep in mind that a kernel module != driver. It's just doing initialization and passing data to/from the driver, which is closed source and in user space.
Post reply on HN