Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

Post reply on HN
Print view View original

471–480 of 510 posts

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#471

Post by thunkshift1 »

yoyohello13 wrote: post #304
neom wrote: post #276

In case you're curious, I googled. It runs this thing called "DGX OS": "DGX OS 6 Features The following are the key features of DGX OS Release 6: Based on Ubuntu 22.04 with the latest long-term Linux kernel version 5.15 for the recent hardware and security updates and updates to software packages, such as Python and GCC. Includes the NVIDIA-optimized Linux kernel, which supports GPU Direct Storage (GDS) without addit…

I wonder what kind of spyware is loaded onto DGX OS. Oh, sorry I mean telemetry.

Correct, highly concerning.. this is totally not the case with existing os’s and products

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#472

Post by croes »

VikingCoder wrote: post #466
croes wrote: post #419

Earlier quoted context omitted.

3 billion revenue and 5 billion loss doesn’t sound like a sustainable business model.

The real question is what the next 3 years look like. If it's another 5 billion burned for 3 billion or less in revenue, that's one thing... But...

How...

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#473

Post by rs38 »

kcb wrote: post #80
trhway wrote: post #62

$3000? The GB10 inside it seems to be a half of GB200 which is like $60K. One can wonder about availability at those $3K.

No way is the GPU half a GB200. I'd expect something much lower end and power conscious. They mention 1 PFLOP for FP4, GB200 is 40 PFLOP.

at least the naming implies factor ~20 less :)

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#474

Post by computably »

kcb wrote: post #86
computably wrote: post #8

Earlier quoted context omitted.

From the size and pricing ($3000) alone, it's safe to conclude it has less raw FLOPs than a 5090. Since it uses LPDDR5X, almost certainly less memory bandwidth too (5090 @ 1.8 TB/s, M4 Max w/ 128GB LPDDR5X @ 546 GB/s). Basically the only advantage is how much VRAM it packs in a small form factor, and presumably greater power efficiency at its smaller scale. The only thing it really competes with is the Mac Studio for…

Making comparisons to the 5090 is silly. That thing draws 500W+ and will require a boat anchor of metal to keep it cool. The device they showed is something more along the lines of a mobile dev kit.

I agree they're not products that compete against each other. Unfortunately, the silly comparison has to be made, as less informed consumers are already claiming that the 128 GB RAM of Project Digits will obsolete workstation/server-class GPUs.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#475

Post by maniroo »

csomar wrote: post #109
Karupan wrote: post #6

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

Am I the only one disappointed by these? They cost roughly half the price of a macbook pro and offer hmm.. half the capacity in RAM. Sure speed matters in AI, but what do I do with speed when I can't load a 70b model. On the other hand, with a $5000 macbook pro, I can easily load a 70b model and have a "full" macbook pro as a plus. I am not sure I fully understand the value of these cards for someone that want to run…

Bro we can connect two ProjectDigits as well. I was only looking at the M4 macbook because 128gb unified memory. Now this beast can cook better LLMs at just 3K with 4TB SSD too. M4 Macbook Max (128 GB unified ram and 4TB Storage) is 5999. So, No more apple for me. I will just get the Digits. And can create a workstation as well.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#476

Post by computably »

llm_nerd wrote: post #241
computably wrote: post #8

Earlier quoted context omitted.

From the size and pricing ($3000) alone, it's safe to conclude it has less raw FLOPs than a 5090. Since it uses LPDDR5X, almost certainly less memory bandwidth too (5090 @ 1.8 TB/s, M4 Max w/ 128GB LPDDR5X @ 546 GB/s). Basically the only advantage is how much VRAM it packs in a small form factor, and presumably greater power efficiency at its smaller scale. The only thing it really competes with is the Mac Studio for…

The product isn't even finalized. It might never come to fruition, and I cannot fathom how they will make the power profile fit. I am skeptical that a $3000 device with 128GB of RAM and a 4TB SSD with the specs provided will even see reality any time within the next year, but let's pretend it will. However we do know that it offers 1/4 the TOPS of the new 5090. It will be less powerful than the $600 5070. Which, of c…

AI TOPS numbers for Blackwell/ 5090 are probably for a niche numeric type like INT8 or INT4.

At FP32 (and FP16, assuming the consumer cards are still neutered), the 5090 apparently does ~105-107 TFLOPS, and the full GB202 ~125 TFLOPS. That means a non-neutered GB202-based card could hit ~250 TFLOPS of FP16, which lines up neatly with 1 PFLOP of FP4.

In reality, FP4 is more-than-linearly efficient relative to FP32. They quoted FP4 and not FP8 / FP16 for a reason. I wouldn't be too surprised if it doesn't even support FP32, maybe even FP16. Plus, they likely cut RT cores and other graphics-related features, making for a smaller and therefore more power efficient chip, because they're positioning this as an "AI supercomputer" and this hardware doesn't make sense for most graphical applications.

I see no reason this product wouldn't come to market - besides the usual supply/demand. There's value for a small niche and particular price bracket: enthusiasts running large q4 models, cheaper but slower vs. dedicated cards (3x-10x price/VRAM) and price-competitive but much faster vs. Apple silicon. It's a good strategic move for maintaining Nvidia's hold on the ecosystem regardless of the sales revenue.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#477

Post by adrian_b »

bmicraft wrote: post #460
adrian_b wrote: post #329

Earlier quoted context omitted.

"Fire breathing" is completely inappropriate. Strix Halo is a replacement for the high-power laptop CPUs from the HX series of Intel and AMD, together with a discrete GPU. The thermal design power of a laptop CPU-dGPU combo is normally much higher than 120 W, which is the maximum TDP recommended for Strix Halo. The faster laptop dGPUs want more than 120 W only for themselves, not counting the CPU. So any claims of be…

> The thermal design power of a laptop CPU-dGPU combo is normally much higher than 120 W Normally? Much higher than 120W? Those are some pretty abnormal (and dare I say niche?) laptops you're talking about there. Remember, that's not peak power - thermal design power is what the laptop should be able to power and cool pretty much continuously. At those power levels, they're usually called DTR: desktop replacement. Yo…

Any laptop that in marketed as "gaming laptop" or "mobile workstation" belongs to this category.

I do not know which is the proportion of gaming laptops and mobile workstations vs. thin and light laptops. While obviously there must be much more light laptops, the gaming laptops cannot be a niche product, because there are too many models offered by a lot of vendors.

My own laptop is a Dell Precision, so it belongs to this class. I would not call Dell Precision laptops as a niche product, even if they are typically used only by professionals.

My previous laptop was some Lenovo Yoga that also belonged to this class, having a discrete NVIDIA GPU. In general, any laptop having a discrete GPU belongs to this class, because the laptop CPUs intended to be paired with discrete GPUs have a default TDP of 45 W or 55 W, while the smallest laptop discrete GPUs may have TDPs of 55 W to 75 W, but the faster laptop GPUs have TDPs between 100 W and 150 W, so the combo with CPU reaches a TDP around 200 W for the biggest laptops.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#478

Post by menaerus »

croes wrote: post #472
VikingCoder wrote: post #466

Earlier quoted context omitted.

The real question is what the next 3 years look like. If it's another 5 billion burned for 3 billion or less in revenue, that's one thing... But...

How...

Recent report says there are 1M paying customers. At ~30USD for 12 months this is ~3.6B of revenue which kinda matches their reported figures. So to break even at their ~5B costs assuming that they need no further major investment in infrastructure they only need to increase the paying subscriptions from 1M to 2M. Since there are ~250M people who engaged with OpenAI free tier service 2x projection doesn't sound too surreal.

Re: Nvidia's Project Digits is a 'personal AI supercomputer'

#480

Post by menaerus »

trhway wrote: post #208
tatersolid wrote: post #190

Earlier quoted context omitted.

> these databases can be executed on GPU with a significant performance gain vs. CPU No, they can’t. GPU databases are niche products with severe limitations. GPUs are fast at massively parallel math problems, they anren’t useful for all tasks.

>GPU databases are niche products with severe limitations. today. For the reasons like i mentioned. >GPUs are fast at massively parallel math problems, they anren’t useful for all tasks. GPU are fast at massively parallel tasks. Their memory bandwidth is 10x of that of the CPU for example. So, typical database operations, massively parallel in nature like join or filter, would run about that faster. Majority of compu…

> So, typical database operations, massively parallel in nature like join or filter, would run about that faster.

Given workload A how much of the total runtime JOIN or FILTER would take in contrast to the storage engine layer for example? My gut feeling tells me not much since to see the actual gain you'd need to be able to parallelize everything including the storage engine challenges.

IIRC all the startups building databases around GPUs failed to deliver in the last ~10 years. All of them are shut down if I am not mistaken.

Post reply on HN