Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia sells tiny new computer that puts big AI on your desktop

arstechnica.com

Post reply on HN
Print view View original

1–10 of 14 posts

Re: Nvidia sells tiny new computer that puts big AI on your desktop

#4

Post by andsoitis »

turbocon wrote: post #3

I want to know if this is any different than all of the AMD AI Max PCs with 128gb of unified memory? The spec sheet say "128 GB LPDDR5x", so how is this better? https://nvdam.widen.net/s/tlzm8smqjx/workstation-datasheet-d...

> AMD AI Max PCs with 128gb of unified memory? The spec sheet say "128 GB LPDDR5x", so how is this better?

Framework's AMD AI Max PCs also come with LPDDR5x-8000 memory: https://frame.work/desktop?tab=specs

Re: Nvidia sells tiny new computer that puts big AI on your desktop

#5

Post by Numerlor »

turbocon wrote: post #3

I want to know if this is any different than all of the AMD AI Max PCs with 128gb of unified memory? The spec sheet say "128 GB LPDDR5x", so how is this better? https://nvdam.widen.net/s/tlzm8smqjx/workstation-datasheet-d...

The GPU is significantly faster and it has cuda, though I'm not sure where it'd fit in the market.

At the lower price points you have the AMD machines which are significantly cheaper, even though they're slower and with worse support. Then there's apple's with higher memory bandwidth and even the nvidia agx Thor is faster in GPU compute at the cost of worse CPU and networking, and at the 3-4K price point even a threadripper system becomes viable that can get significantly more memory

Re: Nvidia sells tiny new computer that puts big AI on your desktop

#8

Post by BoredPositron »

turbocon wrote: post #3

I want to know if this is any different than all of the AMD AI Max PCs with 128gb of unified memory? The spec sheet say "128 GB LPDDR5x", so how is this better? https://nvdam.widen.net/s/tlzm8smqjx/workstation-datasheet-d...

CUDA.

Re: Nvidia sells tiny new computer that puts big AI on your desktop

#9

Post by antinomicus »

adam_patarino wrote: post #2

If you would buy this I’d love to know how you’d use it.

Though the adage “this is the worst it’ll ever be” is parroted daily by AI cultists, the fact is it’s still yet to be proven that currently available LLMs can be made cost effective. For now every ai company is lighting tens of billions of dollars on fire every year and hoping better algorithms, hardware, and user lock in will ensure profits eventually. If this doesn’t happen, they will design more and more “features” in the LLM to monetize it - shopping, ads, sponsored replies, who knows? It may get really awful. And these companies will have so much of our data and eventually the need to make profits will lead them to sell that data and just generally try to extract as much out of us as they can.

This is why in the long run I believe we all should aspire to do LLM inference locally. But unfortunately we just are not anywhere close to par with the SoTA cloud models available. Something like DGX spark would be a decent step in this direction, but this platform appears to mostly be for prototyping / training models meant to eventually be run on data center nvidia hardware.

Personally I think I will probably spec out an M5 max/ultra Mac Studio once that’s a thing, and start trying to do this more seriously. The tools are getting better every day and “this is the worst it’ll ever be” is much more applicable to locally run models.

Post reply on HN