Interesting, but perhaps not surprising, that the OS is Ubuntu-based, with Nvidia software preinstalled.
Given that it runs on ARM chips and is specifically designed for AI tasks, I would be more surprised to see it running Windows by default
Nvidia sells tiny new computer that puts big AI on your desktop
11–14 of 14 posts
Re: Nvidia sells tiny new computer that puts big AI on your desktop
#12If you would buy this I’d love to know how you’d use it.
I would use it for locally hosted RAG or whatever tech has supplanted it instead of paying API fees. We have ~20TB of documents that occasionally need to be scanned and chatted with and $4,000 one time (+ electricity) is chump change compared to the annual costs we would otherwise be looking at.
Re: Nvidia sells tiny new computer that puts big AI on your desktop
#13I want to know if this is any different than all of the AMD AI Max PCs with 128gb of unified memory? The spec sheet say "128 GB LPDDR5x", so how is this better? https://nvdam.widen.net/s/tlzm8smqjx/workstation-datasheet-d...
The GPU is significantly faster and it has cuda, though I'm not sure where it'd fit in the market. At the lower price points you have the AMD machines which are significantly cheaper, even though they're slower and with worse support. Then there's apple's with higher memory bandwidth and even the nvidia agx Thor is faster in GPU compute at the cost of worse CPU and networking, and at the 3-4K price point even a threa…
> The GPU is significantly faster and it has cuda,
But (non-batched) LLM processing is usually limited by memory bandwidth, isn't it? Any extra speed the GPU has is not used by current-day LLM inference.
Re: Nvidia sells tiny new computer that puts big AI on your desktop
#14Earlier quoted context omitted.
The GPU is significantly faster and it has cuda, though I'm not sure where it'd fit in the market. At the lower price points you have the AMD machines which are significantly cheaper, even though they're slower and with worse support. Then there's apple's with higher memory bandwidth and even the nvidia agx Thor is faster in GPU compute at the cost of worse CPU and networking, and at the 3-4K price point even a threa…
> The GPU is significantly faster and it has cuda, But (non-batched) LLM processing is usually limited by memory bandwidth, isn't it? Any extra speed the GPU has is not used by current-day LLM inference.
I believe just inference is bandwidth limited, prompt processing and other tasks on the other hand needs the compute. As I understand it, the workstation is also as a whole focused on the local development process before readying things for the datacenters, not just running LLMs