Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

61–70 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#61

Post by jqpabc123 »

dgellow wrote: post #54
jqpabc123 wrote: post #4

I am not sure how many people will run AI models locally. It still seems like a niche application to me. I'd say this relates directly to the cost of running AI models remotely. And we won't know what the actual cost will be until AI vendors recover the huge pile of cash they've dumped into development (plus interest).

Performances of local models are pretty bad compared to what AI vendors offer, token generation is just too slow to be that useful. And you need to allocate GBs of memories, something that will stay very expensive to buy for a long time. Running local models will stay niche for a while, unless we see breakthroughs

Dumb idea --- how about if we limit local models to specific domains --- medicine for example.

Most doctors don't care much about engineering or accounting or software development or 10000 other things that big vendor models address.

This area is yet to be really explored. Nvidia aims to provide the hardware to do so.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#62

Post by christkv »

adrian_b wrote: post #22
YasuoTanaka wrote: post #3

128GB of unified memory is a dream come true for local LLMs. VRAM has been the ultimate bottleneck for developers.

The competitor for this NVIDIA CPU will not be the now old AMD Strix Halo, but its successor (launched recently), which supports up to 192 GB of unified memory. Thus 128 GB is no longer SOTA. While this NVIDIA system is inferior from the point of view of the memory capacity, its main advantage is that the top models will have a bigger GPU, i.e. with 6144 or 5120 FP32 execution units, compared to 2560 for the AMD GPU…

I don’t think there is much improvement in compute for the new strix halo revision. The next one supposedly adds rdna4 cores or similar and more memory channels

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#63

Post by PedroBatista »

Don't want to be too harsh, maybe I'm missing something, but the CPU is at least 2 years old, internally it has been a complete shitshow and that's a minor hiccup when compared to the firmware and software situation.

It's an interesting "newcomer" and the more the better but calling this a "beast" and a "game changer" is ridiculous to say the least.

Then there is the price..

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#64

Post by dofm »

chpatrick wrote: post #14
jqpabc123 wrote: post #4

I am not sure how many people will run AI models locally. It still seems like a niche application to me. I'd say this relates directly to the cost of running AI models remotely. And we won't know what the actual cost will be until AI vendors recover the huge pile of cash they've dumped into development (plus interest).

I think it's niche now because getting the hardware to run it is expensive and the quantized models don't work as well. If those improve then it would be a no brainer to pay one off for the hardware instead of a fortune for API calls.

I am not really convinced that four bit quantisation is that bad; almost certainly six will be enough. But Google are making claims for their QAT tech in Gemma that they are surely using or testing in Gemini that it preserves nearly source model quality while reducing footprint.

The hardware for 50 tokens per second with a four bit quantisation of Gemma 4 26B or the sparse Qwen 3.6 is not really that expensive: it’s a secondhand M1 Max.

Beyond that, I agree. I think moving planning tasks to local is a now thing, not that it really has much impact on token spend. I also think many small coding tasks are fully within the grasp of the above two models.

The main issue right now is that the software landscape is rather confusing, but I reckon uncomplicated Gemma 4 26B QAT support with MTP is a few weeks away.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#65

Post by dgellow »

cyberziko wrote: post #2

good to know, hope the price will be affordable, having a pc becoming a luxury :)

I’m not sure if you’re aware but there is a supply chain shortage for pretty much everything needed for a PC that isn’t expected to be solved this year or next year. There is no way that can be affordable

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#66

Post by dgellow »

sisve wrote: post #58
dgellow wrote: post #43

Earlier quoted context omitted.

Memory isn’t getting cheap soon, and you need a lot of it for local models

All depends. The current technology will be cheaper in a year or two. The best cutting edge stuff will properly be even more expensive. But in 10 years time... we can run current SOTA models (or models that are equally good ) on our local hardware

Ah yes, if you count in decades, for sure I expect to run them locally

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#67

Post by Zetaphor »

netsharc wrote: post #57
iLoveOncall wrote: post #30

Earlier quoted context omitted.

> "Ranked in the top 2% of scientists globally (Stanford/Elsevier 2025) and among GitHub's top 1000 developers" - side note but this guy puts this everywhere, gives me probably the inverse of what he is marketing for. Lol yeah seriously, that stinks "I ask AI to generate a huge amount of bullshit and upload it to pad irrelevant stats". Absolute loser.

I found his website, https://www.lemire.me/en/ , and the "2%" brag is the very first sentence, geez. Being the top x% is what OnlyFans girls brag about, professor... And it's not exactly brain surgery, is it? https://www.youtube.com/watch?v=THNPmhBl-8I

> Daniel Lemire’s blog is one of the top 50 most popular blogs on Hacker News, the standard tech news aggregation site.

Citation needed

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#68

Post by infecto »

tosh wrote: post #31
infecto wrote: post #16

Earlier quoted context omitted.

As he likes to share often, "He ranks among the top 2% of scientists globally (Stanford/Elsevier 2025) and is one of GitHub's top 1000 most followed developers. "

based on citations and github stars? or what's the context there?

I was adding further citation based on his own claims. Not sure what context is missing.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#69

Post by dangus »

AmazingTurtle wrote: post #37

while unified memory may offer better performance than unsoldered DDR system memory, it still won't be as great as 1.8TB/s bandwidth on high end consumer GPUs right now. nvidias master plan may be making it the new normal to have "only" 400GB/s bandwidth, thus gatekeeping local model usage further behind "more memory but not as fast as the cloud can do it"

I think it’s an interesting theory but a bit too conspiracy theory-ish.

Nvidia just wants to sell stuff to everyone.

And I think for professionals doing local AI work, products like Strix Halo and Apple Silicon are a competitive threat.

A big part of maintaining the leading software ecosystem is ensuring you have competitive hardware for all your users.

I also think the RTX Spark product is relatively low effort for Nvidia. Grab a Mediatek CPU and slap an Nvidia GPU on the die. Sure, that’s oversimplifying it, but still.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#70

Post by dagmx »

MrBuddyCasino wrote: post #26
Apreche wrote: post #12

Earlier quoted context omitted.

I heard the memory bandwidth is not just slower than on a GPU, as expected, but is significantly slower than Apple’s unified memory.

CPU/GPU is decent (800 GB or so), memory is slowish (300GB or so). Some Apple M are slower, some are faster.

Where did you get those numbers from?

DGX Spark has a maximum of 273 GB/s bandwidth in ideal scenarios (hard to reach)

That puts it between an M5 (153) and M5 Pro (307)

Post reply on HN