Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

301–310 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#301

Post by pbalau »

saltcured wrote: post #292
pbalau wrote: post #271

Earlier quoted context omitted.

What is the difference between unified memory and shared memory? Shared memory existed since the first CPU with an embedded GPU came to market and you could set in BIOS how much memory goes to what component. I do have an opinion about how unified memory could be different, but I want a proper explanation.

I'm not sure everyone uses the terms consistently, but the difference is that the old "shared" memory was reserving a section to act as VRAM under the control of the GPU, ignored by the OS. The CPU ran the same kind of code pretending there is a "bus transfer" between host memory and graphics memory. In unified memory, all the memory is host memory and data can go from program to GPU with zero copy movements. The add…

That's my understanding, or, maybe a better word would be "guess". The CPU telling the GPU: this is your memory now.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#302

Post by cthalupa »

aabdi wrote: post #266
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

If this thing only has as much gpu bandwidth as the spark, it’s kinda pointles

Not true. This is aimed squarely at the Strix Halo and Mac markets. It's basically just strictly better than the Strix, and it's not clear cut vs that Macs in any sort of blanket statement.

My M5 Max 128gb MBP decodes faster than one of my Sparks, but the Spark's prefill is so much faster it can often answer the same query before the mac's prefill is finished. If you have large prompts, low cacheability, etc., a spark might be a very good options.

Not to mention you get can get two sparks and the MBP will be 85%+ of the cost at half the RAM.

I'm kind of tempted to pick one up. Leave running big models to my dual dgx setup, and all the misc. random stuff on an rtx.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#303

Post by mrweasel »

dh2022 wrote: post #263

A beast if a Windows PC to do what? Run Trams, Excel, Outlook, and a browser all at the same time? We could do that just fine in 2010…

Just give it a year or two and Windows will drag that sucker into the mud and run everything just a sluggish as ever.

The idea that any hardware performance increase will be eaten up by terrible software is an evergreen. A computer that could serve as the single server for a medium size enterprise 20 years ago, is no longer able to serve as a desktop for a receptionist. I'm not even sure we're talking diminishing returns anymore, we're probably past the point of maximum yield and into the negative returns at this point.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#304

Post by wmf »

jandrese wrote: post #297
testing22321 wrote: post #290

Earlier quoted context omitted.

> The Unified Memory pool is the “game changer” M1 knocking from 2020. Gamed changed, past tense, six years ago. This is catch-up.

Hell, SGI O2s from 1996 had this. For all of the hype the performance gains were pretty modest.

UMA was never about performance and it still isn't. Spark is slower than a 5090.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#305

Post by cthalupa »

dagmx wrote: post #76

This feels fluff to me on the part of the author (whose work I don’t want to trivialize) but I don’t think they’ve actually looked deeper than a paper spec sheet on this. 1. Yes it has the same number of cores as a 5070 mobile. It’s also running at a shared peak of 2/3 the bandwidth and a shared peak of 2/3 the TDP. The GPU by itself will likely perform at half the dedicated units performance 2. Apple may not have SV…

Prefill is another advantage vs. Apple. It's way way way way faster on a spark than it is even on an m5 max.

Same model, same quant, same query, as close to as matched settings as I can get from vllm, and for workloads with large prompts + low cacheability, one of my sparks will often be done responding before the mbp is done with prefill.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#306

Post by JMiao »

jandrese wrote: post #297
testing22321 wrote: post #290

Earlier quoted context omitted.

> The Unified Memory pool is the “game changer” M1 knocking from 2020. Gamed changed, past tense, six years ago. This is catch-up.

Hell, SGI O2s from 1996 had this. For all of the hype the performance gains were pretty modest.

did they learn why? were there other gains?

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#307

Post by cm2187 »

stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

And conveniently, by making your machine non upgradeable, it allows the manufacturer to enforce market segmentation / charge a huge premium for small RAM upgrade (a la Apple)

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#308

Post by Rohansi »

testing22321 wrote: post #290
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> The Unified Memory pool is the “game changer” M1 knocking from 2020. Gamed changed, past tense, six years ago. This is catch-up.

FYI it existed long before that. Shared memory between CPU and iGPU has been a thing for a long time.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#309

Post by kgeist »

zozbot234 wrote: post #287
kgeist wrote: post #256

Earlier quoted context omitted.

I have to disagree with most claims. I run Qwen3.6-27b at 260k context and 40-60 tok/sec. It handles most coding problems as well as Sonnet 4.6 under OpenCode on our production tasks. (As an experiment, I run the same prompts for the same issues in parallel for Qwen 3.6 and Sonnet 4.6 and usually see little difference in performance). I see zero degradation from quantization in practice. Settings: RTX 5090, 5-bit wei…

A 260k context (close to the stock maximum for Qwen, though it's possible to extend it) will take ~16GB RAM for storing the KV cache, barring quantization tricks which severely degrade quality. That's a whole lot more than what DeepSeek requires for a similar context length, and makes it infeasible to batch multiple inferences together. This used to be the status quo for consumer inference, in fact it still is for mo…

[dead]

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#310

Post by zozbot234 »

Salgat wrote: post #283
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

That was the main reason for the big hype around Memristors 15 years ago. High density, high speed persistent memory to completely remove the need for hdd/ssds, potentially even removing the need for external memory altogether. So frustrating that it still seems like we're a long ways from that becoming reality. There's some renewed interest in Memristors as they can simulate neural network connections in models, so…

The one example of persistent memory that managed to reach the mass market was Intel Optane/3dXPoint (still popular today among people looking to save on RAM costs) and that used a kind of phase-change memory, which is but tangentially related to memristors. ReRAM is somewhat closer, but it's also been less successful so far.
Post reply on HN