Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

291–300 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#291

Post by nalekberov »

stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

It’s also the reason, why you will never be able to repair or upgrade your computer in the future. From technological point of view these are indeed big advancements.

However, I couldn’t care less about faster CPU when:

1. It limits my ability to upgrade my system

2. Windows gets increasingly bloated and slower

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#292

Post by saltcured »

pbalau wrote: post #271
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

What is the difference between unified memory and shared memory? Shared memory existed since the first CPU with an embedded GPU came to market and you could set in BIOS how much memory goes to what component. I do have an opinion about how unified memory could be different, but I want a proper explanation.

I'm not sure everyone uses the terms consistently, but the difference is that the old "shared" memory was reserving a section to act as VRAM under the control of the GPU, ignored by the OS. The CPU ran the same kind of code pretending there is a "bus transfer" between host memory and graphics memory.

In unified memory, all the memory is host memory and data can go from program to GPU with zero copy movements. The addresses of buffers can be shared via appropriate MMU translation support, so that the application and graphics subsystem are communicating effectively through the basic RAM cache coherency protocols over the same buffers.

Edit to add: Aside from the zero copy transfer potential, it also means dynamic allocation strategies can shift the balance between host and graphics allocations on the fly. Individual image and message buffers can be allocated on the fly instead of setting a static split between the two worlds.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#293

Post by Rohansi »

pbalau wrote: post #271
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

What is the difference between unified memory and shared memory? Shared memory existed since the first CPU with an embedded GPU came to market and you could set in BIOS how much memory goes to what component. I do have an opinion about how unified memory could be different, but I want a proper explanation.

Marketing, mostly? But perhaps also more flexibility with how much memory the GPU can directly access without reserving it.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#294

Post by [deleted] »

pbalau wrote: post #271
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

What is the difference between unified memory and shared memory? Shared memory existed since the first CPU with an embedded GPU came to market and you could set in BIOS how much memory goes to what component. I do have an opinion about how unified memory could be different, but I want a proper explanation.

[deleted]

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#295

Post by bombcar »

testing22321 wrote: post #290
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> The Unified Memory pool is the “game changer” M1 knocking from 2020. Gamed changed, past tense, six years ago. This is catch-up.

I want unified but not uniform - everything can address anything, but you can add slower RAM to the system without requiring an entirely new chip. NUMA is cool.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#296

Post by merb »

nalekberov wrote: post #291
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

It’s also the reason, why you will never be able to repair or upgrade your computer in the future. From technological point of view these are indeed big advancements. However, I couldn’t care less about faster CPU when: 1. It limits my ability to upgrade my system 2. Windows gets increasingly bloated and slower

LPCAMM2

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#297

Post by jandrese »

testing22321 wrote: post #290
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> The Unified Memory pool is the “game changer” M1 knocking from 2020. Gamed changed, past tense, six years ago. This is catch-up.

Hell, SGI O2s from 1996 had this. For all of the hype the performance gains were pretty modest.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#298

Post by cthalupa »

pbalau wrote: post #271
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

What is the difference between unified memory and shared memory? Shared memory existed since the first CPU with an embedded GPU came to market and you could set in BIOS how much memory goes to what component. I do have an opinion about how unified memory could be different, but I want a proper explanation.

For these in specific, they appear basically transparently to the GPU. There's a lot of software/firmware stuff for this, but also a different hardware architecture - while the RAM is on the CPU die, the nvlink-c2c gives it extremely low latency and 600GB/s bandwidth between the GPU and CPU.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#299

Post by zozbot234 »

Salgat wrote: post #285
jmyeet wrote: post #213

Earlier quoted context omitted.

Unified memory is only a feature because NVidia so aggressively uses VRAM for market segmentation. The 5090 ($2k MSRP but realistically $3-3.5k) is almost the same as the RTX 6000 Pro (~$10k). Same memory bandwidth (1800GB/s). Slightly different CUDA cores (21k vs 24k). Big difference? VRAM (32GB vs 96GB). NVidia ultimately doesn't want to upset this segmentation so the RTX Spark will never undermine their other offe…

To this day I do not get why Intel doesn't just offer massive memory options for their cards. Just charge what it costs to add the extra memory, no upcharge, and they will never be able to keep up with demand. Cheap VRAM is enough to justify a lot of open source investment into challenging CUDA.

> To this day I do not get why Intel doesn't just offer massive memory options for their cards.

They seem to? Intel Arc is the cheapest option by far for a discrete card with 32GB VRAM.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#300

Post by api »

Silhouette wrote: post #280
wccrawford wrote: post #207

Earlier quoted context omitted.

That's exactly the point. Until recently, AI models that could run on home machines were so bad that it was very hard to imagine anyone wanting to. And, like the overly large machines of 1977, models are getting faster, leaner, and better. It's happening a lot quicker, though.

This is why I'm bearish on Anthropic, OpenAI, and friends. I am not confident that we will continue to see the same pace of improvement in frontier model capabilities as we have seen over the past year or two - not using similar mathematics at least. But I think that getting results that are close enough to the same standard to be a realistic substitute but in a model small enough to run locally may well happen quite…

If model performance asymptotes and CPU/GPU and RAM keep growing, even slowly, then eventually we will have frontier models on desktop that are totally competitive with hosted. It’s only a matter of time.

You already can if you’re willing to spend many thousands of dollars on a beast of a machine. I’m talking about middle tier desktops and laptops here. Maybe eventually even phones.

The only way hosted stays strongly competitive in that world is if they can keep pushing the frontier or by playing the classic social media and SaaS games of network effect building and integrations.

Many people might still use hosted, of course, but what I really mean is that their multiples won’t be justified and they will have little to no moat. AI will become commoditized, like a sophisticated next generation form of an encyclopedia with search.

Post reply on HN