Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

371–380 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#371

Post by NikolaNovak »

stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

The "one big drawback" is the lack of consumer upgrades, and the seemingly arbitrary prices charged by vendors for memory upgrades at time of system purchase. I'm not saying it has to be that way, but seems like it has been so far :-(

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#372

Post by maccard »

stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory

Game dev here. For anyone reading this - it’s not because we’re lazy, it’s because _it’s really hard to do_.

One of the biggest differences between the current generation consoles and the current gen PCs is unified memory.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#373

Post by wtallis »

MBCook wrote: post #345
cm2187 wrote: post #318

Earlier quoted context omitted.

What do you mean by required? Apple's prices are notoriously disconnected from the cost of manufacturing.

I mean is it possible to make unified memory systems with good performance or is it not really feasible due to memory timing/trace length issues? It’s possible if you’re willing to go with much slower RAM than GPUs like but CPUs often use. Thats what integrated graphics laptops have done for a long time right? But can you get high end CPU and GPU performance with unified memory and maintain user upgradable memory in…

> I mean is it possible to make unified memory systems with good performance or is it not really feasible due to memory timing/trace length issues?

LPCAMM and similar solutions exist, but have never been demonstrated running at speeds that match what the leading soldered memory systems are using; there's always been some speed penalty. I'm not sure we've ever seen a system demonstrated using LPCAMM or similar for a 512-bit bus to match Apple's Max tier SoCs, so it's somewhat of an open question whether those solutions can offer upgradability at the high end of the market for unified memory systems.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#374

Post by lmz »

ForOldHack wrote: post #343
QQ00 wrote: post #332

Earlier quoted context omitted.

both. soldered ram is faster. also Apple don't want to offer upgradblity after purchase.

Don't I/you wish. The mechanical junction adds no delay, only manufacturing expense, and the delay of purchasing new systems to keep up with OS bloat. Actually the opposite is true. Socketed RAM can be made to overclock and adjust timings, while soldered ram, no. Two Lenovo's one soldered ( Carbon X1 ), one T590, one slot: Crucial 16GB, 260-pin SODIMM, DDR4 PC4-19200. Exact same processor, the X1 is DDR3 soldered on…

I think you'll find that in the current day, high speed LP(?)DDR5 requires a better signal path than what the SODIMM can provide. Which is why laptop makers initially moved to soldered RAM before moving to CAMM (probably only for the high end ones).

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#375

Post by TylerE »

dofm wrote: post #247
TylerE wrote: post #130

Earlier quoted context omitted.

Because economy of scale isn't really the right metric here. A machine you were you were going to buy anyway essentially has a TCO of $0.

AI models will pretty undeniably affect your electricity bill; yes you already own the computer, but it will cost more to run it if it's doing inference!

To a point, but we're talking a laptop, not a server farm. Even if you're going fullbore wide open 24/7 that's about $150/yr in electricity bills at average rates. Not quite nothing but in terms of AI costs that's pretty close to rounding to zero.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#376

Post by BuyMyBitcoins »

maccard wrote: post #372
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory Game dev here. For anyone reading this - it’s not because we’re lazy, it’s because _it’s really hard to do_. One of the biggest differences between the current generation consoles and the current gen PCs is unified memory.

How much of that difficulty comes from the chosen game engine? I assume the engine is the primary factor in how resources are allocated.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#377

Post by throwaway27448 »

zokier wrote: post #319
throwaway27448 wrote: post #313

Earlier quoted context omitted.

Zero-copy shared memory?

yes, here is 2013 AMD presentation of the topic as example: https://events.csdn.net/AMD/GPUSat%20-%20hUMA_june-public.pd... see slide 14 especially

Ah. Well, what kind of consumer hardware/software combo could I purchase to use this? outside of perhaps the... PS4?

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#378

Post by johnny22 »

saltcured wrote: post #292
pbalau wrote: post #271

Earlier quoted context omitted.

What is the difference between unified memory and shared memory? Shared memory existed since the first CPU with an embedded GPU came to market and you could set in BIOS how much memory goes to what component. I do have an opinion about how unified memory could be different, but I want a proper explanation.

I'm not sure everyone uses the terms consistently, but the difference is that the old "shared" memory was reserving a section to act as VRAM under the control of the GPU, ignored by the OS. The CPU ran the same kind of code pretending there is a "bus transfer" between host memory and graphics memory. In unified memory, all the memory is host memory and data can go from program to GPU with zero copy movements. The add…

Reserved sounds like it would have been a better term now that I'm reading this many years later.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#379

Post by gravypod »

embedding-shape wrote: post #160
gravypod wrote: post #149

Earlier quoted context omitted.

What is more important than core count is how the caching architecture is laid out. They could lay out those 6k cuda cores in a layout which provides much larger blocks of cache to smaller number of cores. That would increase the memory bandwidth which would be better for inference.

Sounds like the memory bandwidth is worse though; > The memory is not as fast as dedicated GPU memory, but it is cheap enough while delivering enough bandwidth to run AI models locally. Also "cheap while delivering enough" certainly sounds like someone is trying to temper expectations. It sounds like something sitting in-between GPU+VRAM inference and CPU+RAM one, not as a step above/besides GPU+VRAM.

Having slower memory may not actually lead to lower memory bandwidth. The cuda cores can be broken up into compute complexes which larger blocks of memory directly attached to the cores. These could be filled with read operations from the bulk system memory. You can start executing and then page the next batch of data in while compute is working. For LLMs you don't have much random memory access, you can sequence your accesses in blocks.

If these chips become popular I am sure you will see LLM architectures taking advantage of the parallelism.

Post reply on HN