Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

Post reply on HN
Print view View original

541–550 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#541

Post by marcosdumay »

trumpdong wrote: post #539
marcosdumay wrote: post #531

Earlier quoted context omitted.

As a rule, increasing the size of cache increases its latency, and how much of it you can use is capped by the quality of your cache management algorithms and the latency of the level above it. Since CPUs are highly optimized, both increasing the latency of the main memory and increasing the size of L3 will probably lead to larger L3 latency.

We might even decide to put 32GB of high-latency cache on the system board and then 12GB of throughput-optimized main memory close to the GPU. ;)

You meant a 128GB (instead of 12GB)?

And yes, a L4 cache can be one way out of that problem. Another way is making the L3 cache lines wider and working the hell out of improving your management algorithm.

It's not a theoretically impossible problem. It's also not something you can solve automatically with a bit more money or some simple decisions. It's possible this is the best architecture available, but it's not certain by any means.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#542

Post by gmueckl »

nearbuy wrote: post #540
gmueckl wrote: post #476

Earlier quoted context omitted.

I'm being very, very conservative with my estimates here. Based on the renderers I know, I could have easily tweaked the numbers to go up to 8000 full screen texture reads per second. That doesn't include texture or geometry or BVH reads or any memory writes. That is all in addition to those operations.

But do you think you'll reach 1.8 TB/s?

Quite likely, but the transfer throughput is required in bursts, not necessarily continously.

Let me put it this way: what I care about is how quickly data arrives after a bunch of shader threads request it. Throughput is one way for hardware to reduce that time. The other way is to hide the latency (GPUs do a lot to keep themselves busy while waiting for memory), but those strategies can only do so much.

Lower memory throughput almost always leads to a longer runtime of GPU calls in practice, and thus lower update rates.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#543

Post by Melatonic »

AnthonyMouse wrote: post #439
stego-tech wrote: post #197

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

> Lets systems optimize utilization based on need, rather than be confined to specific pools The trouble with this is that the different types of memory have different characteristics. Latency for ordinary system memory is actually better than it is for GDDR, because GDDR is optimized for bandwidth. RTX 5090 has 1.8TB/s of memory bandwidth with a 512-bit memory bus. The same bus width for DDR5-9600 would have better…

Isn't GDDR also based on a much earlier DDR implementation than DDR5 ?

It also has way better throughput because it's physically surrounding the chip itself and wired in a way that maximises this.

The real problem is interconnect speed and latency. We have made tons of progress elsewhere but AI is exposing that the interconnect in many systems is just not great. Even future PCIE 6.0 is fairly bandwidth constrained compared to 8 channels of DDR memory or the way we solder GDDR next to the chip.

We moved on from AGP and older formats to PCI-E and I think it's time to do that again. And maybe even "slot" based implementations in general for both RAM (system and graphics) and GPUs.

We need consumer and workstations in summary to use pin based stuff like LPCAMM ram. And the interconnect on the motherboard itself needs to be both wider (more bandwidth) and lower latency. This might require moving on from motherboard being 2 dimension only (a flat board) to something like an L shape to gain more physical board space.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#544

Post by Melatonic »

ssivark wrote: post #504
AnthonyMouse wrote: post #439

Earlier quoted context omitted.

> Lets systems optimize utilization based on need, rather than be confined to specific pools The trouble with this is that the different types of memory have different characteristics. Latency for ordinary system memory is actually better than it is for GDDR, because GDDR is optimized for bandwidth. RTX 5090 has 1.8TB/s of memory bandwidth with a 512-bit memory bus. The same bus width for DDR5-9600 would have better…

How about having a large pool of unified memory and expanding the next layer (L3?) of cache to accommodate more of the CPU's the low-latency RAM usage?

I think that's basically what Cerebras doing ?

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#545

Post by Melatonic »

We need a physical architectural change if we really want to improve this. You can only physically wire so much memory next to a chip in a 2 dimensional (flat) design - limited by the edge lengths of the chip (CPU or GPU)

Basically what we need is a chip that also has pins or some type of attachement system on the top (physically) or maybe below where the chip itself connects to the motherboard.

Imagine a CPU you can just plug in a block of HBM memory on top of (or on "bottom" of). This would allow a much larger physical surface area for putting ram cache near the compute cores itself because you would not be limited to edge lengths.

Cooling the whole thing would be a methodology change (might need liquid coolers that sandwhich in between the ram cache and compute and cool both)

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#547

Post by Melatonic »

to11mtm wrote: post #324
cm2187 wrote: post #307

Earlier quoted context omitted.

And conveniently, by making your machine non upgradeable, it allows the manufacturer to enforce market segmentation / charge a huge premium for small RAM upgrade ( a la Apple)

It doesn't -have- to be that way necessarily... LPCAMM2/SOCAMM2 exist, heck I think Framework is using LPCAMM2 in one of their new laptops. Heck, I'm willing to bet that a lot of manufacturers would rather go that route than soldered in, if for no other reason than the relative cost of warranty work between the two. However, people probably need to stop being obsessed with ultrathin laptops for that to happen.

Yup - we need pin based memory. Period. It's a physics thing.

CPUs don't slot in for a reason

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#548

Post by Melatonic »

sroussey wrote: post #462
arka2147483647 wrote: post #323

Earlier quoted context omitted.

There is LPCAMM2, if manufacturers want to use it. So, it does not have to be soldered.

LPCAMM2 is available in real systems at 7467MT/s and 120ns latency, vs apple (and intel) at 9600MT/s (and apple soldered memory at 100ns latency). I don't know how linear or sensitive CPU and GPU benchmarks are to such a 20% slowdown, but i don't think Apple wants to pay it. And it looks like the next generation will be even closer to the SOC.

LPCAMM2 is also brand new. It likely will improve a lot.

We're also hitting the limit of DDR5 here (before moving to multiplexed)

I would guess if you had LPCAMM2 located physically around the CPU (one or two on each of the 4 CPU edges) you could also reduce that latency.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#549

Post by Melatonic »

wtallis wrote: post #456
AnthonyMouse wrote: post #441

Earlier quoted context omitted.

> LPCAMM and similar solutions exist, but have never been demonstrated running at speeds that match what the leading soldered memory systems are using; there's always been some speed penalty. LPCAMM2 supports up to 9600MT/s, which appears to be the same speed Apple is using. > I'm not sure we've ever seen a system demonstrated using LPCAMM or similar for a 512-bit bus Servers commonly use a 768-bit DDR5 memory bus pe…

> LPCAMM2 supports up to 9600MT/s, which appears to be the same speed Apple is using. The difference here is in what the standard defines on paper vs what is actually shipping in products and readily available off the shelf. Who's selling a whole system with LPCAMM2 certified for 9600MT/s? Intel's current-gen Panther Lake top of the line laptop chips are rated for 9600MT/s when using soldered LPDDR5x but only 7467MT/…

Multiplexed DDR (MRDIMM) can go faster.

But for throughput served with 12 channels have pretty high theoretical even with slower

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#550

Post by Melatonic »

zozbot234 wrote: post #310
Salgat wrote: post #283

Earlier quoted context omitted.

That was the main reason for the big hype around Memristors 15 years ago. High density, high speed persistent memory to completely remove the need for hdd/ssds, potentially even removing the need for external memory altogether. So frustrating that it still seems like we're a long ways from that becoming reality. There's some renewed interest in Memristors as they can simulate neural network connections in models, so…

The one example of persistent memory that managed to reach the mass market was Intel Optane/3dXPoint (still popular today among people looking to save on RAM costs) and that used a kind of phase-change memory, which is but tangentially related to memristors. ReRAM is somewhat closer, but it's also been less successful so far.

Optane was still much slower than Ram. And not that much faster than NVME (theoretically)
Post reply on HN