Live data from Hacker News

Nvidia is proposing a beast of a CPU system for Windows PCs

twitter.com

321–330 of 581 posts

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#321

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

Memory safety is orthogonal to side-channels, and hardware-enforced isolation (e.g. IOMMU) is more powerful than compiler-enforced isolation (but both are good!)

Oh no now I have to worry about shaders row hammering my OS ram /s

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#322
post #249

Earlier quoted context omitted.

I have so many questions… Since Apple already sells unified memory systems, what is the market opportunity you envision? Do you see Nvidia and Apple as competitors, and how? (And I’m not suggesting they’re not, necessarily, but I want to hear where you’re coming from, and they do have very different markets.) Hasn’t Apple used storage size (RAM & disk) for market segmentation for decades? And how does a machine with…

I'm not the person you're replying to, but I wholeheartedly agree with them... Quick background: doing AI inference requires three things. Lots of memory, lots of memory bandwidth, and of course plenty of compute that has access to that memory. Quick reference: nVidia 5090 has 1,792 GB/sec bandwidth. 3090 gets about 1000 GB/sec. DGX Spark and AMD 395 whatever get about 275 GB/sec. Apple M1 Max gets 400GB/sec, M5 Max…

They use different technology for their VRAM though. Apple, AMD Strix and NVidia DGX/RTX Spark use LPDDR, whereas discrete cards will be either GDDR or HBM. That directly impacts the memory bandwidth figures. As for compute available, Apple and AMD still have very good figures there for what's essentially a general-purpose iGPU that ships as part of the stock system, rather than a special-purpose piece of dedicated hardware.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#323
post #307

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

And conveniently, by making your machine non upgradeable, it allows the manufacturer to enforce market segmentation / charge a huge premium for small RAM upgrade ( a la Apple)

There is LPCAMM2, if manufacturers want to use it.

So, it does not have to be soldered.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#324
post #307

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

And conveniently, by making your machine non upgradeable, it allows the manufacturer to enforce market segmentation / charge a huge premium for small RAM upgrade ( a la Apple)

It doesn't -have- to be that way necessarily...

LPCAMM2/SOCAMM2 exist, heck I think Framework is using LPCAMM2 in one of their new laptops.

Heck, I'm willing to bet that a lot of manufacturers would rather go that route than soldered in, if for no other reason than the relative cost of warranty work between the two.

However, people probably need to stop being obsessed with ultrathin laptops for that to happen.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#326
post #307

The Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unifie…

And conveniently, by making your machine non upgradeable, it allows the manufacturer to enforce market segmentation / charge a huge premium for small RAM upgrade ( a la Apple)

how about the LPCAMM route? Framework uses LPCAMM2 in 13 Pro laptop mainboards and claims that it satisfies the iGPU and NPU hardware without needing soldered RAM

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#327

Earlier quoted context omitted.

No. It’s all integrated. Not something you can buy separately or upgrade later.

See [1]. There's not 128GB of on-chip memory. "Integrated" memory in this context means that the GPUs and CPUs all use the same memory. There are on-chip caches, of course. [1] https://www.nvidia.com/en-us/products/rtx-spark/

Ehhh.

It is all in integrated into one monolith “superchip” package. The 128gb of RAM isn’t going to be purchased separately or be upgradable. At least according to all indicators. Which is what I was responding to.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#328
post #271

Earlier quoted context omitted.

What is the difference between unified memory and shared memory? Shared memory existed since the first CPU with an embedded GPU came to market and you could set in BIOS how much memory goes to what component. I do have an opinion about how unified memory could be different, but I want a proper explanation.

Marketing, mostly? But perhaps also more flexibility with how much memory the GPU can directly access without reserving it.

No. Let’s define terms, as others have pointed out they’re not perfect.

Unified memory is what Apple is doing, other phones do, and many low end built in GPUs have done in PCs for ages. There is only one physical memory pool. Both the CPU and GPU can access it at full speed.

This means no copying between pools of memory. No speed penalty accessing the CPU memory from GPU or vice versa. If the GPU only needs 2 GB to draw the desktop it only uses 2 GB of the pool. Or it can use 45 GB if it needs it and the CPU doesn’t. But all memory has to be the same speed, and that ain’t cheap given how fast GPUs like things. I don’t know if expandable memory is possible, and they use the same bus do they compete for bandwidth. Seems theoretically easier to program for to me.

The opposite is what’s been common in graphics cards since the 2D era. CPU and GPU have their own memory and can talk over PCI/AGP/PCI-E. This is what I think they mean by shared memory, if it’s not what’s the point in touting unified?

In this model if the GPU uses 2 GB of its 12 GB total, the other 10 isn’t available to the OS at full speed and I’m not aware of any operating systems that would use it for programs/cache by default. If the GPU needs 45 GB… too bad. You have to page things in and out of GPU memory over the much slower system bus. Starting a game means loading assets into main memory then transferring them to the GPU (newer tech can accelerate this). But the CPU can have slower memory than the GPU saving money. Memory expansion on the CPU side easy. And the CPU saturating its memory bus has no effect on the speed of the GPU memory bus because it’s physically separate. More complicated memory model but it’s the one everyone uses used to.

Which is better is a matter of opinion and workload needs.

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#329
post #15

"I am not sure how many people will run AI models locally. It still seems like a niche application to me. However, it will make decent machines to play video games." I don't know who will be the winner but with some of the recent releases from gemma it seems more probable that you may run some models locally if only from a cost perspective, not even considering business security. Not sure how this type of architectur…

[deleted]

Re: Nvidia is proposing a beast of a CPU system for Windows PCs

#330

Earlier quoted context omitted.

Most people are ok with slower. An AI that lets you edit a family picture, in say 30 seconds, locally is preferable to one that is instantaneous but requires you to submit that picture to examination/storage/training/sale in someone else's AI ecosystem. If i want to crop my ex out of family photos, i should not have to first give that photo to Microsoft. If want an LLM to write a book report for me, i dont want it al…

It’s completely technically possible to have cloud services where customer data is opaque to the provider. Some of Apple’s services are like this already, for example. I think there’s a sweet spot currently with munging your data blindly on the server so that your client device battery still lasts all day. Meanwhile Apple and others push on with making client side models more efficient so that eventually the server c…

dont want to share my pics with "cloud services"
Post reply on HN