[deleted]
Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
61–70 of 149 posts
Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#62Earlier quoted context omitted.
It’s architecturally not a good approach. System RAM is much slower so you should put data that doesn’t need to be used often on it. That knowledge is at the application layer. Adding a CUDA shim makes system RAM appear like VRAM, which gets things to run, but it will never run very well. The benchmarks at the bottom mention memory tiering and manually controlling where things go, but if your application already does…
Not true for unified systems. And for strix halo you need to dedicate the amount which is annoying. You’re basically stating that swapping is also a bad idea. And to take it further, any memory or storage is a bad idea because there’s L1 cache/SRAM which is faster then the rest
> You’re basically stating that swapping is also a bad idea.
Is that a crazy thing to say? I can't recall the last time I was grateful for swap; it might've been before 2010.
Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#63Earlier quoted context omitted.
It’s architecturally not a good approach. System RAM is much slower so you should put data that doesn’t need to be used often on it. That knowledge is at the application layer. Adding a CUDA shim makes system RAM appear like VRAM, which gets things to run, but it will never run very well. The benchmarks at the bottom mention memory tiering and manually controlling where things go, but if your application already does…
Not true for unified systems. And for strix halo you need to dedicate the amount which is annoying. You’re basically stating that swapping is also a bad idea. And to take it further, any memory or storage is a bad idea because there’s L1 cache/SRAM which is faster then the rest
It's not true for unified systems, because they have no secondary RAM that could be used to extend the GPU memory.
It's pretty weird to insist on a counterargument that has no implications or consequences to the presented argument.
Yes, swapping is a bad idea.
Your second argument also falls flat, because the standard CUDA hardware setup doesn't use CXL so cache coherence isn't available. You're left with manual memory synchronization. Pretending that GPUs have cache for system RAM when they don't is pretty suspect.
Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#64You can already do this with some GPU drivers: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amdttm.pages_limit=5242880 ttm.pages_limit=5242880" One downside is your kernel isn't going to reserve that memory away from userland. You will still see all the memory at system level as "free". As the GPU driver starts using it, other apps/the OS will try to use the "free" memory, not knowing how much of it is in use (it may sho…
Would MoE models work better with this approach?
Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#65How does this differ from anything llama.cpp offers, regarding offloading layers? The repo consistently refers to "DDR4". Is there a reason DDR5 won't work with this?
CUDA has had managed memory that pages between VRAM and system RAM for a decade. Problem is doing so is unusably slow for AI purposes. Seems like an unnecessary layer here.
[flagged]
Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#66Could this work on steam deck?
Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#67Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#68Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#69Nobody mentioning how this project is vibecoded slop?
> The code is really bad with completely uneeded parts. The LLM (Qwen 2.5 7B) has hardcoded the i9 14700KF topology, and has variables related to it never used... It's even funnier that the show hardware function always prints the same string. There are even random pip log files. Why did this slop got coverage here?
Re: Nvidia greenboost: transparently extend GPU VRAM using system RAM/NVMe
#70Could be a very useful way to do some overnight tasks using spare RAM. Possibly things like LLM-based categorisation, labelling, data cleansing. That's what comes to mind for me anyway.
Neat part is every task becomes overnight task when you start offloading to RAM.