sidenote: it is possible to use Vulkan to map GPU memory to CPU space and even map it back to CUDA: https://x.com/steeve/status/2055042304344231978?s=20
Use your Nvidia GPU's VRAM as swap space on Linux
71–80 of 142 posts
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#72> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work. Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
In the olden days we called that a "RAM Disk" and it made our Atari STs go really fast!
On the old Amstrad PCWs that were everywhere at least in the UK in the mid 80s to mid 90s you could have up to 512kB of RAM, a fair chunk of which could be a RAM disk. This made compiling stuff in Turbo Pascal really fast too :-)
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#73> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work. Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
Another possible reason that occurred to me: what if you have VRAM but you're not using it all the time? For example, let's say you bought a GPU because you like to play video games. When you're not actively gaming, you probably don't need 16 GB of VRAM just to render the desktop. Might as well use it for something else, right? Edit: Although, this is predicated on the system being able to release VRAM that is acting…
Best case is if gaming and productivity (with high memory use) activities are not concurrent, and productivity applications are stopped before gaming starts, then `swapoff` can easily release swap device without restart.
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#74I've implemented the same idea with OpenCL: https://github.com/theblazehen/vramblk
There is originally https://github.com/Overv/vramfs however that has the overhead of a FUSE filesystem + loop device when using as a swap device.
The performance is rather lackluster however, it's far from a miracle "now you effectively have more ram for a 90% performance drop" - it definitely feels like traditional swapping
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#75> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work. Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
So can VRAM actually be used like regular RAM? E.g. if I have a 16GB module and my GPU has 16GB VRAM, could it be made so that my system reports 32GB RAM? What would be the implications of that?
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#76Given my dev machine has 32GB of RAM and 32GB of VRAM that sits mostly idle when I'm not running AI models, this is not that bad of an idea.
this is the pcmasterrace equivalent of being all upper body and with scrawny legs lol
Actually not that crazy of a spread. E.g. I have 48 GB + 32 GB in my gaming PC because if you go beyond 48 GB you start having to trade off more and more performance to keep the memory controller from falling over, so you really have to have a good reason to want to load more. Server platforms, like Epyc, it tends not to matter as much because you have so many channels for bandwidth and a beefier memory controller to handle them. Then on the VRAM side it's more about what makes sense for the GPU and how you plan on using it there (games or AI or modeling or whatever), and for a lot of cases the 5090 is just a good card to get for one reason or another (it just has a ton of compute + bandwidth for a consumer part).
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#77> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work. Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
So can VRAM actually be used like regular RAM? E.g. if I have a 16GB module and my GPU has 16GB VRAM, could it be made so that my system reports 32GB RAM? What would be the implications of that?
Typical desktop GPU ram does not support being write-back cached by the CPU. With PCIe resizable BAR, you could map the area into ram, so you could technically fit 32GB to memory, but it would have to be uncached (or write-combine cached), which would make it really, really slow.
There are a bunch of datacenter GPUs that support full cache coherency, but if you used them like that the VRAM would be very high latency from the CPU. So it would only be really slow.
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#78> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work. Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
So can VRAM actually be used like regular RAM? E.g. if I have a 16GB module and my GPU has 16GB VRAM, could it be made so that my system reports 32GB RAM? What would be the implications of that?
It behaves like slower ram I assume, due to the increased distance from the CPU and overhead. Still, it’s much faster than normal SWAP which uses a disk or SSD.
How it is reported? As SWAP space, not as RAM.
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#79[dead]
Re: Use your Nvidia GPU's VRAM as swap space on Linux
#80I have long wished it was possible to do this. What a great bit of code. Thanks.