Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use your Nvidia GPU's VRAM as swap space on Linux

github.com

Post reply on HN
Print view View original

91–100 of 142 posts

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#91

Post by 3form »

ErroneousBosh wrote: post #72
yjftsjthsd-h wrote: post #3

> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work. Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.

In the olden days we called that a "RAM Disk" and it made our Atari STs go really fast! On the old Amstrad PCWs that were everywhere at least in the UK in the mid 80s to mid 90s you could have up to 512kB of RAM, a fair chunk of which could be a RAM disk. This made compiling stuff in Turbo Pascal really fast too :-)

Except swap is, like, opposite of RAM disk.

That said, still an nice and fun concept. Though caching got better since I assume :)

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#92

Post by dspillett »

nialv7 wrote: post #44

I mean, you prompted something useful out of an AI, good job. But then use that to ask for donation? Feels weird, man.

As much as I'm avoiding GenAI myself⁰ I think your reaction is what feels a bit weird. You wouldn't be sending a tip for simply prompting the LLM, but for having the original idea and verifying/testing the result. If you don't feel right donating for that, then don't. Seeing a “buy me a coffee” link is hardly onerous, and it isn't exactly in-your-face here (I didn't notice at all until your comment mentioned it).

--------

[0] I want to code, I like the nitty-gritty, and if I want to outsource I'd prefer to outsource to a human¹ than GenAI

[1] they might outsource to GenAI of course, that is their choice and as long as they properly verify the output before handing it on to me I shouldn't have to care

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#95

Post by xxs »

dlt713705 wrote: post #26

Does anyone these days really use swap for anything than S4 suspend ?

>S4 suspend

Is not popular in general, so yes. But also no - I don't use swap ever, if I have to go over the RAM (32GB being low, with 64GB the norm), might as well consider the system dead.

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#96

Post by xxs »

tempoponet wrote: post #50
mathisfun123 wrote: post #48

Earlier quoted context omitted.

this is the pcmasterrace equivalent of being all upper body and with scrawny legs lol

It's fine for dense models where you need them in VRAM, less so for MoE where you're offloading layers to ram. But 32/32 is pretty good for both in the popular ~30b range right now.

running 5090 on 32GB RAM is just weird, still

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#97

Post by ChocolateGod »

Tuna-Fish wrote: post #77
Phelinofist wrote: post #75

Earlier quoted context omitted.

So can VRAM actually be used like regular RAM? E.g. if I have a 16GB module and my GPU has 16GB VRAM, could it be made so that my system reports 32GB RAM? What would be the implications of that?

Typical desktop GPU ram does not support being write-back cached by the CPU. With PCIe resizable BAR, you could map the area into ram, so you could technically fit 32GB to memory, but it would have to be uncached (or write-combine cached), which would make it really, really slow. There are a bunch of datacenter GPUs that support full cache coherency, but if you used them like that the VRAM would be very high latency…

I assume on Linux you could use something like daxctl to tell the kernel to treat the vRAM as normal RAM, but I think this would be Intel/AMD only.

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#98

Post by [deleted] »

ErroneousBosh wrote: post #72
yjftsjthsd-h wrote: post #3

> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work. Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.

In the olden days we called that a "RAM Disk" and it made our Atari STs go really fast! On the old Amstrad PCWs that were everywhere at least in the UK in the mid 80s to mid 90s you could have up to 512kB of RAM, a fair chunk of which could be a RAM disk. This made compiling stuff in Turbo Pascal really fast too :-)

[deleted]

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#99

Post by lloeki »

3form wrote: post #91
ErroneousBosh wrote: post #72

Earlier quoted context omitted.

In the olden days we called that a "RAM Disk" and it made our Atari STs go really fast! On the old Amstrad PCWs that were everywhere at least in the UK in the mid 80s to mid 90s you could have up to 512kB of RAM, a fair chunk of which could be a RAM disk. This made compiling stuff in Turbo Pascal really fast too :-)

Except swap is, like, opposite of RAM disk. That said, still an nice and fun concept. Though caching got better since I assume :)

RAM disk is, like, the brd module on Linux, which allocates and exposes a /dev/ram0 block device.

From the project description this looks like it, exposing a raw block device backed by VRAM (with some trip through the nbd protocol, but that's an implementation detail to have it in userland, it could just as well have been implemented kernel side).

It's just that the usage of this mem-backed block device is different than the thing of yore (copy HD or floppy into RAM)

The more frequent alternative to brd, tmpfs, skips the block device part and does a filesystem directly. I wonder if it could be made so that it's swap directly and skip the block device part entirely like tmpfs.

Re: Use your Nvidia GPU's VRAM as swap space on Linux

#100

Post by sidewndr46 »

kccqzy wrote: post #47
sidewndr46 wrote: post #45

Earlier quoted context omitted.

I just set swappiness to zero years ago and never looked back.

That’s like the complete opposite advice. Chris said the lowest recommended swappiness is 1. I have it set to 100.

Agreed, but it means I still don't run into an OOM condition abruptly since the swap is there. It turns out modern disks are pretty fast, so the system freezes for a bit as physical memory is exhausted. Then I usually stop some of what I am doing and it goes back to normal.
Post reply on HN