All my servers have an 8 GB empty file on disk
brianschrader.com
All my servers have an 8 GB empty file on disk
1–10 of 715 posts
Re: All my servers have an 8 GB empty file on disk
#2I will be doing this. Marvelous idea.
Re: All my servers have an 8 GB empty file on disk
#3hope you're not running -o compress=lz4 , because you are going to be in for a big surprise when you try to pull this emergency lever! you may be shocked to see you don't actually get much space back!
i do wonder how many FS would actually allocate the 8GB if you, for example, opened a file, seeked to 8GB mark, and wrote a character. many file systems support "sparse files"[1]. for example on btrfs, i can run 'dd if=/dev/zero of=example.sparse count=1 seek=2000000' to make a "1GB" file that has just one byte in it. btrfs will only allocate a very small amount in this case, some meta-data to record an "extent", and a page of data.
i was expecting this article to be about a rude-and-crude overprovisioning method[2], but couldn't guess how it was going to work. SSDs notably perform much much better when they have some empty space to make shuffling data around easier. leaving a couple GB for the drive to do whatever can be a colossal performance improvement, versus a full drive, where every operation has to scrounge around to find some free space. i wasn't sure how the author was going to make an empty file that could have this effect. but that's not what was going on here.
[1] https://wiki.archlinux.org/index.php/sparse_file
[2] https://superuser.com/questions/944913/over-provisioning-an-...
Re: All my servers have an 8 GB empty file on disk
#4hope you're not running -o compress=lz4 , because you are going to be in for a big surprise when you try to pull this emergency lever! you may be shocked to see you don't actually get much space back! i do wonder how many FS would actually allocate the 8GB if you, for example, opened a file, seeked to 8GB mark, and wrote a character. many file systems support "sparse files"[1]. for example on btrfs, i can run 'dd if=…
Good point. I didn't think of that. I'm not sure how my CentOS servers handle this scenario, but it seemed to take up the full 8GB however I checked.
Re: All my servers have an 8 GB empty file on disk
#5hope you're not running -o compress=lz4 , because you are going to be in for a big surprise when you try to pull this emergency lever! you may be shocked to see you don't actually get much space back! i do wonder how many FS would actually allocate the 8GB if you, for example, opened a file, seeked to 8GB mark, and wrote a character. many file systems support "sparse files"[1]. for example on btrfs, i can run 'dd if=…
> hope you're not running -o compress=lz4 , because you are going to be in for a big surprise when you try to pull this emergency lever! you may be shocked to see you don't actually get much space back!
This is true. If you are replicating this, copy from /dev/urandom rather than using an empty file.
Re: All my servers have an 8 GB empty file on disk
#6This is clever. Our shaky version of this, historically, has been to run ncdu and hastily delete the first large log file we see. It’s not ideal.
Re: All my servers have an 8 GB empty file on disk
#7Isn’t using LVM and holding some space back a better solution for this?
Also I keep databases on their own partition so that nothing else can accidentally fill up the space and lead to data loss.
Re: All my servers have an 8 GB empty file on disk
#8This is clever. Our shaky version of this, historically, has been to run ncdu and hastily delete the first large log file we see. It’s not ideal.
(that's actually how I solved the original issue that I reference in the post, and how I got the idea for this silly solution)
Re: All my servers have an 8 GB empty file on disk
#9> Copy commands and even deletions can fail or take forever as memory tries to swap to a full disk
That's only a problem if your memory is full as well, and even then, I've never encountered a server that uses a swapfile instead of a swap partition.
Re: All my servers have an 8 GB empty file on disk
#10Isn’t using LVM and holding some space back a better solution for this? Also I keep databases on their own partition so that nothing else can accidentally fill up the space and lead to data loss.
Maybe, but author says:
> On Linux servers it can be incredibly difficult for any process to succeed if the disk is full.
You won't feel too clever if you come to grow your LVM volume into the free space and it won't work because there's no free space on the filesystem! :)
(I don't actually know if this would fail or not - but the point is "rm spacer.img" is pretty much guaranteed not to fail).