I thought this was gonna be about the obscenely large sparse file /var/log/last.
I really wish they would move it from a sparse memmap() file to a btree or something.
All my servers have an 8 GB empty file on disk
291–300 of 715 posts
Re: All my servers have an 8 GB empty file on disk
#292If you happen to use ext as your default filesystem, check the output of tune2fs; it's possible your distro has conveniently defaulted some 2-5% of disk space as "reserved" for just such an occasion. As the root user, in a pinch, you can set that to 0% and immediately relieve filesystem pressure, buying you a little bit more time to troubleshoot whatever the real problem is that filled the disk in the first place.
Re: All my servers have an 8 GB empty file on disk
#293I am reminded of a tweet that suggested adding a sleep() call to your application that makes some part of it needlessly slow, so that you can give users a reason to upgrade when there's a security fix (it's 1 second faster now)!
Apple did something like that... https://www.npr.org/2020/11/18/936268845/apple-agrees-to-pay...
They did it so old phones wouldn't suddenly hard-shutdown at 30% battery. I appreciate that they did that, it's very annoying.
Re: All my servers have an 8 GB empty file on disk
#294Earlier quoted context omitted.
If it's morally incorrect to get to the license server early and snatch a roaming license you might not use to its fullest, is it not also wrong to access the property market early and snatch a bunch of land you might not use to its fullest?
The difference is that market mechanisms are meant to allocate items to those that value them the most (assumed to be the same as their willingness to pay). Claiming a licence by contrast has little cost.
I think a homeless person values an empty house way more than a real estate owner.
Re: All my servers have an 8 GB empty file on disk
#295> On Linux servers it can be incredibly difficult for any process to succeed if the disk is full. Copy commands and even deletions can fail or take forever as memory tries to swap to a full disk and there's very little you can do to free up large chunks of space. This reasoning doesn't make sense. On Linux, swap is preallocated. This is true regardless of whether you're using a swap partition or a swap file. See man…
Not sure about their reasoning.. but if you don't have root ssh enabled, sudo can break if there is no free disk space. I do something similar where I write a 500mb file to /tmp and chmod 777 it so anyone can free it up without needing sudo.
I've experienced far more full disks than I'd want to admit, on many different hardware and software configurations, and I've never seen sudo break. Is this something you've experienced recently?
I definitely agree with your advice and will go double check all my servers if /filler is 777 (not in /tmp since it's sometimes mounted tmpfs), but if sudo does break in that situation, that sounds like a pretty severe and most likely fixable bug.
Re: All my servers have an 8 GB empty file on disk
#296For everyone saying "This isn't a real solution!" I'd like to explain why I think you're wrong. 1) It's not intended to be a Real Solution(tm). It's intended to buy the admin some time to solve the Real Issue. 2) Having a failsafe on standby such as this will save an admin's butt when it's 2am and PagerDuty won't shut up, and you're just awake enough to apply a temp fix and work on it in the morning. 3) Because "FIX…
I find that either a server needs more space, or has files that can be deleted. For the former you just increase the disk space, since most things are VMs these days and increasing space is easy. For the latter you can usually delete enough files to get the service back up before you start the proper cleanup. If you really need some reserve space (physical server), I'd much rather store it in a vg (or zfs/btrfs subvo…
> Will you remember the file exists at 2am? What about the other admins on your team?
Hopefully if you were doing something like this it would be part of your standard incident response runsheet/checklist.
Re: All my servers have an 8 GB empty file on disk
#297Earlier quoted context omitted.
> Property ownership comes at a cost. Sort-of. Typically there is a cost associated with getting rights over a property, either by manufacturing, or in the case of land, by purchase or through the efforts of settling. However, once you have ownership of a property, it's usually relatively cheap to continue own (except, I suppose, if you consider the risk of a communist revolution or something). This is esentially how…
In California you pay ~1.5% in property taxes per year. Owning property is not cheap. In fact you never really own it.
[deleted]
Re: All my servers have an 8 GB empty file on disk
#298Earlier quoted context omitted.
Apple did something like that... https://www.npr.org/2020/11/18/936268845/apple-agrees-to-pay...
They did it so old phones wouldn't suddenly hard-shutdown at 30% battery. I appreciate that they did that, it's very annoying.
That's what Apple likes to say. Luckily they got at least a small fine.
Re: All my servers have an 8 GB empty file on disk
#299In the mid nineties I worked in a research institute. There was a large shared Novell drive which was always on the verge of full. Almost every day we were asked to clean up our files as much as possible. There were no disc quota for some reason. One day I was working with my colleague and when the fileserver was full he went to a project folder and removed a file called balloon.txt which immediately freed up a few p…
Perfect example of the tragedy of the commons. If individuals don't create these balloon files then they won't be able to use the file server when they need it, yet by creating these balloon files the collective action depletes the shared resource of its main function.
[deleted]
Re: All my servers have an 8 GB empty file on disk
#300In the mid nineties I worked in a research institute. There was a large shared Novell drive which was always on the verge of full. Almost every day we were asked to clean up our files as much as possible. There were no disc quota for some reason. One day I was working with my colleague and when the fileserver was full he went to a project folder and removed a file called balloon.txt which immediately freed up a few p…
I worked at a large company during a migration from Lotus to Outlook. We were told we'd get our current Lost email storage + 100MiB as a new email quota limit under Outlook. I made a bunch of 100MiB files of `/dev/random` noise (so they don't compress, compressed size was part of the quota) and emailed them to myself before the migration, to get a few GiB of quota buffer. My co-workers were constantly having to delet…
Email quotas aren’t just a cost thing. It forces deletion of files/communications that aren’t relevant anymore. The last thing the legal department wants is some executive’s laptop with 10 years of undeleted email to make it’s way to discovery.