101–110 of 715 posts
In the mid nineties I worked in a research institute. There was a large shared Novell drive which was always on the verge of full. Almost every day we were asked to clean up our files as much as possible. There were no disc quota for some reason. One day I was working with my colleague and when the fileserver was full he went to a project folder and removed a file called balloon.txt which immediately freed up a few p…
Earlier quoted context omitted.
rm /path/to/big.file is faster than looking up the commands to expand the LVM volume and grow the filesystem.
They’re not hard to memorise. I never look them up.
> Copy commands and even deletions can fail or take forever as memory tries to swap to a full disk That's only a problem if your memory is full as well, and even then, I've never encountered a server that uses a swapfile instead of a swap partition.
Not quite the same situation as described in the article, but it is still possible for the kernel to swap memory in and out of disk even without a swap file/partition. Memory used for storing executable binaries is allowed to be moved out of memory, as a copy of it lives on disk. This means you can still encounter memory thrashing (and thus system unresponsiveness) under low memory situations.
On Linux, this is not necessarily the case, as you can change the file on disk while the executable is running. I don't know if Linux just keeps executable code in memory all the time, or if it is smart enough to detect whether a copy of executable pages still lives on disk.
All my servers have an alarm when disk space goes above 70%. It sends an email every hour once the disk usage goes above 70%. Never had a server go down because of disk space issue after adopting this practise. Also one of the main reasons server disks go full is generally log files. Always remember to "logrotate" your log files and you will not have this issue that much. Yes one more thing, for all user uploaded fil…
We send our log files to S3 too. I mean, we write them locally (EC2) and then push them to S3 every minute.
Then we have a tool that will let us search the log files in S3 and it will parse these rotated log files and join together the relevant pieces depending on what we're looking for (or all of it for a specific time period if we don't know what we're looking for).
This is great because if the server goes down and we can't access it, or the instance is gone, we can still see log files from shortly before the problem occured. We also use bugsnag, etc for real time logging and tracking where possible.
Ah, the classic 'speed-up loop' approach: https://thedailywtf.com/articles/The-Speedup-Loop About the blogpost itself: The disk filled up, and that's one thing you don't want on a Linux server—or a Mac for that matter. When the disk is full nothing good happens. I had this happen few times on a Mac and every time I was shocked that if disk gets full you cannot even delete a file and the only option is to do a full sy…
This just happened to me. I got the best error message I've ever seen. Something akin to "Can not remove file because the disk is full." This wasn't from the Finder, this was command line rm.
On the Mac it's also exacerbated by the fact that swap will use the system drive and can fill up the disk, and can not be stopped. If you have some rogue process consuming RAM, among other things, your disk will suffer until it is full. And, as mentioned, macOS does not behave well with a full disk.
And, even if you've remedied the swap issue (i.e. killed the process), there's no way I know to recover the swap files created without restarting.
Just seems like the design is trouble waiting to happen, and it has happened to me.
When this last happened, somehow it managed to corrupt my external Time Machine volume.
I've had my disks so full that a `rm` command doesn't even work, would this workaround work in those cases too?
> save_my_butt.img
and now it's 0 bytes.
How is this better than sounding alarms when free disk space drops below 8GB? If you’re going to ignore the alarms, then you’re going to have the same problem after you remove your spacer file and the disk fills up again!
In the mid nineties I worked in a research institute. There was a large shared Novell drive which was always on the verge of full. Almost every day we were asked to clean up our files as much as possible. There were no disc quota for some reason. One day I was working with my colleague and when the fileserver was full he went to a project folder and removed a file called balloon.txt which immediately freed up a few p…
You had a community-driven quota system.
How is this better than sounding alarms when free disk space drops below 8GB? If you’re going to ignore the alarms, then you’re going to have the same problem after you remove your spacer file and the disk fills up again!