Live data from Hacker News

All my servers have an 8 GB empty file on disk

brianschrader.com

81–90 of 715 posts

Re: All my servers have an 8 GB empty file on disk

#81
post #69

Showing off that i’m not a sysadmin, but wouldn’t a monitoring daemon work? Once disk usage grows past a certain uncomfortable threshold you get an email/notification to see what’s up. I mean you obviously are monitoring other server vitals anyway right?

Monitoring is a good idea, regardless. However, there are cases where a bug or some other issue can cause disk usage to ramp too quickly for someone to respond to an alert.

Re: All my servers have an 8 GB empty file on disk

#82
post #3

hope you're not running -o compress=lz4 , because you are going to be in for a big surprise when you try to pull this emergency lever! you may be shocked to see you don't actually get much space back! i do wonder how many FS would actually allocate the 8GB if you, for example, opened a file, seeked to 8GB mark, and wrote a character. many file systems support "sparse files"[1]. for example on btrfs, i can run 'dd if=…

Before reading this, I had presumed that sparse files did not overcommit drive space, but apparently, they do. I don't use them regularly and certainly not to "reserve disk space" but I was surprised that you can make sparse files way larger than available free space on the drive. I had assumed they were simply not initialized, but the FS still required amount of free space in case a block is accessed.

Re: All my servers have an 8 GB empty file on disk

#84
post #31

Same idea as this game development legend https://www.dodgycoder.net/2012/02/coding-tricks-of-game-dev... > he had put aside those two megabytes of memory early in the development cycle. He knew from experience that it was always impossible to cut content down to memory budgets, and that many projects had come close to failing because of it. So now, as a regular practice, he always put aside a nice block of memory to…

If true, I hate that story. Think of the better art assets that were needlessly left behind. How is it that said block of memory had never been identified by any profiling?

> Think of the better art assets that were needlessly left behind.

Consider how long it takes to edit or recreate art assets to reduce their size. Depending on the asset, you might be basically starting over from scratch. Rewriting code to reduce its size is likely to be an even worse option, introducing new bugs and possibly running slower to boot. At least smaller, simpler art assets are likely to render faster.

This is also the kind of problem that's more likely to occur later in the schedule, when time is even more scarce. Between these two factors (lack of time and amount of effort required to get art assets which are both decent looking and smaller), I think in practice you're actually more likely to get better quality art assets by having an artificially reduced memory budget from the outset.

Re: All my servers have an 8 GB empty file on disk

#86
post #37

This is like carrying around a pound of beef because you refuse to look up the address of a McDonald's 7 minutes away. Setup quotas or implement some damn monitoring -- if you're not monitoring something as simple and critical as disk usage, what else are you not monitoring?

Not all environments require a stringent SLA. I have some servers that don't have a stringent SLA and aren't worth being woken up at night over if their disk is filling up fast.

Re: All my servers have an 8 GB empty file on disk

#88
post #74
post #54

This points to a much more serious problem. This is 2021 and the technology is from the 90s, with a really poor user experience design. Your car warns you when you're low on fuel, but your server doesn't if you're low on critical resources.

Assuming we're talking about VMs (2021 etc.), for a SME is there any downside to giving 2TB of space to your discs and let dynamic allocation do the work? Perhaps consolidate/defrag once a year. Even monitoring total usage more often than that is probably not worth the effort - just buy ample cheap storage. Also, there was a tradition to split drives into OS, DB, DB Logs. That was mostly a rust performance thing and…

One VM using excessively more disk space than it's supposed to can potentially cause data corruption in all the other VMs on that system. For just spinning VMs up and down for testing, you probably won't run into that issue, but on a production system, it could potentially cause some massive downtime

Re: All my servers have an 8 GB empty file on disk

#89
All my servers have an alarm when disk space goes above 70%. It sends an email every hour once the disk usage goes above 70%. Never had a server go down because of disk space issue after adopting this practise.

Also one of the main reasons server disks go full is generally log files. Always remember to "logrotate" your log files and you will not have this issue that much.

Yes one more thing, for all user uploaded files use external storage like NFS or S3.

Post reply on HN