Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All my servers have an 8 GB empty file on disk

brianschrader.com

Post reply on HN
Print view View original

691–700 of 715 posts

Re: All my servers have an 8 GB empty file on disk

#691

Post by airhead969 »

Stevvo wrote: post #606
herpderperator wrote: post #473

The author seems to forget that ext-based filesystems keep 5% of disk space available for root at all times by default, known as "reserved blocks".[0] That means if a non-root user uses all of the available space, it wasn't really all of the space -- root still has access to 5% free space within that partition. That's exactly the same as the useless 8GB file but in an officially-supported manner. If you run out of di…

Sure, but one of the most common causes of disks being filled on Linux is either the kernel or some process running as root filling the disk with endless repeating log entries.

This is why on *nix systems, each directory of the vfs that is intended to accumulate files OR have different mount options is broken-off into its own volume. / /home /usr /usr/local /var /var/log /var/lib/docker /var/lib/postgresql and so on.

Re: All my servers have an 8 GB empty file on disk

#692

Post by airhead969 »

herpderperator wrote: post #625
Stevvo wrote: post #606

Earlier quoted context omitted.

Sure, but one of the most common causes of disks being filled on Linux is either the kernel or some process running as root filling the disk with endless repeating log entries.

It's a good point that the process that is flooding the disk could be running as root and therefore bypasses the reserved blocks restriction. It's also worth noting that the kernel is not a process and it does not write directly to files in the conventional sense. syslog-ng or its equivalents (which do run as root) will pick up messages from the kernel's circular ring buffer at /dev/kmsg and write them to a text file…

Log files are awful, log rotation is even worse. Logs are a continuous, persistent stream of messages that sometimes gets expunged when too old. Log files try to flatten and serialize them, but the coordination and management of them is a hassle. An FS like Reiser where each log entry is a file arranged in directory trees by minute, hour, day, month, and year, or a sqlite database would make more sense than spewing blindly into a giant text file and praying that it works when another process comes along and moves things around underneath a running program.

Re: All my servers have an 8 GB empty file on disk

#693

Post by kaba0 »

jokethrowaway wrote: post #687
kaba0 wrote: post #548

Earlier quoted context omitted.

> competitive free market economy It was not even believed by Adam Smith. He writes that it only works that way in a controlled environment. That’s why European countries usually rank higher in market freedom than the US, because we don’t have companies getting so cancerously big that they have very real effects on law making (how lobbying is legal is still beyond me)

The USA got rich because of unbridled capitalism. Then richness trickled down through generations of companies while regulations caught up and their government became a behemoth not dissimilar to the ones living in EU and that the USA was running from. Nowadays the USA jurisdiction is comparable to the EU one, but they still have more $$$. If you like freedom rankings I recommend these ones: https://www.heritage.org/…

Did that richness really trickle down? Just look at the distribution of that money.

Re: All my servers have an 8 GB empty file on disk

#694

Post by scaramanga »

jokethrowaway wrote: post #687
kaba0 wrote: post #548

Earlier quoted context omitted.

> competitive free market economy It was not even believed by Adam Smith. He writes that it only works that way in a controlled environment. That’s why European countries usually rank higher in market freedom than the US, because we don’t have companies getting so cancerously big that they have very real effects on law making (how lobbying is legal is still beyond me)

The USA got rich because of unbridled capitalism. Then richness trickled down through generations of companies while regulations caught up and their government became a behemoth not dissimilar to the ones living in EU and that the USA was running from. Nowadays the USA jurisdiction is comparable to the EU one, but they still have more $$$. If you like freedom rankings I recommend these ones: https://www.heritage.org/…

In what time period did this "unbridled capitalism" happen?

Re: All my servers have an 8 GB empty file on disk

#695

Post by wildmanx »

martyvis wrote: post #457
frosted-flakes wrote: post #364

Earlier quoted context omitted.

Is fish not considered meat?

Not in some cultures. My youngest daughter isn't a big meat eater, and certainly hates seafood. She was on an organised tour to Spain with her youth orchestra so ticked the "vegetarian" box for meals. Inevitably her meal came with fish as the protein. Apparently in Spain, fish aren't considered as sentient like chicken or hoofed animals and hence are akin to vegetables.

Which is not just lacking scientific basis but goes straight _against_ what we know today of fish.

Any vegetarian who eats fish because of that reason really needs to update their ethics to the current millennium.

Re: All my servers have an 8 GB empty file on disk

#696

Post by djmips »

SethTro wrote: post #31

Same idea as this game development legend https://www.dodgycoder.net/2012/02/coding-tricks-of-game-dev... > he had put aside those two megabytes of memory early in the development cycle. He knew from experience that it was always impossible to cut content down to memory budgets, and that many projects had come close to failing because of it. So now, as a regular practice, he always put aside a nice block of memory to…

Back in the late eighties a colleague of mine was making a game for the Atari ST and he purposely put in some time wasting code in the game loop so that he had to work against a smaller budget which gave him some contingency for later on when he needed some extra cycles.

Re: All my servers have an 8 GB empty file on disk

#697

Post by arendtio »

With virtual servers this should not be necessary, as it is easy enough to add some disk space. After all, this should not be a common issue in production environments, but more like a once in a decade problem.

With physical servers it might be a different story and might be a good idea. I tend to size filesystems to the requirements I have and enlarge them when required (it gives you a periodic reminder to think about what waste you have accumulated). That way, I can still add space even if the filesystem has been filled up. However, if you do it just to have some space when you need it, it probably is overkill and to have an empty buffer file is a lot easier to handle.

Re: All my servers have an 8 GB empty file on disk

#699

Post by EricE »

I've been doing this for years too. I also learned instead of one big file, halving several files is also useful so you can release space in "chunks". If you need everything at once you can wildcard the delete.

Such flexibility has been invaluable over the years. Thankfully with block storage and modern operating systems/file systems growing volumes can be significantly easier for most servers.

Re: All my servers have an 8 GB empty file on disk

#700

Post by EricE »

Aeolun wrote: post #383
njovin wrote: post #335

Earlier quoted context omitted.

This is similar to how some government agencies retain their budgets. At the end of the budget period they've only spent 80% of their allocated budget, so they throw out a bunch of perfectly good equipment/furniture/etc. and order new stuff so that their budget doesn't get cut the following year, rather than accepting that maybe they were over-budgeted to begin with. Rinse, repeat, thus continuing the cycle of wastin…

I think the problem is that you do not need 100% of your budget every year, but getting it back when you do need it is much harder than keeping it in the first place. Definite case of misaligned incentives.

Ding Ding Ding! They aren't misaligned, they are flat-out perverse incentives.
Post reply on HN