Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All my servers have an 8 GB empty file on disk

brianschrader.com

Post reply on HN
Print view View original

461–470 of 715 posts

Re: All my servers have an 8 GB empty file on disk

#461

Post by sgregnt »

lostlogin wrote: post #454
njovin wrote: post #335

Earlier quoted context omitted.

This is similar to how some government agencies retain their budgets. At the end of the budget period they've only spent 80% of their allocated budget, so they throw out a bunch of perfectly good equipment/furniture/etc. and order new stuff so that their budget doesn't get cut the following year, rather than accepting that maybe they were over-budgeted to begin with. Rinse, repeat, thus continuing the cycle of wastin…

> This is similar to how some government agencies retain their budgets. The non-government sector isn’t immune to this.

If whatever actions turn a non government entity into something inefficient, then the entity wont survive for long and will go out of business (or at least that's the hope of a competitive free market economy)

Re: All my servers have an 8 GB empty file on disk

#462

Post by com2kid »

sgregnt wrote: post #461
lostlogin wrote: post #454

Earlier quoted context omitted.

> This is similar to how some government agencies retain their budgets. The non-government sector isn’t immune to this.

If whatever actions turn a non government entity into something inefficient, then the entity wont survive for long and will go out of business (or at least that's the hope of a competitive free market economy)

> then the entity wont survive for long and will go out of business (or at least that's the hope of a competitive free market economy)

Only if the inefficiency is large enough to overcome other forces.

Or to put it another way, picture if every single individual teams at Google did this to the tune of 100k a year, per team, and assume among 135,000 employees there are 13500 teams.

That's 1.35 billion dollars. Well under 1% of their revenue.

No way is a competitor going to appear that is identical to Google in every way except they have better budget management. Google has too many moats around their business, they can be really inefficient in many many ways and still dominate in multiple markets.

Re: All my servers have an 8 GB empty file on disk

#463

Post by ultimape »

sgregnt wrote: post #461
lostlogin wrote: post #454

Earlier quoted context omitted.

> This is similar to how some government agencies retain their budgets. The non-government sector isn’t immune to this.

If whatever actions turn a non government entity into something inefficient, then the entity wont survive for long and will go out of business (or at least that's the hope of a competitive free market economy)

you can survive a lot of inefficiency if you are a natural monopoly.

Re: All my servers have an 8 GB empty file on disk

#464

Post by anonymousiam »

stonesweep wrote: post #143
birdyrooster wrote: post #47

Earlier quoted context omitted.

Funny because I have always tune2fs -m1 or tune2fs -m0 because the reserved space was never supposed to scale linearly with hard drive capacities and is not useful to userspace in anyway. Have never had any issues and been doing it for decades in commercial applications. In some cases, where you probably shouldn't be using ext3/4 anyways, we are talking about reclaiming TBs of reserved space. It's important to note t…

Fun trivia: you can actually set per-device settings in /etc/mke2fs.conf with all sorts of alternate defaults. https://man7.org/linux/man-pages/man5/mke2fs.conf.5.html

More fun trivia. The original(?) man page for tunefs had an "easter egg": https://unixhistory.livejournal.com/1808.html?nojs=1

Re: All my servers have an 8 GB empty file on disk

#465

Post by akdma »

dilyevsky wrote: post #430
geocrasher wrote: post #164

Earlier quoted context omitted.

In a perfect world, this is true. But we don't have one of those.

That’s not perfect world this is standard industry practice to have host level monitoring

THANK YOU. How are so many people in this thread content with saying “monitoring isn’t perfect, this solution is ingenious”. Ofc nothing is perfect and even when you do everything right things can still go wrong, but if you don’t have a ROBUST monitoring/alert system in place then you’re not even doing the bare minimum. They’re acting like it’s rocket science to set thresholds, and have meaningful alerts and checks in place. Not to mention if you wait until disk full you risk issues like block corruption among others and your 8GB of space doesn’t do anything. It’s why people in this industry are on call, it’s why they have monitoring on their monitoring systems. The bare minimum

Re: All my servers have an 8 GB empty file on disk

#466

Post by kortex »

yabones wrote: post #112
danielrhodes wrote: post #80

My understanding is this is why one should partition a drive. If you have a data partition, a swap partition, and an OS partition, you can get around issues where a server’s lack of disk space hoses the whole system.

100% agree. I think at the bare minimum every system should have two partitions: `/` and `/var` /var is usually where the most data gets added. Logs, database files, caches, and whatever other junk your app spits out. 99% of the time that's what causes the out of space issues. By keeping that separate from root, you've saved your system from being completely hosed when it fills up (which it will). Obviously there are…

/var gets its own disk on my machine! Yay academic machine learning docker containers. What's a few 30GB docker images between friends? So yeah /var gets to live with training data on SSD which gets cold swapped onto spinny platters as needed. /home is another, on the "main sdd".

/usr isn't a bad choice, either.

Re: All my servers have an 8 GB empty file on disk

#467

Post by c3534l »

eevilspock wrote: post #447
victor9000 wrote: post #163

Earlier quoted context omitted.

Perfect example of the tragedy of the commons. If individuals don't create these balloon files then they won't be able to use the file server when they need it, yet by creating these balloon files the collective action depletes the shared resource of its main function.

Tragedy of the Commons is the libertarian "private property is essential" interpretation. It's a cynical take, assuming that human selfishness is the deepest of truths and that there is no use fighting it, that the best solution is to organize society around it. The conventional Game Theory take is that this is a prisoners dilemma, and everyone creating balloon.txt files are defecting. They are making the most ration…

Tragedy of the Commons effects are well-established to exist both in economics and outside it (in ecology, for instance). You seem to be attempting to shoehorn some misguided political take into the situation, even though Tragedy of the Commons is a decent characterization of this particular social pathology.

Re: All my servers have an 8 GB empty file on disk

#468

Post by ric2b »

eevilspock wrote: post #447
victor9000 wrote: post #163

Earlier quoted context omitted.

Perfect example of the tragedy of the commons. If individuals don't create these balloon files then they won't be able to use the file server when they need it, yet by creating these balloon files the collective action depletes the shared resource of its main function.

Tragedy of the Commons is the libertarian "private property is essential" interpretation. It's a cynical take, assuming that human selfishness is the deepest of truths and that there is no use fighting it, that the best solution is to organize society around it. The conventional Game Theory take is that this is a prisoners dilemma, and everyone creating balloon.txt files are defecting. They are making the most ration…

So other people acknowledge the problem, provide a solution, and your response is to say "that is selfish libertarian propaganda, the real solution is some magical evolution"?

Re: All my servers have an 8 GB empty file on disk

#469

Post by duxup »

flamble wrote: post #443
duxup wrote: post #363

Earlier quoted context omitted.

Government? I see that as the SOP for large companies too...

It's possible that once a company reaches a certain size, it's inevitable. Corporations internally have the same top-down centralized organizational structure as a typical government. Market forces can't eliminate that kind of inefficiency if it invariably affects all large enterprises, and the economies of scale enjoyed by such companies outweigh the perverse incentives of sub-organizations. What strikes me as uniqu…

I think the common feature is just humans.

I think we imagine a lot of market forces that no doubt exist, but people aren't logical in the face of them.

Re: All my servers have an 8 GB empty file on disk

#470

Post by ZoF »

luckylion wrote: post #170
zokier wrote: post #113

Earlier quoted context omitted.

Compare the rate an haywire process can fill up disk to your response time to alarms, and you got your answer right there.

Okay, so now you have a disk full, only become aware of it when it's full and your database throws errors. You have an easy way to fix it, just delete the spacer file. But what good does that do? You're still in the mess where your database is really unhappy. On the other hand, if your monitoring was set up well, you got a notification and had time to react to it before it was at 100%. Granted, if you have a process…

No-one writing the OP's post re: 8G of wasted disk on 'all my servers' is dealing with this in meaningful enterprise production environment(although scarily perhaps). If you have an alert process great, this buffer whilst a bit retarded compared to monitoring that you trust(e.g. your above-mentioned disk warning) is still important for him.

This is someone on a budget host who at the point of 'out of disk' either had his warning fail or his lack of care/monitoring let it go unnoticed and now has to resolve the issue. In that context on a host >100G base disk I am totally down with his solution.

This is a real person dealing with his own linux boxes.

If you've dealt with an out-of-disk system, linux or windows, you would know exactly what is meant by this.

Post reply on HN