Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All my servers have an 8 GB empty file on disk

brianschrader.com

Post reply on HN
Print view View original

641–650 of 715 posts

Re: All my servers have an 8 GB empty file on disk

#641

Post by ryoshu »

njovin wrote: post #335
victor9000 wrote: post #163

Earlier quoted context omitted.

Perfect example of the tragedy of the commons. If individuals don't create these balloon files then they won't be able to use the file server when they need it, yet by creating these balloon files the collective action depletes the shared resource of its main function.

This is similar to how some government agencies retain their budgets. At the end of the budget period they've only spent 80% of their allocated budget, so they throw out a bunch of perfectly good equipment/furniture/etc. and order new stuff so that their budget doesn't get cut the following year, rather than accepting that maybe they were over-budgeted to begin with. Rinse, repeat, thus continuing the cycle of wastin…

This happens in private industry too. I can set my watch by the fiscal calendar of certain groups in public companies having to spend their budgets by the end of their year so it doesn't get cut the next year.

Re: All my servers have an 8 GB empty file on disk

#642

Post by glacials »

On the chat team at Twitch in the early days after the Twitch Plays Pokemon crisis [1], we started artificially doubling all chat traffic through our systems, then dropping the doubles just before they would be sent to users. [2]

Not only did it give us a "big red button" to press during emergencies like OP, but it revealed important logical scaling issues before they became real problems.

[1]: tldr; 1 million people playing a single game of Pokemon Red by using chat to send button presses

[2]: https://www.twitch.tv/videos/93572955

Re: All my servers have an 8 GB empty file on disk

#643

Post by not2b »

MereInterest wrote: post #460
dahdum wrote: post #412

Earlier quoted context omitted.

Archiving is likely solving the wrong problem, for legal reasons they don’t want those old emails hanging around.

Sounds like the retention policy is also solving the wrong problem. If for legal reasons you want to destroy any potential evidence, maybe it's a good idea to stop doing illegal actions.

Thanks to patents, everyone in technology is doing "illegal actions" all the time, since you can't do anything without infringing hundreds of patents. And if you can find an email somewhere indicating that someone knows that a competitor has feature X, or knows about the existence of a patent, viola, evidence of knowing infringement! Triple damages under US law.

Re: All my servers have an 8 GB empty file on disk

#645

Post by osrec »

Gravityloss wrote: post #583
osrec wrote: post #406

Earlier quoted context omitted.

Reminds me of this episode of The office: https://en.m.wikipedia.org/wiki/The_Surplus

Office always had such a weird version of what happens in an"office". Having a secretary who people ask to take photocopies? Sounds like the eighties!

I think that's sort of the point. The branch was run by a guy (Michael Scott) that is actually pretty old school and utterly unaware of it.

Re: All my servers have an 8 GB empty file on disk

#646

Post by nybble41 »

abduhl wrote: post #624
littlestymaar wrote: post #579

Earlier quoted context omitted.

> 1) separation of duty: you might not be the best department to invest surplus But with this reasoning there is no surplus, because departments will spend their money at all cost. > 2) cost effectiveness: if you're operating with a deficit, as is generally the case with governments these days, this money is not free, so it could effectively be cheaper to give it back and re-borrow it when you actually need it That's…

Top management doesn’t “owe” them any money when they need it later. Say you budget $100 for dinner tonight and you go out and it costs $75. Do you owe the restaurant $25? While certainly some people might roll the $25 into the next day’s meals, some people might allocate that $25 to another cost center like buying a new car. Budgets are meant to estimate costs and manage cash flow. From a greedy team perspective it’…

> Say you budget $100 for dinner tonight and you go out and it costs $75. Do you owe the restaurant $25?

No, but if you accurately forecast that dinner will cost $100 on average, and this time it only happened to cost $75, you should put most of the savings aside for the other times when it will cost $125 and not reallocate it to be spent on something else.

Consistent over-budgeting is still an issue which would need to be addressed, of course, but a system where any annual cost underrun is treated as over-budgeting and punished by reallocating that part of the budget to other groups ignores the inevitable presence of risk in the budget forecast.

Re: All my servers have an 8 GB empty file on disk

#647

Post by bombcar »

AYBABTME wrote: post #532

Once upon a time, I wanted to cache large and expensive to pull files on many thousands of servers. Problem is the disk space on these servers was at premium and meant to be sold for customer use. The servers did have scratch space on small disks, but that was used by the OS. So I wrote an on-disk cache system that would monitor disk usage, and start to evict and shrink its disk space usage. It would take up to N gig…

Box Drive and OneDrive apps for desktop do this now, but with cached files from the "cloud".

It looks to you like everything is there, but in reality it downloads as you click on things, and empties cache if the free space begins to drop below a set amount.

Re: All my servers have an 8 GB empty file on disk

#648

Post by Mauricebranagh »

ABeeSea wrote: post #300
twistedpair wrote: post #261

Earlier quoted context omitted.

I worked at a large company during a migration from Lotus to Outlook. We were told we'd get our current Lost email storage + 100MiB as a new email quota limit under Outlook. I made a bunch of 100MiB files of `/dev/random` noise (so they don't compress, compressed size was part of the quota) and emailed them to myself before the migration, to get a few GiB of quota buffer. My co-workers were constantly having to delet…

Email quotas aren’t just a cost thing. It forces deletion of files/communications that aren’t relevant anymore. The last thing the legal department wants is some executive’s laptop with 10 years of undeleted email to make it’s way to discovery.

I am sure "legal" might want it but is it not better for society in general if they where discoverable.

A bit like when investigating police/government misconduct and a lot of files turn out to have been destroyed - but of course our data gets kept forever

Re: All my servers have an 8 GB empty file on disk

#649

Post by nybble41 »

dkarras wrote: post #600
eevilspock wrote: post #447

Earlier quoted context omitted.

Tragedy of the Commons is the libertarian "private property is essential" interpretation. It's a cynical take, assuming that human selfishness is the deepest of truths and that there is no use fighting it, that the best solution is to organize society around it. The conventional Game Theory take is that this is a prisoners dilemma, and everyone creating balloon.txt files are defecting. They are making the most ration…

This is self conflicting. You take "human selfishness is the deepest of truths" as a mere assumption, then you say "we need to transcend our base natures". Human selfishness IS nature. It is not just about humans either, all evolution is guided by environment (resource availability). For anything else you need ALL people to NOT be selfish, only some being altruistic does not cut it. Your only other option is to punis…

The way to resolve this particular tragedy of the commons, like most other such cases, is to privatize the commons: make people pay for the disk space they use. If you want a nice big balloon.txt file to reserve space for the future, fine, but you're paying for the space you reserved. How you use it is up to you. In return, the administrators get both the money and incentive they need to buy more storage capacity, ensuring that running out of available space will be less of a concern.

Re: All my servers have an 8 GB empty file on disk

#650

Post by Mauricebranagh »

Cullinet wrote: post #422
DevKoala wrote: post #266

Earlier quoted context omitted.

In California you pay ~1.5% in property taxes per year. Owning property is not cheap. In fact you never really own it.

I'm a brit who was only discouraged from the process of becoming a American (when much younger and on a terrific upwards trajectory which crazy events eventually skewered) and I would be hurt particularly by property tax now and actually the moment my income ceased growing. I can't remember if Florida doesn't have property tax and that then may explain why so many of my family retired there, but everywhere I wanted t…

For information your property tax (council tax) for the most expensive house in Central London can be less than an average 4 bed house in a mid ranked US state.
Post reply on HN