All my servers have an 8 GB empty file on disk • Hacker News Board

Re: All my servers have an 8 GB empty file on disk

#181

Post by geocrasher » Thu, Mar 25, 2021, 9:02 PM UTC

0_____0 wrote: post #169

tehjoker wrote: post #141

Earlier quoted context omitted.

This reminds me of the reserve tank toggle on some motorcycles. When you run out of gas, you switch the toggle and drive directly to a gas station.

always fun when you're barreling down the highway and the engine starts to lean out, prompting you to hurriedly locate and switch the petcock over before the engine stalls completely. suppose then that you go fill up and forget to set the petcock back to normal. 8ball says: "I see a long walk in your future."

I once put a new fuel pump in a Chevy pickup with two tanks on the side of the road because I was switched to the empty tank. Good times.

Re: All my servers have an 8 GB empty file on disk

#182

Post by dylan604 » Thu, Mar 25, 2021, 9:02 PM UTC

ghostly_s wrote: post #95

terramex wrote: post #25

Ah, the classic 'speed-up loop' approach: https://thedailywtf.com/articles/The-Speedup-Loop About the blogpost itself: The disk filled up, and that's one thing you don't want on a Linux server—or a Mac for that matter. When the disk is full nothing good happens. I had this happen few times on a Mac and every time I was shocked that if disk gets full you cannot even delete a file and the only option is to do a full sy…

MacOS has given the user nagging "startup disk is almost full " prompts for as long as I can remember, yours doesn't?

And users have been ignoring that message as long as MacOS has been giving them. Maybe even longer

Re: All my servers have an 8 GB empty file on disk

#183

Post by cmeacham98 » Thu, Mar 25, 2021, 9:02 PM UTC

londons_explore wrote: post #148

mamon wrote: post #135

Earlier quoted context omitted.

But now we have Docker, which means all the containerized workflows will run as root....

I suspect you need to be root in the root user namespace... So docker doesn't get this special power...

Docker does not use user namespaces by default (and some features are unavailable when using them).

Re: All my servers have an 8 GB empty file on disk

#184

Post by dmingod666 » Thu, Mar 25, 2021, 9:02 PM UTC

diego wrote: post #54

This points to a much more serious problem. This is 2021 and the technology is from the 90s, with a really poor user experience design. Your car warns you when you're low on fuel, but your server doesn't if you're low on critical resources.

Exactly, it's 1990s 'cool' - the time it took him to write the blog, he could have written a script that would send him updates on all his devices...

Re: All my servers have an 8 GB empty file on disk

#185

Post by ineedasername » Thu, Mar 25, 2021, 9:02 PM UTC

dmingod666 wrote: post #152

It sounds 'cool' and all for 1995.. but, what about one script that'll email you when the disk is at 80%?

So that if by the time you get the email the issue is at 97% you can immediately give yourself enough breathing room to figure things out with downtime or significantly degraded performance.

Re: All my servers have an 8 GB empty file on disk

#186

Post by dylan604 » Thu, Mar 25, 2021, 9:03 PM UTC

dmingod666 wrote: post #152

It sounds 'cool' and all for 1995.. but, what about one script that'll email you when the disk is at 80%?

Besides immediately after first spin-up, when is a drive not at 80% capacity?

Re: All my servers have an 8 GB empty file on disk

#187

Post by luckylion » Thu, Mar 25, 2021, 9:03 PM UTC

smarx007 wrote: post #151

luckylion wrote: post #131

Earlier quoted context omitted.

> 1) It's not intended to be a Real Solution(tm). It's intended to buy the admin some time to solve the Real Issue. If you don't have monitoring, will you even be aware that your disk is filling up? If you do have monitoring, why are you artificially filling up your disk so that it will be at 100% more quickly instead of just setting your monitoring up to alert you when it's at $whateverItWasSetToMinusEightGB?

An extra failsafe? You can do both. What if your cron/netdata are not forwarding emails for some reason (eg nullmailer gets errors from Mailgun)?

Right, but again, what good does the spacer file do if you're not aware that you're running low on disk space? That is: if your monitoring isn't working, how do you know that you need to quickly make room?

And if your monitoring is working correctly, the spacer file really serves no purpose other than lowering the available disk space.

Re: All my servers have an 8 GB empty file on disk

#188

Post by jessaustin » Thu, Mar 25, 2021, 9:03 PM UTC

0_____0 wrote: post #169

tehjoker wrote: post #141

Earlier quoted context omitted.

This reminds me of the reserve tank toggle on some motorcycles. When you run out of gas, you switch the toggle and drive directly to a gas station.

always fun when you're barreling down the highway and the engine starts to lean out, prompting you to hurriedly locate and switch the petcock over before the engine stalls completely. suppose then that you go fill up and forget to set the petcock back to normal. 8ball says: "I see a long walk in your future."

IME it doesn't take too many hikes to learn that part of the procedure for turning off the engine is "turn the fuel switch off reserve".

Re: All my servers have an 8 GB empty file on disk

#189

Post by nemo1618 » Thu, Mar 25, 2021, 9:03 PM UTC

nijave wrote: post #166

The real question... Why does Linux or at least the common filesystems get stuck so easily running out of disk space? Surely normal commands like `rm` should still function.

> Surely normal commands like `rm` should still function

They do. In my experience, the only disruption to most terminal operations is that tab completion will fail with an error.

Re: All my servers have an 8 GB empty file on disk

#190

Post by woko » Thu, Mar 25, 2021, 9:04 PM UTC

buffet_overflow wrote: post #140

legulere wrote: post #125

[flagged]

I still know people that pre-emptively buy toilet paper "because hoarders might buy the rest of it" with absolutely no introspection.

Hoarders will buy the rest of it.

During the 2-month lockdown a year ago, I would purchase 4 frozen pizzas at a time when I had a chance to buy them, because I was so upset that I could not buy one when I wanted a single one during the 2 previous weeks, because of hoarders who had been faster than me.

People think of toilet papers, but it is not just that. Pasta, rice, flour, yeast, plenty of useful things went missing due to hoarders during lockdown. Thankfully, I don't eat meat, because there was no meat at the supermarkets. Fish was also hard to find in the frozen shelves of the supermarkets. I am not a big fish eater either, so it was fine for me as well, but you get the idea: if you don't hoard a tiny bit, you get increasingly frustrated because hoarders will hoard, and you will have to wait for weeks before you get the chance to have what you want.