Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All my servers have an 8 GB empty file on disk

brianschrader.com

Post reply on HN
Print view View original

681–690 of 715 posts

Re: All my servers have an 8 GB empty file on disk

#681

Post by dylan604 »

sillysaurusx wrote: post #527
dylan604 wrote: post #182

Earlier quoted context omitted.

And users have been ignoring that message as long as MacOS has been giving them. Maybe even longer

I even hacked my MacOS to disable the message. Computers shouldn't nag their owners repeatedly, even if it's in their best interest, unless the computer is about to catch fire.

Wouldn't it be better to take the system's advice and clear off some space rather than playing russian roulette?

I've had the mispleasure of pointing a large video export to the wrong drive and other misdeeds that allowed a drive to fill up. It's not pleasant. A simple reboot sometimes frees up the swap space to allow for more spring cleaning, but I typically just resort to booting into recovery mode and searching the web for the proper terminal command to decrypt/mount the root volume for spring cleaning.

Re: All my servers have an 8 GB empty file on disk

#682

Post by MayeulC »

tored wrote: post #395
scottlamb wrote: post #346

Earlier quoted context omitted.

You recall incorrectly; swap is not needed. It's not just me who runs without it; Google production machines did for many years. "The behavior is often worse without swap" is more vague / subjective. I prefer that a process die cleanly than everything slow to a crawl semi-permanently. I've previously written about swap causing the latter: https://news.ycombinator.com/item?id=13715917 To some extent the bad behavior c…

I'm surprised that the swap implementation (or distros swap configuration) is so bad on Linux. I have a fresh Ubuntu 20.10 desktop installation and a few times it has come to slow crawl where you can't do anything, can't escape to tty or ssh, reboot is basically the only option. I recently learned searching the internets that it was the swapfile configuration. My 16GB RAM machine got a 2GB swapfile from the default i…

I often went into similar issues before upgrading my RAM. SysRq+F is your friend :)

https://en.m.wikipedia.org/wiki/Magic_SysRq_key

That manually triggers OOM kill, making your system responsive again at the risk of killing something it shouldn't. At some point, I was probably using it five times a day. It got better when I started using swap (easier to recover when both full). I really recommend zram.

Re: All my servers have an 8 GB empty file on disk

#683

Post by hackmiester »

terramex wrote: post #25

Ah, the classic 'speed-up loop' approach: https://thedailywtf.com/articles/The-Speedup-Loop About the blogpost itself: The disk filled up, and that's one thing you don't want on a Linux server—or a Mac for that matter. When the disk is full nothing good happens. I had this happen few times on a Mac and every time I was shocked that if disk gets full you cannot even delete a file and the only option is to do a full sy…

I ran into this on OpenWRT back in the day. It had a similar filesystem behavior where you could not delete from a full FS. The solution was to truncate a file that was at least 1 block big, thus freeing up a few kilobytes. Then you can rm a large file, and then you can resume normal cleanup.

Re: All my servers have an 8 GB empty file on disk

#684

Post by Rule35 »

295310e0 wrote: post #57
SethTro wrote: post #31

Same idea as this game development legend https://www.dodgycoder.net/2012/02/coding-tricks-of-game-dev... > he had put aside those two megabytes of memory early in the development cycle. He knew from experience that it was always impossible to cut content down to memory budgets, and that many projects had come close to failing because of it. So now, as a regular practice, he always put aside a nice block of memory to…

If true, I hate that story. Think of the better art assets that were needlessly left behind. How is it that said block of memory had never been identified by any profiling?

Better PMs do this today by having buffer-features they can cut when needed. It'll handle the not-enough-memory issue as well as a meddlesome VP who think you're over-subscribed and wants you to cut to meet your dates.

Also, don't forget you're hearing decades-later retellings of someone else's story. I don't doubt that they trickled this extra space out as changing requirements mandated it, but that they kept from doing so until the team had actually reached a certain level of product-maturity and reclaimed all of their own waste first.

Remember that the PMs goal is to ship. Them blocking some assets but actually shipping is a success. Better 95% of the product than 0%.

Re: All my servers have an 8 GB empty file on disk

#685

Post by stjohnswarts »

innomatics wrote: post #308
badcc wrote: post #15

This trick has certainly saved me more times than I am willing to admit! I usually roll with: `fallocate -l 8G DELETE_IF_OUT_OF_SPACE.img`

Nice. I liked the trick in the article, but was wondering if it might confuse the next admin who needs to figure it out. Was also thinking of customising the login greeting to mention the file (/etc/motd).

what if you named it "THIS_IS_A_BIG_FILE_IN_CASE_THE_DISK_FILLS_UP_DELETE_IN_EMERGENCIES.BIN"

Re: All my servers have an 8 GB empty file on disk

#686

Post by jokethrowaway »

vagrantJin wrote: post #607
herbcso wrote: post #507

Earlier quoted context omitted.

I've never understood this about budgeting. So you allocate. A budget. These are fund YOU ARE PLANNING TO SPEND! So, OK, you DON'T spend them this year. Why the fuck don't you get to SAVE THAT MONEY!? No, instead you are punished for now spending it all and you cannot create a realistic budget for next year - why the hell not!? Sorry, but this frustrates the hell out of me! What am I missing here? What arcane bit of…

Your frustration may come from a good place, believing gov and big orgs want to be efficient. They don't and they don't have to. They are super efficient when it comes to taxes though. At least in my country, every department is trash except the revenue service which is so forward thinking and effective, they put private business to shame.

This definitely rings true.

But I'd say it's not a feature of governments only, any sufficiently big organisation which centralise power ends up being like this. That's why large corporates need nimble startups to innovate: startups either innovate or die.

The government is just the largest example of this phenomenon - and they don't have any analogue for startups, they're just doomed to grow larger and larger over centuries until they collapse.

Look at the USA, once the perfect minarchist experiment and now the largest employer in the world.

Re: All my servers have an 8 GB empty file on disk

#687

Post by jokethrowaway »

kaba0 wrote: post #548
sgregnt wrote: post #461

Earlier quoted context omitted.

If whatever actions turn a non government entity into something inefficient, then the entity wont survive for long and will go out of business (or at least that's the hope of a competitive free market economy)

> competitive free market economy It was not even believed by Adam Smith. He writes that it only works that way in a controlled environment. That’s why European countries usually rank higher in market freedom than the US, because we don’t have companies getting so cancerously big that they have very real effects on law making (how lobbying is legal is still beyond me)

The USA got rich because of unbridled capitalism. Then richness trickled down through generations of companies while regulations caught up and their government became a behemoth not dissimilar to the ones living in EU and that the USA was running from.

Nowadays the USA jurisdiction is comparable to the EU one, but they still have more $$$.

If you like freedom rankings I recommend these ones: https://www.heritage.org/index/ranking

Re: All my servers have an 8 GB empty file on disk

#688

Post by jokethrowaway »

eevilspock wrote: post #589
nickpp wrote: post #577

Earlier quoted context omitted.

> we need to get beyond such limited thinking. We need to transcend our base natures Refusing to accept the human nature as-is and always requiring some sort of "evolved new man" is one of the characteristics of the communist/socialist ideology. Also a handy excuse when the system inevitably fails: it wasn't the system, it was the selfish people who did not implement it correctly.

Ahhh the old "socialism/communism inevitably fails" meme. Let's assume one could even call those failures communism/socialism.[1] How long have we experimented with and developed socialism/communism? 100 years. How long have we been trying to get democracy right? 2,500 years. With many starts, fits and failures, devolving into dictatorships many, many times. The self-proclaimed "greatest democracy in history" is guil…

Capitalism is not exploiting anyone. Capitalism is purely about organising the economy around voluntary transactions.

Exploiting, enslaving and murdering is purely what socialist countries do - and they can get away with all of this, just because they can socialise the cost of all their evil deeds and force people into paying them money.

The only reason capitalism hasn't collapsed is that it's the only way to have a profitable economy. The crooks that you call government recognise that they can steal only so much from the economy before a country collapse.

I'd also argue that we've experimented with elements of socialism and elements of capitalism for the entire existence of civilisation.

Communism can't work unless you have either perfect individuals or a tyrannical states which force resources distribution. In the real world, you end up with socialism. Because people are not perfect, the government which will redistribute resources won't do a perfect job in the best case and will just be completely corrupted in the worst case.

And still, communism was attempted. When did we ever attempt to have an entire capitalist society without a government to ruin it?

Re: All my servers have an 8 GB empty file on disk

#689

Post by tytso »

axiolite wrote: post #474
tytso wrote: post #175

Earlier quoted context omitted.

If you try to use a file system to 99% full --- and it doesn't matter whether it is a 10GB file system or a 10TB file system, you will see significant performance penalties as the file system gets badly fragmented. So that's why having a fixed percentage even for massively big disks still makes sense. Disk space is cheap enough that even 5% of a 14TB disk is really not that much money --- and if you see bad performan…

> If you try to use a file system to 99% full --- and it doesn't matter whether it is a 10GB file system or a 10TB file system, you will see significant performance penalties as the file system gets badly fragmented. Not true, I've checked. I have plenty of Linux ext3 servers running for many years that routinely drop down to 1% free space for extended periods before being cleaned-up, which still have essentially zer…

Like most things, it depends on your workload. If all of the files are written all at once when they created (e.g., no slow append workloads) and the files are all the same size, then that's a very "friendly" workload that will be much less likely to suffer fragmentation. But if you are creating and deleting files that have a large range of sizes, and some files grow gradually over time, then free space fragmentation will tend to occur much more quickly.

Re: All my servers have an 8 GB empty file on disk

#690

Post by Blackthorn »

School-Cotton wrote: post #524
Blackthorn wrote: post #506

Earlier quoted context omitted.

My favorite part of that story is how the initial question about overflow should make it obvious that what they're doing doesn't work, but nobody noticed.

Is it an overflow because `int` was typically 16-bit in those days?

Yeah.
Post reply on HN