All my servers have an 8 GB empty file on disk • Hacker News Board

Re: All my servers have an 8 GB empty file on disk

#251

Post by frenchy » Thu, Mar 25, 2021, 9:40 PM UTC

xupybd wrote: post #146

legulere wrote: post #125

[flagged]

This is the opposite of private property ownership. Property ownership comes at a cost. That cost increases as supply dries up. In this case there is no cost to the individual, only to the group.

> Property ownership comes at a cost.

Sort-of. Typically there is a cost associated with getting rights over a property, either by manufacturing, or in the case of land, by purchase or through the efforts of settling. However, once you have ownership of a property, it's usually relatively cheap to continue own (except, I suppose, if you consider the risk of a communist revolution or something).

This is esentially how the British monarchy earns their generous sums of money. They stole land from the Brits during the Norman conquest about 1000 years ago, and now they rent it back to them for a handsome profit (though the whole thing is rather complicated now and they only get a portion of the money).

Re: All my servers have an 8 GB empty file on disk

#252

Post by MayeulC » Thu, Mar 25, 2021, 9:42 PM UTC

benibela wrote: post #234

I have a dual-boot laptop with windows and linux, and use the ntfs partition to share data between them Recently, I extracted a large archive with Linux on the ntfs, and the partition was full Then Windows did not start anymore Linux would only mount the partition as read-only, because it was marked dirty after the failed start. Finally I found a tool to reset the mark, and delete the files. Now Windows starts again,…

> Finally I found a tool to reset the mark, and delete the files.

fsck?

Re: All my servers have an 8 GB empty file on disk

#253

Post by robin21 » Thu, Mar 25, 2021, 9:42 PM UTC

This is a great idea. Hit this so many times.

Re: All my servers have an 8 GB empty file on disk

#254

Post by driverdan » Thu, Mar 25, 2021, 9:42 PM UTC

dnautics wrote: post #193

0_____0 wrote: post #169

Earlier quoted context omitted.

always fun when you're barreling down the highway and the engine starts to lean out, prompting you to hurriedly locate and switch the petcock over before the engine stalls completely. suppose then that you go fill up and forget to set the petcock back to normal. 8ball says: "I see a long walk in your future."

Surprised there isn't a mechanism that mechanically switches the petcock over when you put a fuel nozzle up to the port

Some newer bikes, like mine, don't have a reserve petcock. They have a low fuel light. No forgetting about the petcock and an obvious warning light instead of sputtering.

Re: All my servers have an 8 GB empty file on disk

#255

Post by tapland » Thu, Mar 25, 2021, 9:43 PM UTC

the-rc wrote: post #243

rgj wrote: post #79

In the mid nineties I worked in a research institute. There was a large shared Novell drive which was always on the verge of full. Almost every day we were asked to clean up our files as much as possible. There were no disc quota for some reason. One day I was working with my colleague and when the fileserver was full he went to a project folder and removed a file called balloon.txt which immediately freed up a few p…

At the opposite end, I heard a story of actually full storage from the beginning of the century, when I worked at a "large medical and research institution in the Midwest". They had expensive SMB shares (NetApp?) that kept getting full all the time. So they did the sane thing in the era of Napster: they started deleting MP3 files, with or without prior warning. Pretty soon, they got an angry call that music could not…

D: I like getting to listen to something in the MRI though.

Re: All my servers have an 8 GB empty file on disk

#256

Post by frenchie14 » Thu, Mar 25, 2021, 9:44 PM UTC

tshaddox wrote: post #99

How is this better than sounding alarms when free disk space drops below 8GB? If you’re going to ignore the alarms, then you’re going to have the same problem after you remove your spacer file and the disk fills up again!

This would work if you have sufficient time between alarm and failure. If some issue or process uses up all of your available disk space in a short time span, you won't have that luxury. Hopefully, the author is using alerts on top of having this failsafe

Re: All my servers have an 8 GB empty file on disk

#257

Post by hikarudo » Thu, Mar 25, 2021, 9:45 PM UTC

aidenn0 wrote: post #123

This won't work with ZFS, as it may be impossible to delete a file on ZFS when disk is full. The equivalent in ZFS is to create an empty dataset with reserved space.

Thanks, I've been wondering about the "proper" way of doing this in ZFS and this method hadn't come up in my searches.

Re: All my servers have an 8 GB empty file on disk

#258

Post by zepearl » Thu, Mar 25, 2021, 9:45 PM UTC

tyingq wrote: post #20

Careful how you create it. Several ways to create large files can make a sparse file, which I don't think removing will actually help later.

I think that in the past I saw that when creating a file with e.g. ...

  dd if=/dev/zero of=deleteme.file bs=1M count=8196

...the "free space" shown by "df" slowly decreased while the file was being created, but then once the operation completed that "free space" magically went back to its original value => the big existing file (full of "0"s) was basically not using any storage.

Is this what you mean?

I just tried to replicate this behaviour but, dammit, I cannot demonstrate that right now as the behaviour so far was the expected one (free storage decreasing when creating the file and sticking to that even after the completion of the operation).

I strongly believe that that's what I saw in the past (when I was preallocating image files to then be used by KVM VMs), but now I'm wondering if I'm imagining things... :P

EDIT: this happened when using ext4 and/or xfs (don't remember) without using any compression.

Re: All my servers have an 8 GB empty file on disk

#259

Post by davidmoffatt » Thu, Mar 25, 2021, 9:46 PM UTC

Dumb idea. Read the man page for tunefs. The file system has some thing called min free which does the same thing. However this does not interfer with wear leveling. Dummy data does.

Re: All my servers have an 8 GB empty file on disk

#260

Post by berkes » Thu, Mar 25, 2021, 9:48 PM UTC

dilyevsky wrote: post #153

geocrasher wrote: post #124

For everyone saying "This isn't a real solution!" I'd like to explain why I think you're wrong. 1) It's not intended to be a Real Solution(tm). It's intended to buy the admin some time to solve the Real Issue. 2) Having a failsafe on standby such as this will save an admin's butt when it's 2am and PagerDuty won't shut up, and you're just awake enough to apply a temp fix and work on it in the morning. 3) Because "FIX…

Setup proper monitoring and never get to the Real Issue to begin with. These sysadmin hacks are not helpful

"proper monitoring" is extremely broad. And, I would say, almost unreachable goal.

You have it mail you when it goes over 80% disk usage (and what if you are on holiday)? Does it mail all colleagues? Who picks it up (I thought Bob picked it up, but Bob thought Anne picked it up. So no one did)? Does it come and wake you in person when it reaches 92%?

Will this catch this async job that fails (but should never) in an endless loop but keeps creating 20MB json files as fast as the disk allows it to?

Is it an alerting that finds anomalies in trends? Will it be fast enough for you to come online before that job has filled the disk?

I've been doing a lot of hosting management and such. And there is one constant: all unforeseen issues are unforeseen.