81–90 of 141 posts
I appreciate the last line > Note: this was written fully by me, human.
> It’s difficult to reason under pressure. Experience, that I didn’t have here, would have helped.
A neat trick I was told is to always have ballast files on your systems. Just a few GiB of zeros that you can delete in cases like this. This won't fix the problem, but will buy you time and free space for stuff like lock files so you can get a working system.
A neat trick I was told is to always have ballast files on your systems. Just a few GiB of zeros that you can delete in cases like this. This won't fix the problem, but will buy you time and free space for stuff like lock files so you can get a working system.
This saved us a couple times. At least until I had time to add monitoring to their old system to track disk usage. It was also helpful to use a tool called ncdu. It helps you visualize where most disk space is getting used up to track down the problem.
Earlier quoted context omitted.
If you want to do it really quickly openssl enc -aes-256-ctr -pbkdf2 -pass pass:"$(date '+%s')" Almost all CPUs have AES native instructions so you'll be able to produce pseudorandom junk really fast. Even my old system will produce it at about 3Gb/s. Much faster than urandom can go.
That's very cool. Sadly running that exact command gets an incomplete file and error "error writing output file". It suggests adding iflag=fullblock (to dd). Running that makes a file of the correct size. But still gives "error writing output file". I suppose that occurs because dd breaks the pipe.
Easiest alternative I guess is to pipe through head. It still grumbles, but it does work
openssl enc -aes-256-ctr -pbkdf2 -pass pass:"$(date '+%s')" fooEarlier quoted context omitted.
Better fill those files with random bytes, to ensure the filesystem doesn’t apply some “I don’t actually have to store all-zero blocks” sparse-file optimization. To my knowledge no non-compressing file system currently does this, but who knows about the future.
If I recall correctly: dd if=/dev/urandom of=/home/myrandomfile bs=1 count=N
head -c 1G /dev/urandom > /home/myrandomfile
Earlier quoted context omitted.
Better fill those files with random bytes, to ensure the filesystem doesn’t apply some “I don’t actually have to store all-zero blocks” sparse-file optimization. To my knowledge no non-compressing file system currently does this, but who knows about the future.
If I recall correctly: dd if=/dev/urandom of=/home/myrandomfile bs=1 count=N
Earlier quoted context omitted.
Surely a 50% warning alarm on disk usage covers this without manual intervention?
> Surely a 50% warning alarm on disk usage covers this without manual intervention? surely you don't need a fire extinguisher in your kitchen, if you have a smoke detector? a "warning alarm" is a terrible concept, in general. it's a perfect way to lead to alert fatigue. over time, you're likely to have someone silence the alarm because there's some host sitting at 57% disk usage for totally normal reasons and they're…
Let's set a fixed threshold -- 100GB, say -- and play out both methods.
Method A: One or more ballast files are created, totalling 100GB. The machine runs out of storage and grinds to a halt. Hopefully someone notices soon or gets a generic alert that it has ceased, remembers that there's ballast files, and deletes one or more of them. They then poke it with a stick and get it going again, and set forth to resolve whatever was causing the no-storage condition (adding disk, cleaning trash, or whatever).
Method B: A specific alert that triggers with Method C: The control. We do nothing, and run out of space. Panic ensues. Articles are written.
---
Both A and B methods have an equal number of alerts for each low-disk condition (But Method A relies on a system to crash, while Method B does not rely upon a crash at all.
I think that the lack of crash makes Method B rather superior all on its own.
(Method C sucks.)
> I rushed to run du -sh on everything I could, as that’s as good as I could manage. I recently came across gdu (1) and have installed/used it on every machine since then. [1]: https://github.com/dundee/gdu
du -hs -- * .??* 2> /dev/null | sort -h | tail -$LINES
A neat trick I was told is to always have ballast files on your systems. Just a few GiB of zeros that you can delete in cases like this. This won't fix the problem, but will buy you time and free space for stuff like lock files so you can get a working system.
They fill app their mobile apps with junk data just to make the APK/IPA bigger. So if they need to push an urgent update, they won't have users that can't update because their phones are full to the brim.
I know two Italian banks that do it, Unicredit and Intesa. The latter was on the news when a user found out that one of the filler files was a burp recording [1].
[1] https://www.ilfattoquotidiano.it/2024/12/20/intesa-san-paolo... (in Italian)