71–77 of 77 posts
"If you're working with huge files or a huge number of files, use XFS" went the slogan. I was hit by this recently - with our 500ish (!!) entries in /node_modules and their dependencies of dependencies of dependencies, a standard 8GB VM formatted with ext4 just simply didn't have enough inodes for us to have two copies of our app (deployment swapover). Had to increase the size to 20GB, just to get the extra inodes. X…
You could have also tuned ext4 when you created it with mke2fs. You can specify with -N the max # of inodes desired. Granted, it's not something very many people consider when creating their filesystems; for one thing it's hard to predict how many you will need.
Things like this is why I use XFS anywhere I'm not using btrfs, or ZFS.
I've had some bad experience with btrfs. When I had to hard-reset a hung computer, some directories became unremovable and unrenameable. All else failed, and I attempted to --repair, it did so much damage that I had to reformat the disk. This was probably about 3-4 years ago. Have you had such issues on your preferred filesystems in the last couple of years? The reason I stay with ext4 is that it has proven amazingly…
Earlier quoted context omitted.
I've had some bad experience with btrfs. When I had to hard-reset a hung computer, some directories became unremovable and unrenameable. All else failed, and I attempted to --repair, it did so much damage that I had to reformat the disk. This was probably about 3-4 years ago. Have you had such issues on your preferred filesystems in the last couple of years? The reason I stay with ext4 is that it has proven amazingly…
In my experience XFS is rock solid. We use it almost everywhere at work. I've never had an issue with hard reboots with it. Only recently started playing with btrfs, so can't speak to it's reliability from personal experience.
Earlier quoted context omitted.
I literally just solved an issue this weekend that was inode exhaustion: there were years of snapshots and traces laying around and the DBAs were confused as to why the database couldn't create a directory to start and was exiting with "No space left on device" when there was 5% storage free. That lead to the second discussion of why you don't fix it by piping filenames one by one to rm, but either use find's inbuilt…
Is that second discussion just a performance thing, or are there filesystem implications for one or the other?
I came to the same conclusion about sharding files from the get go. I had a similar experience where my host wouldn't accept to synchronize a directory with more than 5000 files in it. (Software producing the files is my own note-taking application where each page is stored as an individual file to speed up backup of only the pages that changes).
Wouldn't 5000 files of note-taking easily fit into RAM? Why bother dealing with thousands of small files? Also, rsync does quite a good job with intransferring multi-MB (or even larger) files efficiently while copying only what's actually changed.
Earlier quoted context omitted.
In my experience XFS is rock solid. We use it almost everywhere at work. I've never had an issue with hard reboots with it. Only recently started playing with btrfs, so can't speak to it's reliability from personal experience.
I hard-reboot about once a week, caused by my carelessness and running out of 16GB of RAM. I always recommend against btrfs for system partitions, and if you don't make frequent backups, against user partitions too. Read through "man btrfsck", that should caution you sufficiently. I'll try out XFS on an external HDD I'll buy soon.
Some reading if you're not familiar with sysrq: https://en.wikipedia.org/wiki/Magic_SysRq_key Depending on your distro you might have to enable it, I believe ubuntu (for some dumb reason) ships with emergency sync as the only command available.
Enabling it should amount to adding: kernel.sysrq=1 to /etc/sysctl.conf on older machines or editing /etc/sysctl.d/10-magic-sysrq.conf on newer ones.
also if you are running out of ram that often you might want to consider a swap partition. Or if you have one already try setting a more aggressive swapiness value.