I've been running NILFS2 on my main work NAS for 8 years. It never failed us :)
Linux NILFS file system: automatic continuous snapshots
31–40 of 158 posts
Re: Linux NILFS file system: automatic continuous snapshots
#32Earlier quoted context omitted.
How did Linus not go on a rampage after breaking userspace for an entire year? Is NILFS not part of the kernel mainline, I guess?
> Is NILFS not part of the kernel mainline, I guess? Good guess, but no: https://github.com/torvalds/linux/tree/master/fs/nilfs2 > How did Linus not go on a rampage after breaking userspace for an entire year? I would very much like to know that as well. Any chance it didn't get reported (at least, not as "this broke booting")?
I reported it along with a few other users in https://marc.info/?l=linux-nilfs&m=157540765215806&w=2. I think it just isn't widely enough used that Linus noticed we were broken. If I recall correctly it also wasn't directly fixed so much as incidentally. I just kept checking new kernel versions as they were released until one worked. There was never anything in the change-log (that I recall) about fixing the bug, just another change that happened to fix the issue.
Edit: Looking through the archives, it looks like my memory was somewhat uncharitable. It was reported in November and directly patched in June (https://marc.info/?l=linux-nilfs&m=159154670627428&w=2) so about 7 months after reporting. Not sure what kernel release that would've landed in, so could've been closer to 8.
Re: Linux NILFS file system: automatic continuous snapshots
#33How does this compare to ZFS + cron to create snapshots every X minutes?
There is no comparison. NILFS provides *continuous* snaphots, so you can inspect and rollback changes as needed. It does without a performance penalty compared to other logging filesystems. And without using additional space forever. The backlog rotates forward continuously. It's a really unique feature that makes a lot of sense for desktop use, where you might want to recover files that were created and deleted afte…
[deleted]
Re: Linux NILFS file system: automatic continuous snapshots
#34NILFS is really, really cool. In concept. Unfortunately the tooling and support just isn't there. I ran it for quite some time on my laptop and the continuous snapshoting is everything I hoped it'd be. At one point however there was a change to the kernel that rendered it unbootable. Despite being a known and recorded bug it took forever to get fixed (about a year if I recall correctly) leaving me stuck on an old ker…
How did Linus not go on a rampage after breaking userspace for an entire year? Is NILFS not part of the kernel mainline, I guess?
If I understand correctly, I don't think this is a userspace-breaking bug, as in: a kernel API changed and made a userspace program not work anymore.
It is a bug that prevents the kernel from booting. That's bad, but that's not the same thing. That's not a userspace compatibility issue such as the ones Linus chases. The user space isn't even involved if the kernel cannot boot. Or if it is actually a userspace program that causes a kernel crash, it is a crash, which is not really the same thing as an API change (one could argue, but that's a bit far-fetched, the intents are not the same, etc - I don't see Linus explode on somebody who introduced a crash the way he would explode on someone changing a userspace API).
Re: Linux NILFS file system: automatic continuous snapshots
#35What happens if you run "dd if=/dev/zero of=/any/file/here", thus simply loading the disk with all the zeros it can handle? Do you lose all your snapshots as they are deleted to make room, or does it keep some space aside for this situation? (Not a "gotcha" question, a legitimate question.)
the garbage collector daemon will delete older checkpoints beyond the preserve time to make some room.
Re: Linux NILFS file system: automatic continuous snapshots
#36What happens if you run "dd if=/dev/zero of=/any/file/here", thus simply loading the disk with all the zeros it can handle? Do you lose all your snapshots as they are deleted to make room, or does it keep some space aside for this situation? (Not a "gotcha" question, a legitimate question.)
It's configurable: https://nilfs.sourceforge.io/en/man5/nilfs_cleanerd.conf.5.h.... Cleanerd is responsible for maintaining a certain amount of free space on the system, and you can control the rules for doing so (e.x. a checkpoint won't be eligible for being cleaned until it is 1 week old).
It's also worth knowing NILFS2 has checkpoints and snapshots. What you actually get are continuous "checkpoints". These can be upgraded to snapshots at any time with a simple command. Checkpoints are garbage collected, snapshots are not (until they are downgraded back into checkpoints).
Re: Linux NILFS file system: automatic continuous snapshots
#37NILFS is really, really cool. In concept. Unfortunately the tooling and support just isn't there. I ran it for quite some time on my laptop and the continuous snapshoting is everything I hoped it'd be. At one point however there was a change to the kernel that rendered it unbootable. Despite being a known and recorded bug it took forever to get fixed (about a year if I recall correctly) leaving me stuck on an old ker…
How did Linus not go on a rampage after breaking userspace for an entire year? Is NILFS not part of the kernel mainline, I guess?
> How did Linus not go on a rampage after breaking userspace for an entire year?
Linus' commandment about not breaking userspace is frequently misunderstood. He wants to ensure that user-space /programs/ do not break (even if they rely on buggy behavior that made it into a release), not that the /user/ will never see any breakage of the system whatsoever, which is of course an impossible goal. Device drivers and filesystems are firmly system-level stuff, bugs and backwards-incompatible changes in those areas are regrettable but happen all the same.
Re: Linux NILFS file system: automatic continuous snapshots
#38Earlier quoted context omitted.
>I think NILFS is a hidden gem. I’ve been using it exclusively in my Linux laptops, desktops etc. since ca. 2014 Yes it's really sad, there we have a native and stable check-summing fs, and nearly no one knows about it.
> check-summing fs Is it? Last I'd heard was > nilfs2 store checksums for all data. However, at least the current implementation does not verify it when reading. https://www.spinics.net/lists/linux-nilfs/msg01063.html
Hmm you could be right, i found nothing about that it is calculated at read-time. Just with fsck.
Re: Linux NILFS file system: automatic continuous snapshots
#39Does NILFS do checksums and snapshotting for every single file in the system? One of my biggest complaints about file systems in general is that they are all designed to treat every file the exact same way.
We now have storage systems (even SSDs) that are big enough to hold hundreds of millions of files. Those files can be a mix of small files, big files, temp files, personal files, and public files. Yet every file system must treat your precious thesis paper the same way it treats a huge cat video you downloaded off the Internet.
We need some kind of 'object store' where each object can be given a set of attributes that govern how the file system treats it. Backup, encryption, COW, checksums, and other operations should not be wasted on a bunch of data that no one really cares about.
I have been working on a kind of object file system that addresses this problem.
Re: Linux NILFS file system: automatic continuous snapshots
#40Earlier quoted context omitted.
That was not my experience with NILFS. It outperformed ext4 on my laptop NVME.
The benchmarks here look pretty bad: https://www.phoronix.com/review/linux-58-filesystems/4
The last page looks pretty bad. If you look at the others it's more of a mixed bag, but yeah.
I don't remember what benchmark I ran before deciding to run it on my laptop. Given my work at the time probably pgbench, but I couldn't say for sure. It was long enough ago I also might've been benchmarking against ext3, not 4.