Live data from Hacker News

Linux NILFS file system: automatic continuous snapshots

dataswamp.org

21–30 of 158 posts

Re: Linux NILFS file system: automatic continuous snapshots

#21
post #13
post #6

Earlier quoted context omitted.

"It does without a performance penalty" yeah. it's already so terribly slow that it's unlikely that taking snapshots can make it any slower :-D

That was not my experience with NILFS. It outperformed ext4 on my laptop NVME.

The benchmarks here look pretty bad:

https://www.phoronix.com/review/linux-58-filesystems/4

Re: Linux NILFS file system: automatic continuous snapshots

#22
post #16

Earlier quoted context omitted.

I think all you would need to add is a daemon that automatically deletes the oldest file(s) whenever free space drops below a certain threshold, so that the filesystem GC can reclaim that space for new files.

I know and use 'logrotate'. My point was more on the tracks of a filesystem where a single file can be overwritten over and over again, and it's up to the filesystem to transparently ensure the full capacity of the disk is put towards retaining old versions of the file.

Hmm maybe something like Bluestore?

https://docs.ceph.com/en/latest/rados/configuration/storage-...

Re: Linux NILFS file system: automatic continuous snapshots

#23
post #4

How does this compare to ZFS + cron to create snapshots every X minutes?

There is no comparison. NILFS provides *continuous* snaphots, so you can inspect and rollback changes as needed. It does without a performance penalty compared to other logging filesystems. And without using additional space forever. The backlog rotates forward continuously. It's a really unique feature that makes a lot of sense for desktop use, where you might want to recover files that were created and deleted afte…

Perhaps we can leverage "inotify" API to make ZFS snapshot everytime some file had been changed... But i think ZFS is not really good at handling huge amounts of snapshots. The NILFS2 snapshots are probably more lightweight when compared to ZFS ones.

Re: Linux NILFS file system: automatic continuous snapshots

#24
post #18
post #12

I think NILFS is a hidden gem. I’ve been using it exclusively in my Linux laptops, desktops etc. since ca. 2014. Apart from one kernel regression bug related to NILFS2 it’s worked flawlessly (no data corruption even with the bug just no access to the file system; effectively it forced running older kernel while the bug was fixed). The continuous snapshotting has saved me a couple of times; I’ve just mounted a version…

>I think NILFS is a hidden gem. I’ve been using it exclusively in my Linux laptops, desktops etc. since ca. 2014 Yes it's really sad, there we have a native and stable check-summing fs, and nearly no one knows about it.

> check-summing fs

Is it? Last I'd heard was

> nilfs2 store checksums for all data. However, at least the current implementation does not verify it when reading.

https://www.spinics.net/lists/linux-nilfs/msg01063.html

Re: Linux NILFS file system: automatic continuous snapshots

#25
What happens if you run "dd if=/dev/zero of=/any/file/here", thus simply loading the disk with all the zeros it can handle? Do you lose all your snapshots as they are deleted to make room, or does it keep some space aside for this situation?

(Not a "gotcha" question, a legitimate question.)

Re: Linux NILFS file system: automatic continuous snapshots

#26

Earlier quoted context omitted.

There is no comparison. NILFS provides *continuous* snaphots, so you can inspect and rollback changes as needed. It does without a performance penalty compared to other logging filesystems. And without using additional space forever. The backlog rotates forward continuously. It's a really unique feature that makes a lot of sense for desktop use, where you might want to recover files that were created and deleted afte…

>It's a really unique feature that makes a lot of sense for desktop us Sounds like it could serve as a basis for a Linux implementation of something like Apple Time Machine.

[deleted]

Re: Linux NILFS file system: automatic continuous snapshots

#27
post #11

NILFS is really, really cool. In concept. Unfortunately the tooling and support just isn't there. I ran it for quite some time on my laptop and the continuous snapshoting is everything I hoped it'd be. At one point however there was a change to the kernel that rendered it unbootable. Despite being a known and recorded bug it took forever to get fixed (about a year if I recall correctly) leaving me stuck on an old ker…

How did Linus not go on a rampage after breaking userspace for an entire year? Is NILFS not part of the kernel mainline, I guess?

> Is NILFS not part of the kernel mainline, I guess?

Good guess, but no:

https://github.com/torvalds/linux/tree/master/fs/nilfs2

> How did Linus not go on a rampage after breaking userspace for an entire year?

I would very much like to know that as well. Any chance it didn't get reported (at least, not as "this broke booting")?

Re: Linux NILFS file system: automatic continuous snapshots

#28
post #22
post #16

Earlier quoted context omitted.

I know and use 'logrotate'. My point was more on the tracks of a filesystem where a single file can be overwritten over and over again, and it's up to the filesystem to transparently ensure the full capacity of the disk is put towards retaining old versions of the file.

Hmm maybe something like Bluestore? https://docs.ceph.com/en/latest/rados/configuration/storage-...

I definitely need to dive into Ceph, thanks for the pointer :-)

Re: Linux NILFS file system: automatic continuous snapshots

#29
post #14

I had issues with file locking when running some legacy database software on NILFS2. Probably caused data corruption in that database (not the FS itself). SF website of NILFS2 suggests that there are some unimplemented features, one of them being synchronous IO, which might have caused that issue? https://nilfs.sourceforge.io/en/current_status.html In some cases, the NILFS2 is safer storage for your data than ZFS. So…

> In some cases, the NILFS2 is safer storage for your data than ZFS.

What cases? Do you just mean due to continuous snapshots protecting against accidental deletes or such, or are there more "under the covers" things it fixes?

Re: Linux NILFS file system: automatic continuous snapshots

#30
post #23

Earlier quoted context omitted.

There is no comparison. NILFS provides *continuous* snaphots, so you can inspect and rollback changes as needed. It does without a performance penalty compared to other logging filesystems. And without using additional space forever. The backlog rotates forward continuously. It's a really unique feature that makes a lot of sense for desktop use, where you might want to recover files that were created and deleted afte…

Perhaps we can leverage "inotify" API to make ZFS snapshot everytime some file had been changed... But i think ZFS is not really good at handling huge amounts of snapshots. The NILFS2 snapshots are probably more lightweight when compared to ZFS ones.

The NILFS snapshots are practically free (for a logging filesystem, obviously).
Post reply on HN