Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux NILFS file system: automatic continuous snapshots

dataswamp.org

Post reply on HN
Print view View original

71–80 of 158 posts

Re: Linux NILFS file system: automatic continuous snapshots

#71

Post by heavyset_go »

didgetmaster wrote: post #39

Does NILFS do checksums and snapshotting for every single file in the system? One of my biggest complaints about file systems in general is that they are all designed to treat every file the exact same way. We now have storage systems (even SSDs) that are big enough to hold hundreds of millions of files. Those files can be a mix of small files, big files, temp files, personal files, and public files. Yet every file s…

You can turn off CoW, checksumming, compression, etc at the file and directory levels using btrfs.

Re: Linux NILFS file system: automatic continuous snapshots

#72

Post by heavyset_go »

compsciphd wrote: post #59

we used NILFS 15 years ago in dejaview - https://www.cs.columbia.edu/~nieh/pubs/sosp2007_dejaview.pdf We combined nilfs + our process snapshotting tech (we tried to mainline it, but it didn't go, but many of the concepts ended up in CRIU though) + our remote display + screen reading tech (i.e. normal APIs) to create an environment that could record everything you ever saw visually and textually. enable you to search…

This is cool, thanks for sharing it.

Re: Linux NILFS file system: automatic continuous snapshots

#74

Post by mike256 »

pkulak wrote: post #43
goodpoint wrote: post #5

Earlier quoted context omitted.

There is no comparison. NILFS provides *continuous* snaphots, so you can inspect and rollback changes as needed. It does without a performance penalty compared to other logging filesystems. And without using additional space forever. The backlog rotates forward continuously. It's a really unique feature that makes a lot of sense for desktop use, where you might want to recover files that were created and deleted afte…

> There is no comparison. What if I compare it to BTRFS + Snapper? No performance penalty there, plus checksumming.

About 2 years ago I believed the same. Then I used BTRFS as a store for VM images (with periodoc snapshot) and performance went down to really really bad. After I deleted all snapshots performance was good again. There is a big performance penalty in btrfs with more than about 100 snapshots.

Re: Linux NILFS file system: automatic continuous snapshots

#75

Post by conradev »

nix23 wrote: post #18
newcup wrote: post #12

I think NILFS is a hidden gem. I’ve been using it exclusively in my Linux laptops, desktops etc. since ca. 2014. Apart from one kernel regression bug related to NILFS2 it’s worked flawlessly (no data corruption even with the bug just no access to the file system; effectively it forced running older kernel while the bug was fixed). The continuous snapshotting has saved me a couple of times; I’ve just mounted a version…

>I think NILFS is a hidden gem. I’ve been using it exclusively in my Linux laptops, desktops etc. since ca. 2014 Yes it's really sad, there we have a native and stable check-summing fs, and nearly no one knows about it.

BTRFS is also a native copy on write filesystem that verifies a configurable checksum and supports snapshots.

The snapshots are not automatic, but short of that it is pretty feature complete

Re: Linux NILFS file system: automatic continuous snapshots

#76

Post by spookthesunset »

didgetmaster wrote: post #65
spookthesunset wrote: post #60

Earlier quoted context omitted.

It might sound weird but the hard part of what you describe is not the technology but how to design the UX in a way that you aren’t babysitting everything. And doing that is not at all easy. For all anybody knows your cat video is “worth more” to you than your thesis paper. How can you get the system to determine the worth of each file without manually setting an attribute each time you create a file? And if you let…

You could always set the default behavior to be uniform for all files (e.g. protect everything or protect nothing) and just forget about it. But it would be nice to be able to manually set the protection level for specific files that are the exception. If I was copying an important file into an unprotected environment, I could change how it was handled (likewise if I was downloading some huge video I didn't care abou…

I’d think on a directory basis would be the ideal

Re: Linux NILFS file system: automatic continuous snapshots

#78

Post by okasaki »

darau1 wrote: post #77

What's the difference between a snapshot, and a checkpoint?

from TA:

> A checkpoint is a snapshot of your system at a given point in time, but it can be deleted automatically if some disk space must be reclaimed. A checkpoint can be transformed into a snapshot that will never be removed.

Re: Linux NILFS file system: automatic continuous snapshots

#79

Post by nix23 »

conradev wrote: post #75
nix23 wrote: post #18

Earlier quoted context omitted.

>I think NILFS is a hidden gem. I’ve been using it exclusively in my Linux laptops, desktops etc. since ca. 2014 Yes it's really sad, there we have a native and stable check-summing fs, and nearly no one knows about it.

BTRFS is also a native copy on write filesystem that verifies a configurable checksum and supports snapshots. The snapshots are not automatic, but short of that it is pretty feature complete

That's why i specifically wrote -> stable...

Re: Linux NILFS file system: automatic continuous snapshots

#80

Post by regularfry »

jerf wrote: post #25

What happens if you run "dd if=/dev/zero of=/any/file/here", thus simply loading the disk with all the zeros it can handle? Do you lose all your snapshots as they are deleted to make room, or does it keep some space aside for this situation? (Not a "gotcha" question, a legitimate question.)

I know this isn't what you're getting at, but is it smart enough to create a sparse file when you specifically pick zero as your filler byte?
Post reply on HN