Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linux NILFS file system: automatic continuous snapshots

dataswamp.org

Post reply on HN
Print view View original

151–158 of 158 posts

Re: Linux NILFS file system: automatic continuous snapshots

#151

Post by ComputerGuru »

heavyset_go wrote: post #107
ComputerGuru wrote: post #67

Earlier quoted context omitted.

> Apart from one kernel regression bug related to NILFS2 it’s worked flawlessly Maybe on x86? I’ve tried repeatedly to use it on ARM for RaspberryPi where it would have been perfect, but always ran into various kernel panics as soon as the file system is mounted or accessed.

I've used NILFS2 on flash storage on some old non-RPi ARMv7 hardware for a while without a problem. Switched to F2FS for performance reasons, though.

This was on the root partition, which is subjected to a lot more concurrency issues than an sd card normally would fwiw.

Re: Linux NILFS file system: automatic continuous snapshots

#152

Post by compsciphd »

ephbit wrote: post #140
compsciphd wrote: post #136

Earlier quoted context omitted.

I'm not anywhere near a user of btrfs send/recieve api, but I was under the impression that it essentially streams commands and hence could conceptually also be used to delete snapshots on the remote side?

I don't know about the API but as far as I understand the man page, there doesn't appear to be a way to delete a subvolume on the remote side by doing btrfs receive. That btrfs receive may well be exploitable (because of some bug) if you feed it with malicious data is IMO a different topic.

so I'm looking at Oracle's docs

https://docs.oracle.com/cd/E37670_01/E37355/html/ol_srbackup...

while it doesn't allow you to delete a subvolume (i.e. what a snapshot is built off of), it does seem like one could conceptually just overwrite it via send/recieve? perhaps its a bit smarter and doesn't let snapshots be modified, but I'm now a bit unconvinced that it can be viewed as simply an "append only" interface.

Re: Linux NILFS file system: automatic continuous snapshots

#153

Post by ephbit »

compsciphd wrote: post #152
ephbit wrote: post #140

Earlier quoted context omitted.

I don't know about the API but as far as I understand the man page, there doesn't appear to be a way to delete a subvolume on the remote side by doing btrfs receive. That btrfs receive may well be exploitable (because of some bug) if you feed it with malicious data is IMO a different topic.

so I'm looking at Oracle's docs https://docs.oracle.com/cd/E37670_01/E37355/html/ol_srbackup... while it doesn't allow you to delete a subvolume (i.e. what a snapshot is built off of), it does seem like one could conceptually just overwrite it via send/recieve? perhaps its a bit smarter and doesn't let snapshots be modified, but I'm now a bit unconvinced that it can be viewed as simply an "append only" interface.

> ... it does seem like one could conceptually just overwrite it via send/recieve?

Don't think so.

The man page for btrfs receive states:

> btrfs receive will fail in the following cases:

> 1. receiving subvolume already exists

I also just tried it out. I created some subvol1-ro in a source filesystem, then sent it to the destination filesystem. Then deleted the source subvol1-ro. Then created a new subvol1-ro containing a different file and tried to send it to the same destination.

Result:

> ERROR: creating subvolume subvol1-ro failed: File exists

Re: Linux NILFS file system: automatic continuous snapshots

#154

Post by compsciphd »

ephbit wrote: post #153
compsciphd wrote: post #152

Earlier quoted context omitted.

so I'm looking at Oracle's docs https://docs.oracle.com/cd/E37670_01/E37355/html/ol_srbackup... while it doesn't allow you to delete a subvolume (i.e. what a snapshot is built off of), it does seem like one could conceptually just overwrite it via send/recieve? perhaps its a bit smarter and doesn't let snapshots be modified, but I'm now a bit unconvinced that it can be viewed as simply an "append only" interface.

> ... it does seem like one could conceptually just overwrite it via send/recieve? Don't think so. The man page for btrfs receive states: > btrfs receive will fail in the following cases: > 1. receiving subvolume already exists I also just tried it out. I created some subvol1-ro in a source filesystem, then sent it to the destination filesystem. Then deleted the source subvol1-ro. Then created a new subvol1-ro contai…

so, if I use a subvolume (not as a snapshot, but as something I can modify) I can't use send/receive on it? it seems a little weird. I'd assume I should be able to send modifications to the remote end (and my argument is, if one can send modifications, one should be able to send modifications that effectively delete all content). I temper this argument that even if true, perhaps subvolumes marked as snapshots one cannot do this to.

Re: Linux NILFS file system: automatic continuous snapshots

#155

Post by harvie »

yjftsjthsd-h wrote: post #29
harvie wrote: post #14

I had issues with file locking when running some legacy database software on NILFS2. Probably caused data corruption in that database (not the FS itself). SF website of NILFS2 suggests that there are some unimplemented features, one of them being synchronous IO, which might have caused that issue? https://nilfs.sourceforge.io/en/current_status.html In some cases, the NILFS2 is safer storage for your data than ZFS. So…

> In some cases, the NILFS2 is safer storage for your data than ZFS. What cases? Do you just mean due to continuous snapshots protecting against accidental deletes or such, or are there more "under the covers" things it fixes?

Yeah. i mean just due to continuous snapshots. Otherwise i don't really trust it that much. It's like this outsider kid which is kinda cool. But still needs to grow up and sell it :-D

Re: Linux NILFS file system: automatic continuous snapshots

#156

Post by harvie »

mustache_kimono wrote: post #41
harvie wrote: post #23

Earlier quoted context omitted.

Perhaps we can leverage "inotify" API to make ZFS snapshot everytime some file had been changed... But i think ZFS is not really good at handling huge amounts of snapshots. The NILFS2 snapshots are probably more lightweight when compared to ZFS ones.

> Perhaps we can leverage "inotify" API to make ZFS snapshot everytime some file had been changed... ZFS and btrfs users are already living in the future: inotifywait -r -m --format %w%f -e close_write "/srv/downloads/" | while read -r line; do # command below will snapshot the dataset # upon which the closed file is located sudo httm --snap "$line" done See: https://kimono-koans.github.io/inotifywait/

What is httm? I like this script as a proof of concept.

But i still can imagine failure modes, eg. inotify might start acting weird when ZFS remounts the watched directory, OOM killer terminates it without anyone noticing, bash loop go haywire when package manager updates that script (bash is running directly from the file and when it changes during execution, it might just continue running from the same byt offset in completely different script).

All these things actualy happened to me in the past. Not to say that if you have multiple datasets in ZFS you cannot inotify wait on all of them at once, so you will have to manage one bash process per dataset. And performance of bash and sudo might not be that awesome.

So for real reliability you would probably want this to actualy run in ZFS/kernel context...

Re: Linux NILFS file system: automatic continuous snapshots

#157

Post by ephbit »

compsciphd wrote: post #154
ephbit wrote: post #153

Earlier quoted context omitted.

> ... it does seem like one could conceptually just overwrite it via send/recieve? Don't think so. The man page for btrfs receive states: > btrfs receive will fail in the following cases: > 1. receiving subvolume already exists I also just tried it out. I created some subvol1-ro in a source filesystem, then sent it to the destination filesystem. Then deleted the source subvol1-ro. Then created a new subvol1-ro contai…

so, if I use a subvolume (not as a snapshot, but as something I can modify) I can't use send/receive on it? it seems a little weird. I'd assume I should be able to send modifications to the remote end (and my argument is, if one can send modifications, one should be able to send modifications that effectively delete all content). I temper this argument that even if true, perhaps subvolumes marked as snapshots one can…

AFAIK only read only snapshots can be sent and received. It kind of makes sense, since a writable snapshot could be written to during the send/receive so that the remote result would not match any clearly defined local state.

A received ro snapshot can be used as a base for a new writable snapshot.

Re: Linux NILFS file system: automatic continuous snapshots

#158

Post by mustache_kimono »

harvie wrote: post #156
mustache_kimono wrote: post #41

Earlier quoted context omitted.

> Perhaps we can leverage "inotify" API to make ZFS snapshot everytime some file had been changed... ZFS and btrfs users are already living in the future: inotifywait -r -m --format %w%f -e close_write "/srv/downloads/" | while read -r line; do # command below will snapshot the dataset # upon which the closed file is located sudo httm --snap "$line" done See: https://kimono-koans.github.io/inotifywait/

What is httm? I like this script as a proof of concept. But i still can imagine failure modes, eg. inotify might start acting weird when ZFS remounts the watched directory, OOM killer terminates it without anyone noticing, bash loop go haywire when package manager updates that script (bash is running directly from the file and when it changes during execution, it might just continue running from the same byt offset i…

> What is httm? I like this script as a proof of concept.

See: https://github.com/kimono-koans/httm

> But i still can imagine failure modes, eg. inotify might start acting weird when ZFS remounts the watched directory, OOM killer terminates it without anyone noticing, bash loop go haywire when package manager updates that script (bash is running directly from the file and when it changes during execution, it might just continue running from the same byt offset in completely different script).

I mean, sure, scripts gonna script. You're gonna have to make the POC work for you. But, for instance, I'm not sure half of your issues are problems with a systemd service. I'm not sure one is a problem with a well designed script, which accounts for your particular issues, and a systemd service.

> All these things actualy happened to me in the past. Not to say that if you have multiple datasets in ZFS you cannot inotify wait on all of them at once, so you will have to manage one bash process per dataset. And performance of bash and sudo might not be that awesome.

Yes, you can?

Just re this POC, you can inotifywait a single directory, which contains multiple datasets, and httm will correctly determine and snapshot the correct one upon command. Your real bottleneck here is not sudo or bash. It's the zfs command waiting for a transaction group sync, or arranging for the trans group (or even something else, but its definitely zfs?), to snap.

You can also use `httm -m` to simply identify the dataset and use a channel program and/or a separate script to sync. sudo and bash may not have the performance for your use case, hell, they are composable with everything else?

> So for real reliability you would probably want this to actualy run in ZFS/kernel context...

Yeesh, I'm not sure? Maybe for your/a few specific use cases? Note, inotify (a kernel facility) is your other bottleneck. You're never going to want to watch more than a few/10s of thousand files. The overhead is just going to be too great.

But for most use cases (your documents folder)? Give httm and inotifywait a shot.

Post reply on HN