Live data from Hacker News

Linux NILFS file system: automatic continuous snapshots

dataswamp.org

141–150 of 158 posts

Re: Linux NILFS file system: automatic continuous snapshots

#141
post #140

Earlier quoted context omitted.

I'm not anywhere near a user of btrfs send/recieve api, but I was under the impression that it essentially streams commands and hence could conceptually also be used to delete snapshots on the remote side?

I don't know about the API but as far as I understand the man page, there doesn't appear to be a way to delete a subvolume on the remote side by doing btrfs receive. That btrfs receive may well be exploitable (because of some bug) if you feed it with malicious data is IMO a different topic.

ok, you are probably right, glanced at send.h and it seems it only allows a limited set of "btrfs" fs operations.

that would fit with my description (at least in a sense) it enables an "append" only mode to a remote side (where "append" means cant mess with previous snapshots remotely).

Re: Linux NILFS file system: automatic continuous snapshots

#142

Earlier quoted context omitted.

sadly (as with much work form phd students like I was), the closest one could get to it today is trying to duplicate it. i.e. combining criu with nilfs (but a lot of the work that we did to get process downtime to minimal numbers requires being in kernel, as described in paper) and unsure criu can do it. In addition our screenrecording mechanism was our own "proprietary" (not really proprietary as fully described in…

Very cool. I could see a system like this: 1. being the layer upon which AI-assisted computing is trained and built 2. being used for video game QA. For certain heisenbugs it'd be invaluable to be able to restore point-in-time repro to resume just before it happened

the lab I was in did explore it (not work I was directly involved in, so don't have much direct insight into it, besides knowing that we were exploring how to use it for debugging).

https://www.cs.columbia.edu/~nieh/pubs/sigmetrics2010_scribe... https://www.cs.columbia.edu/~nieh/pubs/sigmetrics2011_transp...

Re: Linux NILFS file system: automatic continuous snapshots

#143

Earlier quoted context omitted.

NILFS is absolutely wonderful; it was very unfortunate that Linus chose to dub btrfs as the ext4 successor all those years ago, because it cut off a lot of interest in the plethora of interesting work that was going on at the time. A decade later and btrfs is still riddled with problems and incomplete, people are still using xfs and ext4 for lack of trust, one kernel dev has a side hobby trying to block openzfs, and…

> one kernel dev has a side hobby trying to block openzfs Can you elaborate?

I don't know if everything has been collected in an easy-to-digest form, but GKH has gone out of his way a few times to shut out OpenZFS.

Re: Linux NILFS file system: automatic continuous snapshots

#144
post #132
post #128

Earlier quoted context omitted.

I think you don't understand COW or non-COW filesystems? They don't work the way you are asserting.

I could say the same. Be more specific please. If everything I've said is wrong it should be easy to point out something demonstrably false.

For one, you get no performance benefit over non-cow unless you update in place. It’s what every ‘fast and easy’ filesystem has to do - fat (including exfat), ext3, ext4, etc.

The failure modes are well documented - and got worse in many cases trying to work around performance issues due to journaling, but the journaling doesn’t resolve the issue fully because they can’t store all the data they need without making the performance issues worse. See https://en.m.wikipedia.org/wiki/Ext4 and ‘Delayed allocation and data loss’ for one example.

This isn’t a solved (or likely solvable in a reasonable way) problem with non-COW filesystems, which is one of the reasons why all newer filesystems are COW. The other being latency hits from tracking down COW delta blocks aren’t a big issue now due to SSDs and having enough RAM to have decent caches and pre-allocation buffers.

Also, COW doesn’t need to allocate (or re-read/re-checksum) the entire prior block when someone changes something, unlike modify in place. Due to alignment issues, doing SOME usually makes sense, but it’s highly configurable.

It only needs to add new metadata with updated mapping information for the updated range in the file, and then checksum/write out the newly updated data (only), plus or minus alignment issues or whatever. It acts like a patch. That’s literally the whole point of COW filesystems.

Update in place has an already allocated block it has to deal with in real time, either now consuming less space in it’s already allocated area (leaving tiny fragmentation) or by having to allocate a new block and toss the old one, which will have worse real time performance than a COW system, as it’s doing the new block allocation (which is more space than a COW write, unless the COW write is for the entire blocks contents!), plus going back and removing the old block.

ZFS record size for instance is just the maximum size of one of the patch ‘blocks’. The actual records are only the size of the actual write data + Filesystem overhead.

ZFS only then goes back and removes old records when they aren’t referenced by anyone, which is typically async/background, and doesn’t need to happen as part of the write itself.

This allows freeing up entire regions of pool space easier, and fragmentation becomes much less of an issue.

Re: Linux NILFS file system: automatic continuous snapshots

#145
post #56

Earlier quoted context omitted.

Why is it ‘wasted’? Those things are mostly free on modern hardware. The challenge with your thesis here is that the only one who can know what is ‘that important’ is YOU , and your decision making and communication bandwidth is already the limiting factor. For many users, that cat video would be heartbreaking to lose, and they don’t have term papers to worry about. So having to decide or think what is or is not ‘imp…

Nothing is free or even 'mostly free' when managing data. Data security (encryption), redundancy (backups), and integrity (checksums, etc.) all impose a cost on the system. Getting each piece of data properly classified will always be a challenge (AI or other tools may help with that), but it would still be nice to be able to do it. If I have a 50GB video file that I could easily re-download off the Internet, it woul…

Worried if it gets lost or mangled? Not necessarily. Worried if it’s happening and I have no idea, and it’s spreading - due to hardware or software problems? And I’ll only discover it when something I do actually really care about and can’t easily replace? Absolutely!

The big issue regarding duplication is really more a identification/‘supply chain’ issue. The last thing I want to be doing is trying to figure out how to get that other file from somewhere (that works), from whoever ‘has another copy’, when it gets mangled and I need a replacement ASAP.

If you’re thinking of our local filesystems as potentially just a cache, we have no easy or secure way right now to fingerprint or recreate the other entries in the cache from sources (minus browser caches or the like, but even then, re-retrieving it may return different or not content).

So backups are really keeping the computing equivalent of local ‘ability to manufacture’ handy as a mitigation against risk. It’s not waste, anymore than keeping the ability to manufacture it’s own tanks and weapons in-country is a waste for a nation. It’s an insurance cost against real world problems, and causes a moral hazard with other actors if not done.

And CRC32/CRC32C (even in Java!) is > 16GB/s per core in modern processors, and more than adequate for block sized (typ. 2GB/s per core, and is more than adequate for… well every use case we’re currently aware of.

Modern OS’s really don’t have any excuses for not doing it.

Re: Linux NILFS file system: automatic continuous snapshots

#146
post #144
post #132

Earlier quoted context omitted.

I could say the same. Be more specific please. If everything I've said is wrong it should be easy to point out something demonstrably false.

For one, you get no performance benefit over non-cow unless you update in place. It’s what every ‘fast and easy’ filesystem has to do - fat (including exfat), ext3, ext4, etc. The failure modes are well documented - and got worse in many cases trying to work around performance issues due to journaling, but the journaling doesn’t resolve the issue fully because they can’t store all the data they need without making th…

>For one, you get no performance benefit over non-cow unless you update in place. It’s what every ‘fast and easy’ filesystem has to do - fat (including exfat), ext3, ext4, etc.

That is just a matter of priorities then. And just because you might opt to not update in place in some situations doesn't mean that you can never do it.

I'm not sure what you mean by "Delayed allocation and data loss", I don't find it relevant to this discussion at all since that isn't about filesystem-corruption but application data corruption. And COW also suffers from this - unless you have NILFS/automatic continuous snapshots. Now with COW you probably have a much greater chance of recovering the data with forensic tools (also discussed in this thread regarding ZFS) but with huge downsides and hardly an relevant argument for COW in the vast majority of usecases anyway.

ZFS minimum block size corresponds to disk sector size so for most practical purposes it is the same as your typical non-COW filesystem there. Writing 1 byte requires you to read 4 kb, update it in memory, recalculate checksum, and then writing it down again.

How you remove old records shouldn't depend on COW should it?

My only statement was that checksums isn't in any way dependent on COW.

The discussion about compression is invalid as it is a common feature of non-COW filesystems anyway.

Haven't seen a proper argument for the corruption claims. And that you get corrupted data if you interrupt a write is not a huge deal. Mind you corrupted write. Not corrupted filesystem. The data was toast anyway. A typical COW would at best save you one "block" of data which is hardly worth celebrating anyway. Your application will not care if you wrote 557 out of 1000 blocks or 556 out of 1000 blocks your document is trashed anyway. You need to restore from backup (or from a previous snapshot, which of course is typical killer feature of COW)).

There are also several ways to solve the corruption issue. ReFS for instance has data checksums and metadata checksums but only do copy-on-write for the metadata. (edit: was wrong about this, it uses COW for data too if data checksumming is enabled)

dm-integrity can be used at a layer below the filesystem and solves it with the journal https://www.kernel.org/doc/html/latest/admin-guide/device-ma...

Yes, COW is popular and for good reasons. As is checksumming. It isn't surprising that modern filesystems employ both. Especially since the costs of both have been becoming less and less relevant at the same time.

Re: Linux NILFS file system: automatic continuous snapshots

#147
post #131

I remember DEC/HP releasing the source to the digital unix AdvFS filesystem on sourceforge with the intent of porting it over to linux, but it never materialized. AdvFS had many advanced features. The source is still available and within it are some PDF slides that explain a lot of it's features.

https://en.wikipedia.org/wiki/AdvFS

I noticed 'log based recovery'. Basically ReiserFS.

Re: Linux NILFS file system: automatic continuous snapshots

#148

Earlier quoted context omitted.

> one kernel dev has a side hobby trying to block openzfs Can you elaborate?

I don't know if everything has been collected in an easy-to-digest form, but GKH has gone out of his way a few times to shut out OpenZFS.

Exactly. Removing or changing the licensing on APIs in a way carefully targeted to break ZFS, but not affect actual proprietary drivers such as the NVidia graphics drivers.

Re: Linux NILFS file system: automatic continuous snapshots

#149

Earlier quoted context omitted.

Super cool work. Are any tools like this available today? I know some VM tools have snapshotting but full history and high speed scrubbing sounds awesome.

sadly (as with much work form phd students like I was), the closest one could get to it today is trying to duplicate it. i.e. combining criu with nilfs (but a lot of the work that we did to get process downtime to minimal numbers requires being in kernel, as described in paper) and unsure criu can do it. In addition our screenrecording mechanism was our own "proprietary" (not really proprietary as fully described in…

You might be able to do the screen recording today using Wayland portals, or nested display servers a la Xpra. That could make per-app recording feasible and relatively transparent.

https://xpra.org

Re: Linux NILFS file system: automatic continuous snapshots

#150
post #12

I think NILFS is a hidden gem. I’ve been using it exclusively in my Linux laptops, desktops etc. since ca. 2014. Apart from one kernel regression bug related to NILFS2 it’s worked flawlessly (no data corruption even with the bug just no access to the file system; effectively it forced running older kernel while the bug was fixed). The continuous snapshotting has saved me a couple of times; I’ve just mounted a version…

NILFS is absolutely wonderful; it was very unfortunate that Linus chose to dub btrfs as the ext4 successor all those years ago, because it cut off a lot of interest in the plethora of interesting work that was going on at the time. A decade later and btrfs is still riddled with problems and incomplete, people are still using xfs and ext4 for lack of trust, one kernel dev has a side hobby trying to block openzfs, and…

> it was very unfortunate that Linus chose to dub btrfs as the ext4 successor

That quote is from Ted T'so (https://en.wikipedia.org/wiki/Btrfs#History) or do you have a link to Linus' quote?

Post reply on HN