What happens when you store a virtual machine hard disk image on this? When you boot the VM, is the entire image duplicated via CoW every time something changes in the VM?
Linux NILFS file system: automatic continuous snapshots
121–130 of 158 posts
Re: Linux NILFS file system: automatic continuous snapshots
#122Re: Linux NILFS file system: automatic continuous snapshots
#123Earlier quoted context omitted.
Yup. It’s a pretty fundamental thing. COW and data checksums (and usually automatic/inline compression) co-exist that way because it’s otherwise too expensive performance wise, and potentially dangerous corruption wise. For instance, if you modify a single byte in a large file, you need to update the data on disk as well as the checksum in the block header, and other related data. Chances are, these are in different…
> Chances are, these are in different sectors, and also require re-reading in all the other data in the block to compute the checksum. Anywhere in that process is a chance for corruption of the original data and the update. Not much different than any interrupted write though. And a COW needs to reread just as much. > If the byte changes the final compressed size, it may not fit in the current block at all, causing a…
Literally none of what you are saying is true.
A COW write only needs to checksum the newly written bytes. A non-cow filesystem needs to checksum all data contained in the block (unchanged prior with now new values).
Additionally, a non-COW filesystem needs to update all metadata checksums/values for existing blocks. It's a much more pathological case (interrupted write wise) than a COW filesystem, because if it writes data, but hasn't written the checksum yet - the block is now corrupt. If it writes the checksum, but not the data, the block is now corrupt. And there is no way to know which one is correct post-facto, without storing the old data and the new data somewhere. Which has the exact same overhead or worse as COW. And the data in a FS block is usually many large multiples of the sector size, which makes writes pretty hard to do in any sort of atomic way. Journaling helps, but not with performance here! Since you'd need to store the prior values + the new values, or you're still guaranteed to lose data.
Compression wise, this isn't (as much) of an issue for COW filesystems, because it only needs to compress the newly written data, which can be allocated without concern to the previous allocation size, which is still there, allocated. It can mean less efficient compression if these are small, fragmented writes of course, which is why most of them have some sort of batching mechanism in place. Alternatively, it can copy a chunk of the block, though that can cause write amplification, and is usually minimized.
But you don't run across potential pathological fragmentation issues, like where you compress prior blocks which now take significantly less space, or new blocks take more space and require reshuffling everywhere.
Re: Linux NILFS file system: automatic continuous snapshots
#124Earlier quoted context omitted.
If they are corrupted on disk the behavior is not so deterministic as a ‘broken image’ and a reload. Corrupted on disk content causes software crashes, hangs, and other broken behavior users definitely don’t like. Especially when it’s the filesystem metadata which gets corrupted. Because merely trying to read it can cause severe issues at the filesystem level. I take it you haven’t dealt with failing storage much bef…
I maintain database and object-storage clusters for a living. Dealing with failing storage is half my job. > Especially when it’s the filesystem metadata which gets corrupted. We're not talking about filesystem metadata, though. Filesystem metadata is all "of a piece" — if you have a checksumming filesystem, then you can't not checksum some of the filesystem metadata, because all the metadata lives in (the moral equi…
And yet, I've had several block level bit flips on SSDs and NVMe over the years in high heat/high load environments that weren't that.
And one caused a kernel panic because the fs metadata now pointed towards impossible values.
So tell me again how none of this matters?
Re: Linux NILFS file system: automatic continuous snapshots
#125I've always wondered why NILFS (or similar) isn't used for cases where ransomware is a risk. I'm honestly surprised that it's not mandated to use a append-only / log-structured filesystem for some critical systems (think patient records), where the cost of losing data is so high, rarely mutated, and trading it off for wasting storage isn't that bad (after all, HDD storage is incredibly cheap, and nobody said you had…
you don't need a log structured fs to do this, you could just have regular zfs/btrfs snapshots too. BUT if an attack has the ability to delete an entire file system / encrypt it, they really have the ability to delete the snapshots as well, the only reason they might not is due to "security through obscurity". now, what I have argued is that an append only file system which works in a SAN like environment (i.e. you h…
[deleted]
Re: Linux NILFS file system: automatic continuous snapshots
#126Earlier quoted context omitted.
> Chances are, these are in different sectors, and also require re-reading in all the other data in the block to compute the checksum. Anywhere in that process is a chance for corruption of the original data and the update. Not much different than any interrupted write though. And a COW needs to reread just as much. > If the byte changes the final compressed size, it may not fit in the current block at all, causing a…
Literally none of what you are saying is true. A COW write only needs to checksum the newly written bytes. A non-cow filesystem needs to checksum all data contained in the block (unchanged prior with now new values). Additionally, a non-COW filesystem needs to update all metadata checksums/values for existing blocks. It's a much more pathological case (interrupted write wise) than a COW filesystem, because if it writ…
>A COW write only needs to checksum the newly written bytes. A non-cow filesystem needs to checksum all data contained in the block (unchanged prior with now new values).
No, a COW will re-read the entire block. Make the change and update the checksum and then write it back (to a new location, obviously). Way more than the newly written bytes - but way less than the entire file of course. Just as a non-COW fs will.
>If it writes the checksum, but not the data, the block is now corrupt. And there is no way to know which one is correct post-facto, without storing the old data and the new data somewhere.
Which is exactly what the journal is for - and you already have a journal. Or you don't update it in-place. Just because it isn't COW doesn't mean you always have to update in place.
>Compression wise, this isn't (as much) of an issue for COW filesystems, because it only needs to compress the newly written data, which can be allocated without concern to the previous allocation size, which is still there, allocated. It can mean less efficient compression if these are small, fragmented writes of course, which is why most of them have some sort of batching mechanism in place. Alternatively, it can copy a chunk of the block, though that can cause write amplification, and is usually minimized.
You can do exactly the same for a non-COW filesystem.
>But you don't run across potential pathological fragmentation issues, like where you compress prior blocks which now take significantly less space, or new blocks take more space and require reshuffling everywhere.
COW has the same issue. COW always need to "reshuffle"(?) data somewhere.
Re: Linux NILFS file system: automatic continuous snapshots
#127Re: Linux NILFS file system: automatic continuous snapshots
#128Earlier quoted context omitted.
Literally none of what you are saying is true. A COW write only needs to checksum the newly written bytes. A non-cow filesystem needs to checksum all data contained in the block (unchanged prior with now new values). Additionally, a non-COW filesystem needs to update all metadata checksums/values for existing blocks. It's a much more pathological case (interrupted write wise) than a COW filesystem, because if it writ…
> A COW write only needs to checksum the newly written bytes. A non-cow filesystem needs to checksum all data contained in the block (unchanged prior with now new values). No, a COW will re-read the entire block. Make the change and update the checksum and then write it back (to a new location, obviously). Way more than the newly written bytes - but way less than the entire file of course. Just as a non-COW fs will.…
I think you don't understand COW or non-COW filesystems?
They don't work the way you are asserting.
Re: Linux NILFS file system: automatic continuous snapshots
#129No post body was provided.
Re: Linux NILFS file system: automatic continuous snapshots
#130Earlier quoted context omitted.
Ahh, so you intend, "If you can't use ZFS/btrfs, use dm-integrity"?
No. I don't use ZFS since it's not licensed correctly, so I have no opinion on it. And BTRFS raid is not safe enough for use. So I'm saying "Use filesystem on mdadm on dm-integrity".
>I don't use ZFS since it's not licensed correctly,
Oh look a hobby-lawyer!! Please Linus, license your code "correctly" it's called ISC not GPL.