Is anybody using bcachefs yet?
A data corruption bug in OpenZFS?
61–70 of 115 posts
Re: A data corruption bug in OpenZFS?
#62Earlier quoted context omitted.
This a feature I was completely unaware of. Why would you choose to use a sparse file instead of multiple files?
Imagine a torrent client (or http client downloading a file in parallel using HTTP range requests). It creates an empty file and then it has downloaded a 1MB of data to write at offset 100GB and wants to write it to disk. It does not want to pay the price of waiting for 100GB of zeroes to be written. The other blocks will all be downloaded and written eventually, all out of order. If the filesystem had an atomic oper…
Fallocate is a much better interface for this than sparse files. The torrent client does not care how the underlying filesystem provides the ability to randomly write a large file. And fallocate is a much clearer signal to the filesystem than a sparse file.
Re: A data corruption bug in OpenZFS?
#63Earlier quoted context omitted.
No, I get it. I'm saying that's a bad design. The data structure for a VM system is a big tree of discontiguous mappings, which matches the API used for accessing it. If you make a random access to memory at an arbitrary spot, you expect to get a VM trap. If you want to map memory, you're expected to know the layout and manage the "holes" yourself (or else to let the OS manage your memory space for you). The data str…
> Making the filesystem, implemented from first principles to handle the second style of interaction, actually be implemented in terms of the first under the hood, is a source of needless complexity and bugs. And it was here, too. Aren’t all modern file systems implemented as a tree of discontinuous regions? That’s the whole reason block allocators exist, why file fragmentation is a thing (and defragmentation process…
That's true of a storage backend, but not the metaphor presented. Again, the analogy would be a heap: heaps are discontiguous internally too, but you don't demand that users of malloc() understand that there can be a hole in the middle of their memory! Again, the bug here was (seems to have been, it's subtle) a glitch in the tracking of holes in files that didn't ever need to have been there in the first place.
Re: A data corruption bug in OpenZFS?
#64Earlier quoted context omitted.
How much is recovery of let's say 500gb a month/1 full restore a year ?
Googling says 2c/GB, cheaper (10x) in bulk.
You might wanna double-check your math. I used the AWS pricing calculator, said I wanted to store 8000GB in Glacier Deep Archive in us-east-2, and wanted to recover it using 16000 API requests (wild guess). That, plus $0.05-$0.09/GB transfer came out to about $960 to recover.
Glacier is always super cheap as long as you don’t need to recover, and then it’s ferocious.
Re: A data corruption bug in OpenZFS?
#65Earlier quoted context omitted.
The number of file descriptors you can have open by a single program is limited and eats up kernel resources.
Even for a torrent client, the number of active file descriptors is a function of the number of peer connections (e.g. a few dozen). It doesn't scale with the size of the output file.
Think databases like RocksDB not torrents.
Re: A data corruption bug in OpenZFS?
#66Periodic reminder to check if your backups are working, and if you can also restore them. It doesn't matter which file system or operating system you use, make sure to backup your stuff. In a way that's immune to ransomware as well, so not just a RAID-1/5/Z or another form of hot/warm storage (RAID is not a backup, it's an uptime/availability mechanism) but cold storage. (I snapshot and tar that snapshot every night,…
[deleted]
Re: A data corruption bug in OpenZFS?
#67Is anybody using bcachefs yet?
I'm keeping an eye on it but it's not there yet e.g. https://github.com/koverstreet/bcachefs/issues/619#issuecomm...
Re: A data corruption bug in OpenZFS?
#68Earlier quoted context omitted.
Imagine a torrent client (or http client downloading a file in parallel using HTTP range requests). It creates an empty file and then it has downloaded a 1MB of data to write at offset 100GB and wants to write it to disk. It does not want to pay the price of waiting for 100GB of zeroes to be written. The other blocks will all be downloaded and written eventually, all out of order. If the filesystem had an atomic oper…
Out of curiosity, do you know off-hand how torrent clients do it on filesystems that don't support sparse files? There must be either a preallocate-the-whole-thing step, or a gather-the-pieces-together-and-write-out-the-large-file step. The latter would seem to briefly double the disk space needed at the end of the download, so I suspect they do the former.
AFAIK, they preallocate; and even on filesystems which support sparse files, most bittorrent clients have an option to always preallocate (to both reserve space and reduce fragmentation).
Re: A data corruption bug in OpenZFS?
#69It's important to note that the recent showstopper bugs have all been in OpenZFS, with the Oracle nee Sun ZFS being unaffected by either.
Re: A data corruption bug in OpenZFS?
#70I wonder if any large storage provider has been affected by this. I know Hetzner Storage Box and rsync.net both use ZFS under the hood.