> The git repo is hidden on shithub Heh :)
Is this classic 9front tomfoolery?
GEFS on OpenBSD: A Early Preview
31–40 of 78 posts
Re: GEFS on OpenBSD: A Early Preview
#32What a wonderful surprise! The nearest thing seem to be modern (v5) XFS + dm-integrity, I'll have to see a comparison once it's stable enough. A thing ZFS suffers from is fragmentation (no way to defragment in-place nor preallocate so stuff like bittorrent doesn't play well with it), which it justifies with its CoW design, wonder if/how it mitigates the problem.
If we're looking at 'future' filesystems, is fragmentation really an issue in an SSD world? Not that there isn't a lot of spinning rust (and will be for quite a while), I don't think it's unreasonable to assume "most block storage is going to be SSD in the future" when allocating resources to priorities.
The day I can furnish my NAS in SSDs for roughly the same price as HDDs probably won't come before any current filesystem is obsoleted for some reason or another, methinks.
Re: GEFS on OpenBSD: A Early Preview
#33Earlier quoted context omitted.
Who said anything about storage servers? I'm using zfs on laptops and desktops right now because I want data checksums and a filesystem that doesn't have a history of breaking horribly (I dropped btrfs after the second time it hosed my rootfs). Given the license issue with zfs - and in particular, the technical fallout like needing dkms - I'd be very pleased to replace it.
I have been hearing noise recently that btrfs is risky and unstable but (knocks on wood) i've been running it for years now with zero issues. What am I missing?
Failures in storage software are very rare. Which makes it very hard to test... (you need to run it a lot, for a very long time if you hope to find errors by chance).
Also, some failure modes are worse than others. The failures known as DI (data integrity) are the worst. Even though they aren't expected to happen to everyone at a certain frequency (because, again, mature storage software is comparatively very reliable), even a single DI error that happened to any user sets up a major alarm.
In the storage industry, the running joke is that after first DI in your product you lose funding, after the second DI you loose the product.
And it did happen to Btrfs quite a bit... I've seen it with my own eyes when a system didn't come back after power failure. (But I'm in the business of testing software storage products, so, it's less surprising that it happened to me).
So... it's perfectly plausible that you have never seen Btrfs fail, and it's been more error prone than eg. EXT4. The error rate is low enough so that if you don't actively try to cause the error you will never experience one. But, over a large group of diverse use patterns, the rate is still worse than expected.
Re: GEFS on OpenBSD: A Early Preview
#34Earlier quoted context omitted.
> I dropped btrfs after the second time it hosed my rootfs btrfs fans use the "you're using it wrong" excuse a lot. I recall a failure mode that activated when you fill the FS to 100% and their response was "you should never fill a filesystem to capacity"
Otoh, good luck bringing a CoW filesystem back from 100%. Delete a file? Sure, let me just make a copy of all the metadata that was pointing at it using... the zero blocks I have left. Tradeoffs are a bitch, bitch.
Which is why ZFS reserves "slop space" to make sure that doesn't happen, instead of defaulting to making it easy for users to corner themselves like that.
Re: GEFS on OpenBSD: A Early Preview
#35From https://orib.dev/gefs.pdf - > While snapshot consistency is useful to keep data consistent, disks often fail over time. In order to detect corruption, block pointers contain a hash of the data that they point at. If corrupted data is returned by the underlying storage medium, this is detected via block hashes. And if a programmer error causes the file system to write garbage to disk, this can often be caught ear…
I've always wondered about similar designs: Doesn't calculating a hash of every block, on every read and every write, create lots of overhead? Why isn't that a problem? Some systems have dedicated crypto co-processors for confidentiality (encryption) - e.g., I think drives with FDE, and I think Apple Silicon SoCs might have them. Can those be repurposed for hash calculation? What about systems that lack them?
Programs like filesystems typically have their own schedulers that aggregate writes, it would be an extreme performance hit if they wrote each checksum individually, as a separate I/O operation. They are certainly bundled with some other data that needs to be written.
And if you are concerned about the compute rather than storage, then writing to a block device is still slow enough so that computing a checksum isn't important performance-wise.
Re: GEFS on OpenBSD: A Early Preview
#36Earlier quoted context omitted.
Who said anything about storage servers? I'm using zfs on laptops and desktops right now because I want data checksums and a filesystem that doesn't have a history of breaking horribly (I dropped btrfs after the second time it hosed my rootfs). Given the license issue with zfs - and in particular, the technical fallout like needing dkms - I'd be very pleased to replace it.
I have been hearing noise recently that btrfs is risky and unstable but (knocks on wood) i've been running it for years now with zero issues. What am I missing?
btrfs is almost 20 years old now, so lots of people only used it back when it was newer and far buggier.
In my experience, btrfs is actually more reliable than other filesystems due to its checksumming abilities, but when it does fail, it's much harder to fix than with other filesystems (which will often try to continue on even when stuff is broken).
Re: GEFS on OpenBSD: A Early Preview
#37From https://orib.dev/gefs.pdf - > While snapshot consistency is useful to keep data consistent, disks often fail over time. In order to detect corruption, block pointers contain a hash of the data that they point at. If corrupted data is returned by the underlying storage medium, this is detected via block hashes. And if a programmer error causes the file system to write garbage to disk, this can often be caught ear…
> It's got CoW, snapshots, and data checksums. Therefore, it's good enough to compete with ZFS while being way smaller and permissively licensed.
Does it have a built-in RAID layer? Because if it doesn't, then it can't compete with ZFS in many use cases. For example, what does "data may then be recovered from […] RAID restoration" mean?
With ZFS, if you have a (e.g.) mirrored/RAID-1 configuration, and you fetch some data from one drive and the checksum is wrong, ZFS can check the other drive, and if that checksum is good it can (a) pass the good data up, and (b) use the good data to fix the bad data. Most mirroring systems can't do that both-drives checking: ZFS is self-healing.
(This isn't to say that GEFS won't be useful in many other situations.)
Re: GEFS on OpenBSD: A Early Preview
#38>Error handling is largely commented out.
First we do the first 90%, and then we do the last 90%.
Re: GEFS on OpenBSD: A Early Preview
#39This is great timing considering I'm currently dealing with FFS corruption after my OpenBSD server lost power during a storm.
Are you sure it's not just a dodgy SSD that simply failed to persist data when losing power mid-write? I've had my share of sudden power cuts upon OpenBSD during the past 20+ years, and FFS has so far never gone corrupt on me.
I've seen file corruption with FFS on some of my OpenBSD servers, though them being VMs is likely a factor there.
Re: GEFS on OpenBSD: A Early Preview
#40Earlier quoted context omitted.
I've always wondered about similar designs: Doesn't calculating a hash of every block, on every read and every write, create lots of overhead? Why isn't that a problem? Some systems have dedicated crypto co-processors for confidentiality (encryption) - e.g., I think drives with FDE, and I think Apple Silicon SoCs might have them. Can those be repurposed for hash calculation? What about systems that lack them?
Both ZFS and modern btrfs support a large set of checksums. Both implement sha256, which does impose a heavy speed penalty. ZFS allows you to adjust the checksum on the fly, using something faster (Fletcher) if desired. In btrfs, a global checksum is set at filesystem creation; xxhash is the best modern option. There is a website: https://xxhash.com Deduplication adds concerns for a strong hash free of collisions.
Fletcher has been the default for quite some time.