Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GEFS on OpenBSD: A Early Preview

marc.info

Post reply on HN
Print view View original

51–60 of 78 posts

Re: GEFS on OpenBSD: A Early Preview

#51

Post by rbc »

doublepg23 wrote: post #5

This is great timing considering I'm currently dealing with FFS corruption after my OpenBSD server lost power during a storm.

I've run OpenBSD continuously at home without backup power for something like eight years. They probably loose power about once or twice a quarter. I have yet to detect lost data that was important. Measures are taken. If a storm is coming through, I'll shutdown till it passes.

I've been lucky with release driver support. The little Lenovo ThinkCentre's being used seem to chug along without crashing on driver issues, at least with OpenBSD releases.

The standard OpenBSD partitioning scheme is also being used. Boot time fsck has never failed when the storage was properly attached and in a good state. Backups are performed using pax. So far, so good.

Re: GEFS on OpenBSD: A Early Preview

#54

Post by rzerowan »

limagnolia wrote: post #53

I've always wondered why HAMMER2 from DragonFlyBSD hasn't gotten more attention from other OSes?

I think its the way their kernel architecture has diverged to this point in time.Seeing as it depends on structures/primitives in DRagonfly that dont exist in the other BSDs. Making a port a bit of a challenge.

Re: GEFS on OpenBSD: A Early Preview

#55

Post by scheme271 »

sippingabonedry wrote: post #27
yjftsjthsd-h wrote: post #23

Earlier quoted context omitted.

Who said anything about storage servers? I'm using zfs on laptops and desktops right now because I want data checksums and a filesystem that doesn't have a history of breaking horribly (I dropped btrfs after the second time it hosed my rootfs). Given the license issue with zfs - and in particular, the technical fallout like needing dkms - I'd be very pleased to replace it.

> I dropped btrfs after the second time it hosed my rootfs btrfs fans use the "you're using it wrong" excuse a lot. I recall a failure mode that activated when you fill the FS to 100% and their response was "you should never fill a filesystem to capacity"

That failure mode is still there although it's been mitigated a bit. I hit it a few months ago but was able to recover after a few hours of doing various stuff.

Re: GEFS on OpenBSD: A Early Preview

#56

Post by sippingabonedry »

scheme271 wrote: post #55
sippingabonedry wrote: post #27

Earlier quoted context omitted.

> I dropped btrfs after the second time it hosed my rootfs btrfs fans use the "you're using it wrong" excuse a lot. I recall a failure mode that activated when you fill the FS to 100% and their response was "you should never fill a filesystem to capacity"

That failure mode is still there although it's been mitigated a bit. I hit it a few months ago but was able to recover after a few hours of doing various stuff.

Yeah that's not how filesystems are supposed to work.

You should be able to dd /dev/zero to a file and not worry about the entire filesystem shitting the bed.

Re: GEFS on OpenBSD: A Early Preview

#57

Post by throw0101a »

ThePowerOfFuet wrote: post #40
chasil wrote: post #24

Earlier quoted context omitted.

Both ZFS and modern btrfs support a large set of checksums. Both implement sha256, which does impose a heavy speed penalty. ZFS allows you to adjust the checksum on the fly, using something faster (Fletcher) if desired. In btrfs, a global checksum is set at filesystem creation; xxhash is the best modern option. There is a website: https://xxhash.com Deduplication adds concerns for a strong hash free of collisions.

Fletcher has been the default for quite some time.

Specifically "fletcher4":

* https://openzfs.github.io/openzfs-docs/man/master/7/zfsprops...

* https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Data...

* https://en.wikipedia.org/wiki/Fletcher%27s_checksum

* https://people.freebsd.org/~asomers/fletcher.pdf

* https://www.intel.com/content/www/us/en/developer/articles/t...

Originally documented in the paper "An Arithmetic Checksum for Serial Transmissions" (referenced also in RFC 1146, in the context of TCP).

Re: GEFS on OpenBSD: A Early Preview

#58

Post by oridb »

throw0101a wrote: post #37
yjftsjthsd-h wrote: post #10

From https://orib.dev/gefs.pdf - > While snapshot consistency is useful to keep data consistent, disks often fail over time. In order to detect corruption, block pointers contain a hash of the data that they point at. If corrupted data is returned by the underlying storage medium, this is detected via block hashes. And if a programmer error causes the file system to write garbage to disk, this can often be caught ear…

> It's got CoW, snapshots, and data checksums. Therefore, it's good enough to compete with ZFS while being way smaller and permissively licensed. Does it have a built-in RAID layer? Because if it doesn't, then it can't compete with ZFS in many use cases. For example, what does "data may then be recovered from […] RAID restoration" mean? With ZFS, if you have a (e.g.) mirrored/RAID-1 configuration, and you fetch some…

Author here: correct, it's currently on the user to deal with corrupted data. I'm not opposed to adding a RAID layer, but I also want to explore what adding trim-style feedback to a generic RAID layer may look like, so that different file systems could share the work.

We'll see where things go.

Re: GEFS on OpenBSD: A Early Preview

#59

Post by oridb »

yellowapple wrote: post #39
daneel_w wrote: post #18

Earlier quoted context omitted.

Are you sure it's not just a dodgy SSD that simply failed to persist data when losing power mid-write? I've had my share of sudden power cuts upon OpenBSD during the past 20+ years, and FFS has so far never gone corrupt on me.

I've seen file corruption with FFS on some of my OpenBSD servers, though them being VMs is likely a factor there.

Qcow2 will zero blocks on power loss, at least in some configurations.

Re: GEFS on OpenBSD: A Early Preview

#60

Post by crest »

sippingabonedry wrote: post #27
yjftsjthsd-h wrote: post #23

Earlier quoted context omitted.

Who said anything about storage servers? I'm using zfs on laptops and desktops right now because I want data checksums and a filesystem that doesn't have a history of breaking horribly (I dropped btrfs after the second time it hosed my rootfs). Given the license issue with zfs - and in particular, the technical fallout like needing dkms - I'd be very pleased to replace it.

> I dropped btrfs after the second time it hosed my rootfs btrfs fans use the "you're using it wrong" excuse a lot. I recall a failure mode that activated when you fill the FS to 100% and their response was "you should never fill a filesystem to capacity"

They also fucked up their parity RAID repair so that the next disk failure on a different disk will destroy the data, argued that the pathological B-tree performance reported by a user couldn't be real, because they didn't understood the tree depth would only be logarithmic to the base of the fanout if the entries are the same size. insert shocked pikachu face
Post reply on HN