Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZeroFS vs. Amazon S3 Files

zerofs.net

Post reply on HN
Print view View original

31–34 of 34 posts

Re: ZeroFS vs. Amazon S3 Files

#31

Post by the_duke »

ahofmann wrote: post #11
pierrebarre wrote: post #6

Earlier quoted context omitted.

That’s a fair concern. The closest thing right now is a deterministic simulation suite that injects storage faults and crashes at arbitrary points, then checks the recovered data against reference models. It runs hourly with fresh seeds. CI also runs pjdfstest, xfstests, stress-ng, ZFS scrubs, and Jepsen crash/failover tests: https://github.com/Barre/ZeroFS#testing ZeroFS is still pretty young as storage projects go,…

Why the hell was this answer dead? I vouched for it, because I don't see what might be wrong with it.

No clue, I also vouched.

Re: ZeroFS vs. Amazon S3 Files

#33

Post by pierrebarre »

snapplebobapple wrote: post #26
selfhoster1312 wrote: post #24

Earlier quoted context omitted.

I've been curious about ZeroFS. My usecase is running a real POSIX filesystem on top of garage for integration with non-S3 services. I've had very bad (short) experiment with JuiceFS (1). Is it worth benchmarking `zerofs mount` with garage? My usecase is NAS storage of many small files + some big files (think big shared SMB for a non-profit). I need: - fsync durability (as promised on zerofs homepage), including sqli…

Did you get garagefs to work well? I ended up moving to rustfs for my plakar and restic backup station because the performance was abysmal on garage. i also dont really get the point of zerofs. Seems like you are building a posix fs on top of s3 that is built on top of a posix fs... seems like a lot of extra steps that probably degrade performance vs just backing up that first posix fs semi regular.

> i also dont really get the point of zerofs. Seems like you are building a posix fs on top of s3 that is built on top of a posix fs

That can literally be the stack, but the same argument could be made about object storage itself: it ultimately writes data to local disks, so why not write files there directly?

The value of object storage isn’t just its API. It’s the durability, distribution, failure handling, and capacity behind that API. ZeroFS lets POSIX applications use those properties. Its server and cache are replaceable and the bucket remains the source of truth.

If one machine and its disks are enough for your data and reliability needs, then yes, a local filesystem is simpler. ZeroFS isn’t intended to improve that case.

Re: ZeroFS vs. Amazon S3 Files

#34

Post by tandr »

pierrebarre wrote: post #21
tandr wrote: post #14

Earlier quoted context omitted.

I'm thinking about using this and have a few questions: 1. How are hardlinks and duplicate files (same content, different paths) handled? 2. Does deduplication work on a block/chunk level for partially matching files, or does it only look at whole files? 3. Is there any specific integration or handling for Copy-on-Write (CoW)? Thank you!

Hardlinks work as you’d expect: multiple paths point to the same inode and data, so a write through one path is visible through the others. Two separate files with the same contents are stored separately. There’s no deduplication, either whole-file or block-level. That’s intentional, mostly because of the impact it would have on locality. If by CoW you mean reflinks, those aren’t currently planned either. They avoid…

Thank you for the quick answer.
Post reply on HN