Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ZeroFS vs. Amazon S3 Files

zerofs.net

Post reply on HN
Print view View original

11–20 of 34 posts

Re: ZeroFS vs. Amazon S3 Files

#11

Post by ahofmann »

pierrebarre wrote: post #6
hhthrowaway1230 wrote: post #5

I really want this, but it feels a little scary to trust all my files. I wouls like it if there was some contineous suite trying to corrupt the files and then see the failure cases!

That’s a fair concern. The closest thing right now is a deterministic simulation suite that injects storage faults and crashes at arbitrary points, then checks the recovered data against reference models. It runs hourly with fresh seeds. CI also runs pjdfstest, xfstests, stress-ng, ZFS scrubs, and Jepsen crash/failover tests: https://github.com/Barre/ZeroFS#testing ZeroFS is still pretty young as storage projects go,…

Why the hell was this answer dead? I vouched for it, because I don't see what might be wrong with it.

Re: ZeroFS vs. Amazon S3 Files

#13

Post by pierrebarre »

chillfox wrote: post #8
pierrebarre wrote: post #4

Author here. Thanks for posting this! It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.

Thanks for building this, I am just about to give it ago with my self-hosted Garage cluster. Does running `stat` against a file require pulling the whole file from s3, or can that be handled by the metadata? Do you know what backup performance is like for something like borg/borgmatic or restic, especially on follow up runs where most files are just checked. Is there any particular Redis/Valkey config you recommend w…

Thanks!

stat doesn’t pull the file contents from S3; it only accesses the metadata tree, which is usually cached.

I haven’t benchmarked Borg or Restic specifically. Sequential writes can comfortably reach several Gbit/s. For follow-up runs, if they only stat unchanged files, that should stay entirely in metadata.

The default Redis/Valkey configuration should work fine for conditional_put. NFSv4 is unlikely for now. It would add a lot of surface area, and I’m pretty happy with where the 9P extensions are today.

Re: ZeroFS vs. Amazon S3 Files

#14

Post by tandr »

pierrebarre wrote: post #4

Author here. Thanks for posting this! It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.

I'm thinking about using this and have a few questions:

    1. How are hardlinks and duplicate files (same content, different paths) handled?
    2. Does deduplication work on a block/chunk level for partially matching files, or does it only look at whole files?
    3. Is there any specific integration or handling for Copy-on-Write (CoW)?
Thank you!

Re: ZeroFS vs. Amazon S3 Files

#15

Post by the_duke »

pierrebarre wrote: post #4

Author here. Thanks for posting this! It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.

I was investigating the design a little. Two big questions:

A) You notably don't write a recovery log (WAL/journal) for things not yet flushed, so data can be lost. Do you have plans to add this? I think it would be pretty crucial.

B) the system is single writer. Do you have plans for adding horizontal scalability so a writer can be dynamically selected and routed to, transparent to the client? (Or with client cooperation, but without forcing sharding on the user)

Re: ZeroFS vs. Amazon S3 Files

#17

Post by bayesnet »

Lucasoato wrote: post #2

The article diagrams can’t be seen well if the device has the dark mode as default, just a suggestion for the author. Opened with Safari in iOS

The page appears to be dark themed even without dark mode enabled; if I use Noir to force-add dark mode, then the diagrams indeed become difficult to read but this seems like a Noir issue more than a site issue

Re: ZeroFS vs. Amazon S3 Files

#18

Post by jijji »

s3 is expensive... there are a lot of cheap options. I think I pay $48/month for a linux vps with 8 cpus and 16tb of storage with interserver.net... the same storage on amazon s3 is $377/month lol

Re: ZeroFS vs. Amazon S3 Files

#19

Post by nubg »

pierrebarre wrote: post #4

Author here. Thanks for posting this! It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.

This blog post seems to have been written by an AI based on a prompt you gave it? Can you confirm or deny this?

Re: ZeroFS vs. Amazon S3 Files

#20

Post by Onavo »

jijji wrote: post #18

s3 is expensive... there are a lot of cheap options. I think I pay $48/month for a linux vps with 8 cpus and 16tb of storage with interserver.net... the same storage on amazon s3 is $377/month lol

Their bandwidth costs are overpriced yes but in terms of storage you are paying for RAID3 replicated across three data centers (configurable) and high availability. The engineering behind it is also formally verified with strict theoretical bounds on data loss. $48*3 = ~$150 is within the same ballpark if you factor in the managed services and cloud overhead.
Post reply on HN