ZeroFS vs. Amazon S3 Files

#1

Post by cbrewster » Sat, Jul 11, 2026, 6:15 PM UTC

ZeroFS vs. Amazon S3 Files
zerofs.net

Re: ZeroFS vs. Amazon S3 Files

#2

Post by Lucasoato » Sat, Jul 11, 2026, 8:42 PM UTC

The article diagrams can’t be seen well if the device has the dark mode as default, just a suggestion for the author.

Opened with Safari in iOS

Re: ZeroFS vs. Amazon S3 Files

#3

Post by ranger_danger » Sat, Jul 11, 2026, 9:22 PM UTC

Lucasoato wrote: post #2

The article diagrams can’t be seen well if the device has the dark mode as default, just a suggestion for the author. Opened with Safari in iOS

I only saw one diagram but it is perfectly visible for me on the desktop.

Re: ZeroFS vs. Amazon S3 Files

#4

Post by pierrebarre » Sat, Jul 11, 2026, 9:26 PM UTC

Author here. Thanks for posting this!

It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.

Re: ZeroFS vs. Amazon S3 Files

#5

Post by hhthrowaway1230 » Sat, Jul 11, 2026, 9:32 PM UTC

I really want this, but it feels a little scary to trust all my files. I wouls like it if there was some contineous suite trying to corrupt the files and then see the failure cases!

Re: ZeroFS vs. Amazon S3 Files

#6

Post by pierrebarre » Sat, Jul 11, 2026, 9:37 PM UTC

hhthrowaway1230 wrote: post #5

I really want this, but it feels a little scary to trust all my files. I wouls like it if there was some contineous suite trying to corrupt the files and then see the failure cases!

That’s a fair concern. The closest thing right now is a deterministic simulation suite that injects storage faults and crashes at arbitrary points, then checks the recovered data against reference models. It runs hourly with fresh seeds.

CI also runs pjdfstest, xfstests, stress-ng, ZFS scrubs, and Jepsen crash/failover tests: https://github.com/Barre/ZeroFS#testing

ZeroFS is still pretty young as storage projects go, so I completely understand wanting to see it prove itself over time.

Re: ZeroFS vs. Amazon S3 Files

#7

Post by hhthrowaway1230 » Sat, Jul 11, 2026, 9:39 PM UTC

Also would love to know how was we could iterate over the files. ie a bit how duckdb allows for paruet fikes scanning and reading would be nice to see how fast we can query the fs for ai/ml training workloads

Re: ZeroFS vs. Amazon S3 Files

#8

Post by chillfox » Sat, Jul 11, 2026, 9:58 PM UTC

pierrebarre wrote: post #4

Author here. Thanks for posting this! It’s been quite a ride building ZeroFS, and I’m happy to answer any questions.

Thanks for building this, I am just about to give it ago with my self-hosted Garage cluster.

Does running `stat` against a file require pulling the whole file from s3, or can that be handled by the metadata?

Do you know what backup performance is like for something like borg/borgmatic or restic, especially on follow up runs where most files are just checked.

Is there any particular Redis/Valkey config you recommend when using it for `conditional_put`, or just default config?

Is there any chance for NFSv4 support?

Re: ZeroFS vs. Amazon S3 Files

#9

Post by [deleted] » Sat, Jul 11, 2026, 10:05 PM UTC

hhthrowaway1230 wrote: post #5

I really want this, but it feels a little scary to trust all my files. I wouls like it if there was some contineous suite trying to corrupt the files and then see the failure cases!

[deleted]

Re: ZeroFS vs. Amazon S3 Files

#10

Post by smartbit » Sat, Jul 11, 2026, 10:06 PM UTC

Lucasoato wrote: post #2

The article diagrams can’t be seen well if the device has the dark mode as default, just a suggestion for the author. Opened with Safari in iOS

I have the same in Brave on iOS: to see the the diagrams I need to turn of Night Mode.