Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TernFS – An exabyte scale, multi-region distributed filesystem

xtxmarkets.com

Post reply on HN
Print view View original

1–10 of 111 posts

Re: TernFS – An exabyte scale, multi-region distributed filesystem

#3

Post by heipei »

mrbluecoat wrote: post #2

Cool project and kudos for open sourcing it. Noteworthy limitation: > TernFS should not be used for tiny files — our median file size is 2MB.

Yeah, that was the first thing I checked as well. Being suited for small / tiny files is a great property of the SeaweedFS system.

Re: TernFS – An exabyte scale, multi-region distributed filesystem

#5

Post by pandemic_region »

mrbluecoat wrote: post #2

Cool project and kudos for open sourcing it. Noteworthy limitation: > TernFS should not be used for tiny files — our median file size is 2MB.

What happens if you put a tiny file on it then? Bad perf, possible file corruption, ... ?

Re: TernFS – An exabyte scale, multi-region distributed filesystem

#6

Post by redundantly »

pandemic_region wrote: post #5
mrbluecoat wrote: post #2

Cool project and kudos for open sourcing it. Noteworthy limitation: > TernFS should not be used for tiny files — our median file size is 2MB.

What happens if you put a tiny file on it then? Bad perf, possible file corruption, ... ?

Probably wasting space and lower performance.

Re: TernFS – An exabyte scale, multi-region distributed filesystem

#7

Post by rostayob »

ttfvjktesd wrote: post #4

How does TernFS compare to CephFS and why not CephFS, since it is also tested for the multiple Petabyte range?

(Disclaimer: I'm one of the authors of TernFS and while we evaluated Ceph I am not intimately familiar with it)

Main factors:

* Ceph stores both metadata and file contents using the same object store (RADOS). TernFS uses a specialized database for metadata which takes advantage of various properties of our datasets (immutable files, few moves between directories, etc.).

* While Ceph is capable of storing PBs, we currently store ~600PBs on a single TernFS deployment. Last time we checked this would be an order of magnitude more than even very large Ceph deployments.

* More generally, we wanted a system that we knew we could easily adapt to our needs and more importantly quickly fix when something went wrong, and we estimated that building out something new rather than adapting Ceph (or some other open source solution) would be less costly overall.

Re: TernFS – An exabyte scale, multi-region distributed filesystem

#8

Post by jleahy »

pandemic_region wrote: post #5
mrbluecoat wrote: post #2

Cool project and kudos for open sourcing it. Noteworthy limitation: > TernFS should not be used for tiny files — our median file size is 2MB.

What happens if you put a tiny file on it then? Bad perf, possible file corruption, ... ?

It's just not optimised for tiny files. It absolutely would work with no problems at all, and you could definitely use it to store 100 billion 1kB files with zero problems (and that is 100 terabytes of data, probably on flash, so no joke). However you can't use it to store 1 exabyte of 1 kilobyte files (at least not yet).
Post reply on HN