Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A distributed Posix file system built on top of Redis and S3

github.com

Post reply on HN
Print view View original

31–40 of 120 posts

Re: A distributed Posix file system built on top of Redis and S3

#31

Post by daviesliu »

kortex wrote: post #5

This is neat! I am quite a fan of all the go based file systems that are springing up. Question: what are the main compare and contrast points between juice and seaweed fs? Here is a compendium for those interested: https://github.com/gostor/awesome-go-storage

Comparing to SeaweedFS, JuiceFS is more feature complete rather than basic read/write functionalities.

The core of seaweedfs is to manage many small blobs, you can use SeaweedFS together with JuiceFS to have a full featured POSIX file system.

Re: A distributed Posix file system built on top of Redis and S3

#32

Post by Benjamin_Dobell »

dis-sys wrote: post #24

before you want to use it in your project, make sure to have a look at the code - their codebase is pretty much comment free, very little number of tests. Other than the marketing term "posix file system", there is not any proof on such claim. I am also not sure how it is a "distributed" file system given its storage is entirely done by S3. Should I call my backup program that backups my data to S3 every night a "dis…

Looks like they run against a whole suite (several actually) of third-party tests: https://github.com/juicedata/juicefs/blob/main/fstests/Makef...

Re: A distributed Posix file system built on top of Redis and S3

#33

Post by daviesliu »

CoolGuySteve wrote: post #22

How much does it cost to use on S3? For example, how many GetObject and other misc non-free API calls does it use? And does it store data using intelligent tiering?

For small files (less than 4MB), the number of Get/Put/Delete is the same comparing using S3 directly. For larger file, each object in S3 is about 4MiB. Most of S3 Client will do the similar thing to GET/PUT small parts in parallel to speed things up. Overall, JuiceFS should use the similar number of GET/PUT request comparing to use S3 directly.

Second, all the List and Head request go to Redis, they are free, so you may save some cost on API costs.

Third, the frequently read data will be cached in your local disks, so you will also save some cost on GET/PUT requests.

Re: A distributed Posix file system built on top of Redis and S3

#34

Post by daviesliu »

CoolGuySteve wrote: post #22

How much does it cost to use on S3? For example, how many GetObject and other misc non-free API calls does it use? And does it store data using intelligent tiering?

The underlying S3 bucket still have intelligent tiering, you can also put life cycle rules on it.

Re: A distributed Posix file system built on top of Redis and S3

#36

Post by daviesliu »

dis-sys wrote: post #24

before you want to use it in your project, make sure to have a look at the code - their codebase is pretty much comment free, very little number of tests. Other than the marketing term "posix file system", there is not any proof on such claim. I am also not sure how it is a "distributed" file system given its storage is entirely done by S3. Should I call my backup program that backups my data to S3 every night a "dis…

The architect of JuiceFS is very close to GFS, which use a single node master for many years, even now. Since Redis is only responsible for metadata, a single node can serve hundreds of millions of files, and tens of thousands of IOPS, that should be enough for many use cases.

The term `distributed`, means that JuiceFS is not a `local` file system, or can only be used by single machine. JuiceFS should be qualified as distributed system, even the core part is the client, which could be used by many machines in the same time.

Re: A distributed Posix file system built on top of Redis and S3

#37

Post by daviesliu »

Benjamin_Dobell wrote: post #32
dis-sys wrote: post #24

before you want to use it in your project, make sure to have a look at the code - their codebase is pretty much comment free, very little number of tests. Other than the marketing term "posix file system", there is not any proof on such claim. I am also not sure how it is a "distributed" file system given its storage is entirely done by S3. Should I call my backup program that backups my data to S3 every night a "dis…

Looks like they run against a whole suite (several actually) of third-party tests: https://github.com/juicedata/juicefs/blob/main/fstests/Makef...

Since POSIX is so complicated that we would not trust the test we wrote to prove the compatibility.

So we use several third-party test suite, including fsx, pjdfstest, fsracer, flock. We also use third-party tools for benchmark, fio, mdtest and others.

Re: A distributed Posix file system built on top of Redis and S3

#39

Post by keyle »

So... a file system on top of a virtualized file system hosted on someone else's computer across the land, with "Outstanding Performance: The latency can be as low as a few milliseconds" ... Milliseconds disk access is outstanding?

I mean, amazing, and, maybe you know, use a file system.

Re: A distributed Posix file system built on top of Redis and S3

#40

Post by catmanjan »

keyle wrote: post #39

So... a file system on top of a virtualized file system hosted on someone else's computer across the land, with "Outstanding Performance: The latency can be as low as a few milliseconds" ... Milliseconds disk access is outstanding? I mean, amazing, and, maybe you know, use a file system.

S3 can be cheaper
Post reply on HN