231–240 of 329 posts
We are still working hard on it, hoping that we can help people with different workloads with different tech!
Founder of JuiceFS here, congrats to the Launch! I'm super excited to see more people doing creative things in the using-S3-as-file-system space. When we started JuiceFS back in 2017, applied YC for 2 times but no luck. We are still working hard on it, hoping that we can help people with different workloads with different tech!
If the caching layer can return success before writing through to s3, it means you built a strongly consistent distributed in memory database.
Or, the consistency guarantee is actually less, or data is partitioned and cannot be quickly shared across clients.
I'm really curious to understand how this was implemented.
Wow, coincidentally I posted GlassBD ( https://news.ycombinator.com/item?id=42164058 ) a couple of days ago. Making S3 strongly consistent is not trivial, so I'm curious about how you achieved this. If the caching layer can return success before writing through to s3, it means you built a strongly consistent distributed in memory database. Or, the consistency guarantee is actually less, or data is partitioned and can…
Wow, coincidentally I posted GlassBD ( https://news.ycombinator.com/item?id=42164058 ) a couple of days ago. Making S3 strongly consistent is not trivial, so I'm curious about how you achieved this. If the caching layer can return success before writing through to s3, it means you built a strongly consistent distributed in memory database. Or, the consistency guarantee is actually less, or data is partitioned and can…
Hey, thanks for reaching out. The caching layer does return success before writing to S3 -- that's how we get good performance for all operations, including those which aren't possible to do in S3 efficiently (such as random writes, renames, or file appends). Because the caching layer is durable, we can safely asynchronously apply these changes to the S3 bucket. Most operations appear in the S3 bucket within a minute…
How are concurrent updates to the same file handled? Either only one client can open in write at any one time, or you need fencing tokens.
Earlier quoted context omitted.
Hey, thanks for reaching out. The caching layer does return success before writing to S3 -- that's how we get good performance for all operations, including those which aren't possible to do in S3 efficiently (such as random writes, renames, or file appends). Because the caching layer is durable, we can safely asynchronously apply these changes to the S3 bucket. Most operations appear in the S3 bucket within a minute…
Very nice, I like the approach. I assume data is partitioned and each file is handled by an elected leader? If data is replicated, you still need a consensus algorithm on updates. How are concurrent updates to the same file handled? Either only one client can open in write at any one time, or you need fencing tokens.
For concurrent updates, the standard practice for remote file systems is to use file locking to coordinate concurrent writes. Otherwise, NFS doesn't have any guarantees about WRITE operation ordering. If you're talking about concurrent writes which occur from NFS and S3 simultaneously, this leads to undefined behavior. We think that this is okay if we do a good job at detecting and alerting the user if this occurs because we don't think that there are applications currently written to do this kind of simultaneous data editing (because Regatta didn't exist yet).
Earlier quoted context omitted.
Very nice, I like the approach. I assume data is partitioned and each file is handled by an elected leader? If data is replicated, you still need a consensus algorithm on updates. How are concurrent updates to the same file handled? Either only one client can open in write at any one time, or you need fencing tokens.
Without getting too much into internals which could change at any time, yes. You have to replicate, partition, and serve consensus over data to achieve high-durability and availability. For concurrent updates, the standard practice for remote file systems is to use file locking to coordinate concurrent writes. Otherwise, NFS doesn't have any guarantees about WRITE operation ordering. If you're talking about concurren…
Consistency at the individual file can be guaranteed this way, but I don't think this works across multiple files (as you need a global total order of operations). In any case, this is a pragmatic solution, and I like the tradeoffs. Comparing against NFS rather than Spanner seems the right way to look at it.
Does it mean I can use Lambda + SQLite + Regatta to build a real pay-as-you-go ACID SQL storage? Edit: an production-ready (high durability) ACID SQL storage
Yes! This is my expectation. Lots of the big companies have already done this with in-house architecture. With Regatta, we want to democratize building stateless applications that can take advantage of the low-cost storage of S3.
Earlier quoted context omitted.
Do you pay for metadata accesses? Does running a `find` across the filesystem cost anything? What about system calls that don't transfer data? Can I move or rename a file without paying to copy and then delete the associated S3 object?
Today, we only charge for cache usage (storage) and data transfer between Regatta and S3. If your metadata access doesn't require transfer to S3, then it doesn't cost anything! However, renames do require transfer to S3 (because we have to move the object on the backend).