Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 329 posts

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#71

Post by bassp »

huntaub wrote: post #49
bassp wrote: post #44

This feels, intuitively, like it would be very hard to make crash consistent (given the durable caching layer in between the client and S3). How are you approaching that?

It depends on what you mean by crash-consistent. I would expect that we handle crash-consistency at the client fine (since it is the same crash-consistency of NFSv3) and craash-consistency at the server also fine (since we are able to detect using etags what version of an object is in the backing data storage). Tell me a bit more about what you're thinking.

For sure! Upon reflection, maybe I’m less curious about crash consistency (corruption or whatever) per-se, and more about what kinds of durability guarantees I can expect in the presence of a crash.

I’m specifically interested in how you’re handling synchronization between the NFS layer and S3 wrt fsync. The description says that data is “asynchronously” written back out to S3. That implies to me that it’s possible for something like this to happen:

1. I write to a file and fsync it

2. Your NFS layer makes the file durable and returns

3. Your NFS layer crashes (oh no, the intern merged some bad terraform!) before it writes back to S3

4. I go to read the file from S3… and it’s not there!

Is that possible? IE is the only way to get a consistent view of the data by reading “through” the nfs layer, even if I fsync?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#72

Post by jitl »

I’m very interested in this as a backing disk for SQLite/DuckDB/parquet, but I really want my cached reads to come straight from instance-local NVMe storage, and to have a way to “pin” and “unpin” some subdirectories from local cache.

Why local storage? We’re going to have multiple processes reading & writing to the files and need locking & shared memory semantics you can’t get w/ NFS. I could implement pin/unpin myself in user space by copying stuff between /mnt/magic-nfs and /mnt/instance-nvme but at that point I’d just use S3 myself.

Any thoughts about providing a custom file system or how to assemble this out of parts on top of the NFS mount?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#75

Post by huntaub »

mritchie712 wrote: post #65

Pretty sure we're in your target market. We [0] currently use GCP Filestore to host DuckDB. Here's the pricing and performance at 10 TiB. Can you give me an idea on the pricing and performance for Regatta? Service Tier: Zonal Location: us-central1 10 TiB instance at $0.35/TiB/hr Monthly cost: $2,560.00 Performance Estimate: Read IOPS: 92,000 Write IOPS: 26,000 Read Throughput: 2,600 MiB/s Write Throughput: 880 MiB/s…

Yes, you should be in our target market. I don't think that I can give a cost estimate without having a good sense of what percentage of your data you're actively using at any given time, but we should absolutely support the performance numbers that you're talking about. I'd love to chat more in detail, feel free to send me a note at hleath [at] regattastorage.com.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#77

Post by count »

huntaub wrote: post #51
count wrote: post #45

I don't see any other question about it, so maybe I just missed the obvious answer, but how do you handle POSIX ACLs? If the data is stored as an object in S3, but exposed via filesystem, where are you keeping (if at all?) the filesystem ACLs and metadata? Also, NFSv3 and not 4?

Great call out. Some kinds of data, like ACLs and specific kinds of metadata, don't live in S3. Full disclosure, we don't support ACLs today (but plan to soon). We keep file system metadata in the durable cache. For some files (where users haven't changed permissions, etc), we are able to release that cached metadata when the file is no longer in use. For other files (where permissions have been changed by the user),…

Thanks, I keep hoping someone comes up with some magic :)

Is the intent to run this in-vpc?

And how do you differentiate from AWS Storage Gateway?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#78

Post by huntaub »

weinzierl wrote: post #74

The title says POSIX but then it talks about NFS. So, what is it? Does it guarantee all POSIX semantics or not?

You are correct in that NFS is not strictly-speaking POSIX compliant to the letter of the law, due to the caching behavior. This is an NFSv3 file system, so it shares those semantics. The point that I'm trying to emphasize is that the file system supports standard file operations which aren't possible through other FUSE adapters, or possible to perform efficiently on S3 (such as append, rename, and symbolic links) -- which provides broad compatibility with file-based applications.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#79

Post by ec109685 »

discodave wrote: post #69
huntaub wrote: post #63

Earlier quoted context omitted.

Those costs only apply to data transfer into and out of AWS. If you're running EC2 instances in AWS, your Regatta file system is in AWS, and your S3 bucket is in AWS -- then you shouldn't incur additional data transfer fees.

Where you say AWS, you mean "a single AWS region" But anyway, from your YCombinator blurb: "When you’re done editing data, it automatically flows back to S3 within a few minutes" Does this mean Regatta trades consistency for cost (S3 and EBS and local storage are all CP systems these days)?

It async replicates to s3, while providing a consistent interface to storage clients.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#80

Post by huntaub »

austinpena wrote: post #73

Reminds me of https://www.lucidlink.com/ for video editors. I quite like the experience with them.

That's exactly right, I've spoken with a ton of folks who have had a good experience with Lucid Link. I think that we are in a slightly different part of the market (in that we aren't targeting video editors, and more of data-intensive applications which may use thousands of IOPS), but I appreciate that the technology is likely similar.
Post reply on HN