Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

news.ycombinator.com

Post reply on HN
Print view View original

111–120 of 329 posts

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#111

Post by huntaub »

weinzierl wrote: post #110
huntaub wrote: post #78

Earlier quoted context omitted.

You are correct in that NFS is not strictly-speaking POSIX compliant to the letter of the law, due to the caching behavior. This is an NFSv3 file system, so it shares those semantics. The point that I'm trying to emphasize is that the file system supports standard file operations which aren't possible through other FUSE adapters, or possible to perform efficiently on S3 (such as append, rename, and symbolic links) --…

Which is nice and useful of course but there is ton of things that can't reliably be done with that (like running any database you that comes to mind) which makes it important to be precise here.

Is there something specific that you worry about when running a database on a networked file system? I would imagine that any database which is correctly fsync'ing the data to the write-ahead-log should work just fine.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#112

Post by jedberg »

mikecwang wrote: post #50

Does it mean I can use Lambda + SQLite + Regatta to build a real pay-as-you-go ACID SQL storage? Edit: an production-ready (high durability) ACID SQL storage

Curious as to why you would want to build that yourself when so many solutions already exist (Supabase, NeonDB, AWS Aurora or RDS, etc.)?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#113

Post by bobnamob »

huntaub wrote: post #59
Shoop wrote: post #39

Earlier quoted context omitted.

What are the consistency semantics?

All connected file system clients see strong, read-after-write consistency. Most file operations are synchronized to S3 within a few minutes of completion.

Do you do anything to handle/detect write conflicts?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#115

Post by inopinatus »

I rejected EFS as a common caching and shared files layer, despite being technologically an excellent fit for my stack, because it is astronomically expensive. The value created didn’t match the cost.

When I got in touch about that, I was confronted with a wall of TCO papers, which tells me the product managers evidently believe their target segment to be Gartner-following corporate drones. This was a further deterrent.

We threw that idea away and used memcached instead, with common static files in a package in S3.

I guess I’m suggesting, don’t be like EFS when it comes to pricing or reaching customers.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#116

Post by koolba »

huntaub wrote: post #6
koolba wrote: post #4

Neat stuff. I think everybody with an interest in NFS has toyed with this idea at some point. > Under the hood, customers mount a Regatta file system by connecting to our fleet of caching instances over NFSv3 (soon, our custom protocol). Our instances then connect to the customer’s S3 bucket on the backend, and provide sub-millisecond cached-read and write performance. This durable cache allows us to provide a strong…

Thanks for the question! > How do you handle the cache server crashing before syncing to S3? Do the cache servers have local disk as well? Our caching layer is highly durable, which is (in my opinion) the key for doing this kind of staging. This means that once a write is complete to Regatta, we guarantee that it will eventually complete on S3. For this reason, server crashes and intermittent S3 availability issues a…

Is it fair to say this is best suited for small files that will be written infrequently?

There’s no partial write for s3 so editing a small range of a 1 GiB file would repeatedly upload the full file to the backing s3 right?

Or is the s3 representation not the same hierarchy as the presented mount point? (ie something opaque like a log structured / append only chunked list)

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#117

Post by the_duke »

huntaub wrote: post #6
koolba wrote: post #4

Neat stuff. I think everybody with an interest in NFS has toyed with this idea at some point. > Under the hood, customers mount a Regatta file system by connecting to our fleet of caching instances over NFSv3 (soon, our custom protocol). Our instances then connect to the customer’s S3 bucket on the backend, and provide sub-millisecond cached-read and write performance. This durable cache allows us to provide a strong…

Thanks for the question! > How do you handle the cache server crashing before syncing to S3? Do the cache servers have local disk as well? Our caching layer is highly durable, which is (in my opinion) the key for doing this kind of staging. This means that once a write is complete to Regatta, we guarantee that it will eventually complete on S3. For this reason, server crashes and intermittent S3 availability issues a…

So, I assume you use a journal in the cache server.

A few related questions:

* Do you use a single leader for a specific file system, or do you have a cluster solution with consensus to enable scaling/redundancy?

* How do you guarantee read-after-write consistency? Do you stream the journal to all clients and wait for them to ack before the write finishes? Or at least wait for everyone to ack the latest revisions for files, while the content is streamed out separately/requested on demand?

* If the above is true, I assume this is strictly viable for single-DC usage due to latency? Do you support different mount options for different consistency guarantees?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#118

Post by huntaub »

kmclean wrote: post #114

Just want to say this is super cool. I'm excited to see what people build on top of it.. seems like it could enable a new category of hosted data platforms-as-a-service (platform-as-a-services?).

This is more or less exactly what I'm hoping for. I think that people are excited to build stateless applications, but often that requires really specialized application and storage knowledge to pull off. My hope is that people can use this generic storage layer to build the next generation of stateless applications (including things like databases) without having to become storage experts themselves. I'm also excited to see what they build.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#119

Post by huntaub »

jedberg wrote: post #112
mikecwang wrote: post #50

Does it mean I can use Lambda + SQLite + Regatta to build a real pay-as-you-go ACID SQL storage? Edit: an production-ready (high durability) ACID SQL storage

Curious as to why you would want to build that yourself when so many solutions already exist (Supabase, NeonDB, AWS Aurora or RDS, etc.)?

One of my hopes for Regatta is that we're able to power the next generation of these data platforms. These things work because the designers had specialized storage knowledge that allowed them to carefully build serverless data products. I hope that Regatta is generic enough to allow anyone to build a serverless data product moving forward, without having to think about their storage infrastructure.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#120

Post by huntaub »

the_duke wrote: post #117
huntaub wrote: post #6

Earlier quoted context omitted.

Thanks for the question! > How do you handle the cache server crashing before syncing to S3? Do the cache servers have local disk as well? Our caching layer is highly durable, which is (in my opinion) the key for doing this kind of staging. This means that once a write is complete to Regatta, we guarantee that it will eventually complete on S3. For this reason, server crashes and intermittent S3 availability issues a…

So, I assume you use a journal in the cache server. A few related questions: * Do you use a single leader for a specific file system, or do you have a cluster solution with consensus to enable scaling/redundancy? * How do you guarantee read-after-write consistency? Do you stream the journal to all clients and wait for them to ack before the write finishes? Or at least wait for everyone to ack the latest revisions for…

These are questions that are super specific to our implementation, that I'm hesitant to share publicly because they could change any at any time. I can share that we're designed to horizontally scale the performance of each file system, and our custom protocol will enable Lustre-like scale out performance. As for single- vs. multi-DC, I think that you'd be surprised at how much latency budget there is (a cross-DC round trip in AWS can be anywhere from 200us-700us, and EBS gp3 latencies are around 1000us).
Post reply on HN