Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 329 posts

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#101

Post by mritchie712 »

_bare_metal wrote: post #84
mritchie712 wrote: post #65

Pretty sure we're in your target market. We [0] currently use GCP Filestore to host DuckDB. Here's the pricing and performance at 10 TiB. Can you give me an idea on the pricing and performance for Regatta? Service Tier: Zonal Location: us-central1 10 TiB instance at $0.35/TiB/hr Monthly cost: $2,560.00 Performance Estimate: Read IOPS: 92,000 Write IOPS: 26,000 Read Throughput: 2,600 MiB/s Write Throughput: 880 MiB/s…

Out of curiosity, why not go bare metal in a managed colocation? Is that for the geographic spread? Or unpredictable load? Every few months of this spend is like buying a server Edit: back at my pc and checked, relevant bare metal is ~$500/m, amortized: https://baremetalsavings.com/c/LtxKMNj Edit 2: for 100tb..

agreed, one month of 50 TiB is $12,800!

we're using Filestore out of convenience right now, but actively exploring alternatives.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#102

Post by huntaub »

mritchie712 wrote: post #98
huntaub wrote: post #75

Earlier quoted context omitted.

Yes, you should be in our target market. I don't think that I can give a cost estimate without having a good sense of what percentage of your data you're actively using at any given time, but we should absolutely support the performance numbers that you're talking about. I'd love to chat more in detail, feel free to send me a note at hleath [at] regattastorage.com.

I'll send you a note! Found this in the docs: > By default, Regatta file systems can provide up to 10 Gbps of throughput and 10,000 IOPS across all connected clients. Is that the lower bound? The 50 TiB filestore instance has 104 Gbps read through put (albeit at a relatively high price point).

That's just the limit that we apply to new file systems. We should be able to support your 104 Gbps of read throughput.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#103

Post by huntaub »

doctorpangloss wrote: post #99
huntaub wrote: post #55

Earlier quoted context omitted.

Great question! We fill the same role as AWS Storage Gateway (and I used to work closely with that team when I was at AWS, lots of respect for what they do). AWS Storage Gateway is built primarily as an appliance to be installed on instances in your own data center to ease migration to the cloud. Many customers do deploy Storage Gateway on EC2 because they want these features in the cloud itself. However, the "applia…

S3 File Gateway sounds a lot like your product.

Also true! If you look at their site, they're really targeting folks to deploy it into their data centers to provide on-premises caching of resources in AWS, rather than providing a high-speed cache within AWS for file-based applications.

https://aws.amazon.com/storagegateway/file/s3/

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#104

Post by aidos »

huntaub wrote: post #87
debarshri wrote: post #86

There are quite some noteworthy alternatives like s3fs, rclone, goofys etc.

This is accurate! A lot of people have spent a lot of time trying to build a good file system abstraction on cheap, S3 storage. However, Regatta differs from these solutions in two important ways. First, Regatta is a shared, durable caching layer that sits between your instances and S3. This means that Regatta is able to efficiently perform operations (like directory renames) and provide strong consistency to other f…

Super interesting product. I have a couple of questions:

In terms of storing in s3 - is that in your buckets? Sound like the plan is to run the caching on your infrastructure, are there plans to allow customers to run those instances themselves?

Presumably the format within s3 is your own bespoke format? What does the migration strategy look like for people looking to move into or out of your infrastructure? They effectively pull everything down from their s3 to the local “filesystem”?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#105

Post by eerikkivistik »

huntaub wrote: post #100
eerikkivistik wrote: post #96

Can you elaborate on a few things with regards to your pricing: * What does "$0.05 / gigabyte transferred" mean exactly. Transferred outside of AWS or accessed as in read and written data? * "$0.20/GiB-mo of high-speed cache" – how is the high-speed cache amount computed?

Sure, and we have more details on pricing here which may answer your questions: https://docs.regattastorage.com/details/pricing We need to update the home page with these details, but $0.05 is only charged on transfer between Regatta and S3. We calculate your cache usage minutely and tally it into a monthly usage amount that we then bill for.

Thanks for clearing that up. Few followup questions:

You don't actually directly charge for storage itself, so I assume this a "bring your own s3 bucket" type of deal, correct?

How long does data, that is no longer being accessed sit in the cache and count towards billing?

As for availability, are you in the process or do you have plans to also support Google Cloud?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#106

Post by huntaub »

aidos wrote: post #104
huntaub wrote: post #87

Earlier quoted context omitted.

This is accurate! A lot of people have spent a lot of time trying to build a good file system abstraction on cheap, S3 storage. However, Regatta differs from these solutions in two important ways. First, Regatta is a shared, durable caching layer that sits between your instances and S3. This means that Regatta is able to efficiently perform operations (like directory renames) and provide strong consistency to other f…

Super interesting product. I have a couple of questions: In terms of storing in s3 - is that in your buckets? Sound like the plan is to run the caching on your infrastructure, are there plans to allow customers to run those instances themselves? Presumably the format within s3 is your own bespoke format? What does the migration strategy look like for people looking to move into or out of your infrastructure? They eff…

I love this because it allows me to highlight the parts of the system that I'm most excited about. The Regatta caching runs on our infrastructure, but it connects to buckets that our customers control. We read and write data into the customer's bucket in a regular, native (not bespoke) format -- so you can connect a Regatta file system directly to a bucket that already exists, with data in it, and use that data from a file system without any data migration!

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#107

Post by huntaub »

eerikkivistik wrote: post #105
huntaub wrote: post #100

Earlier quoted context omitted.

Sure, and we have more details on pricing here which may answer your questions: https://docs.regattastorage.com/details/pricing We need to update the home page with these details, but $0.05 is only charged on transfer between Regatta and S3. We calculate your cache usage minutely and tally it into a monthly usage amount that we then bill for.

Thanks for clearing that up. Few followup questions: You don't actually directly charge for storage itself, so I assume this a "bring your own s3 bucket" type of deal, correct? How long does data, that is no longer being accessed sit in the cache and count towards billing? As for availability, are you in the process or do you have plans to also support Google Cloud?

> You don't actually directly charge for storage itself, so I assume this a "bring your own s3 bucket" type of deal, correct?

That's correct -- we store data in the customer S3 bucket.

> How long does data, that is no longer being accessed sit in the cache and count towards billing?

We keep data in the cache for up to 1 hour after you've stopped accessing it.

> As for availability, are you in the process or do you have plans to also support Google Cloud?

We have plans to support Google Cloud. If you're interested in using us from GCP, I'd recommend setting up some time to chat (either use the website or email me at hleath [at] regattastorage.com). We are prioritizing where we launch our infrastructure next based on customer demand.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#108

Post by eerikkivistik »

huntaub wrote: post #107
eerikkivistik wrote: post #105

Earlier quoted context omitted.

Thanks for clearing that up. Few followup questions: You don't actually directly charge for storage itself, so I assume this a "bring your own s3 bucket" type of deal, correct? How long does data, that is no longer being accessed sit in the cache and count towards billing? As for availability, are you in the process or do you have plans to also support Google Cloud?

> You don't actually directly charge for storage itself, so I assume this a "bring your own s3 bucket" type of deal, correct? That's correct -- we store data in the customer S3 bucket. > How long does data, that is no longer being accessed sit in the cache and count towards billing? We keep data in the cache for up to 1 hour after you've stopped accessing it. > As for availability, are you in the process or do you ha…

I might just take you up on that.

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#109

Post by huntaub »

Jayakumark wrote: post #61
huntaub wrote: post #30

Earlier quoted context omitted.

This is exactly the problem that we solve! You don't need any local disk on your EC2 instance in order to use Regatta or work with data in S3. Our high-speed caching layer plays the role as this local disk for you, so that you can work with data sets that are hundreds of TiBs, even if you only have a 20 GiB EBS volume on your instance.

What is the acceptable latency , if we have to use this outside of Ec2 , lets say mounting S3 from on-prem/GCP/Azure ?

Well, in my opinion, I want to deliver the lowest latency possible. I expect that we will have Regatta running in GCP and Azure within the next 6 months. I'd love to connect if there's a place on-prem that you're looking to use Regatta. Would you shoot an email to hleath [at] regattastorage.com, and we could chat about what you're looking for?

Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

#110

Post by weinzierl »

huntaub wrote: post #78
weinzierl wrote: post #74

The title says POSIX but then it talks about NFS. So, what is it? Does it guarantee all POSIX semantics or not?

You are correct in that NFS is not strictly-speaking POSIX compliant to the letter of the law, due to the caching behavior. This is an NFSv3 file system, so it shares those semantics. The point that I'm trying to emphasize is that the file system supports standard file operations which aren't possible through other FUSE adapters, or possible to perform efficiently on S3 (such as append, rename, and symbolic links) --…

Which is nice and useful of course but there is ton of things that can't reliably be done with that (like running any database you that comes to mind) which makes it important to be precise here.
Post reply on HN