I wish you luck, having looked at doing something similar years back, I don’t see the market. In the case of what I was involved in, it pivoted to enterprise backup.
Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
211–220 of 329 posts
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#212Congrats! What a great solution, wish you success.
NIT: The forced smooth scrolling on the landing page drives me crazy! haha
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#213This is honestly the coolest thing I've seen coming out of YC in years. I have a bunch of questions which are basically related to "how does it work" and please pardon me if my questions are silly or naive! 1. If I had a local disk which was 10 GB, what happens when I try to contend with data in the 50 GB range (as in, more that could be cached locally?) Would I immediately see degradation, or thrashing, at the 10 GB…
Wow, thanks for the nice note! No questions are silly, and I'll also note that we now have a docs site ( https://docs.regattastorage.com ) and feel free to email me (hleath [at] regattastorage.com) if I don't fully address your questions. > If I had a local disk which was 10 GB, what happens when I try to contend with data in the 50 GB range (as in, more that could be cached locally?) Would I immediately see degradat…
Are you planning to support android? How? AFAIK android doesn't have FUSE or NFS.
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#214Earlier quoted context omitted.
Wow, thanks for the nice note! No questions are silly, and I'll also note that we now have a docs site ( https://docs.regattastorage.com ) and feel free to email me (hleath [at] regattastorage.com) if I don't fully address your questions. > If I had a local disk which was 10 GB, what happens when I try to contend with data in the 50 GB range (as in, more that could be cached locally?) Would I immediately see degradat…
Are you planning to support android? How? AFAIK android doesn't have FUSE or NFS.
I don't think that I'm planning support for Android, did I mistakingly mention it somewhere?
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#215Are there any tech details/architecture of the system? Also, congrats on launching.
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#216That is interesting, but I haven't read how it is implemented yet.
The hard part is a cache layer with immediate consistency. It likely requires RAFT (or, otherwise, works incorrectly). Integration of this cache layer with S3 (offloading cold data to S3) is easy (not interesting).
It should not be compared to s3fs, mountpoint, geesefs, etc., because they lack consistency and also slow and also don't support full filesystem semantics, and break often.
It could be compared with AWS EFS. Which is also slow (but I didn't try to tune it up to maximum numbers).
For ClickHouse, this system is unneeded because ClickHouse is already distributed (it supports full replication or shared storage + cache), and it does not require full filesystem semantics (it pairs with blob storages nicely).
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#217Hi! Would this work for a instance that uses Batman to backup Postgres servers?
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#218That is interesting, but I haven't read how it is implemented yet. The hard part is a cache layer with immediate consistency. It likely requires RAFT (or, otherwise, works incorrectly). Integration of this cache layer with S3 (offloading cold data to S3) is easy (not interesting). It should not be compared to s3fs, mountpoint, geesefs, etc., because they lack consistency and also slow and also don't support full file…
Thanks for the note, great to hear from you! I think that what Clickhouse does is great, and I expect that more applications want to take advantage of the low prices of S3 cold storage without needing to build their own application-level abstractions. I'm hopeful that this allows more of these next-generation serverless data products to exist.
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#219Hi! Would this work for a instance that uses Batman to backup Postgres servers?
Yes! I don't know of any reason why that wouldn't work. I've worked with lots of customers who need simple, low-cost storage for database backups.
Re: Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS
#220Are there any tech details/architecture of the system? Also, congrats on launching.
Thank you! We have docs at https://docs.regattastorage.com. There is an architecture page which might answer your questions. If you have deeper questions, feel free to ask in the thread or shoot me an email at hleath [at] regattastorage.com.