221–230 of 329 posts
I wish you luck, having looked at doing something similar years back, I don’t see the market. In the case of what I was involved in, it pivoted to enterprise backup.
Also using a translation layer on top of S3 will not save your costs.
The main reason of adopting object storage is to avoid the burden associated with POSIX file system APIs. And this renders the major motivation using an object storage pointless. Also using a translation layer on top of S3 will not save your costs.
I agree with you, Object Storage accels at making the storage interface super simple to use (POSIX is incredibly complex). However, that doesn't change the reality that nearly all software still reads and writes data from a local file system interface.
The specifics of whether or not using a translation layer will save you costs comes down a lot to what you're comparing it to. If you have an EBS volume that's 20% full, then I guarantee you that Regatta's storage costs will be cheaper than EBS, even if you don't ever tier to S3. It's just a cherry on top for workloads which may have unpredictable access patterns and don't want all of their data to be hot when not in use.
S3 semantics are generally fairly terrible for file storage (no atomic move/rename is just one example) but using it as block storage a la ZFS is quite clever.
I've looked for a solution to write many small files fast (safely). Think about cloning thr Linux kernel git repo. Whatever I tested, the NFS protocol was always a bottleneck.
Is this meaningfully different from https://github.com/s3ql/s3ql ? S3 semantics are generally fairly terrible for file storage (no atomic move/rename is just one example) but using it as block storage a la ZFS is quite clever.
Why are these solutions always using NFS? I'm asking out of curiosity, not judgement. I've looked for a solution to write many small files fast (safely). Think about cloning thr Linux kernel git repo. Whatever I tested, the NFS protocol was always a bottleneck.
Earlier quoted context omitted.
> It's pure magic when you mount a filesystem on clean machine and all your data is "just there." I totally agree! I am hoping that Regatta can power a future where teams don't need more than ~8 GiB of local storage for their operating system, and can store the rest on something like Regatta to get rid of the waste of overprovisioned block volumes.
That would sell like hot cakes to the public sector.
At first glance it’s not clear how this is unique from Nasuni.
I have mutual friends with some of the Nasuni folks, and I have a lot of respect for what they do. In particular, Nasuni stores data in a proprietary block format in your S3 bucket, so you can't connect it to existing data sets or use that data directly from S3 out the other side. Whereas with Regatta, we store data in its native format in S3 so you can do these things.