291–300 of 329 posts
It takes somtimes years to fill it up with photos, vidoes and other documents. Sounds like one could build a great killer low amortized – pay as you fill it up – service for people to compete with dropbox.
Noob Question: When an average person buys 2TB storage from a cloud provider one pays upfront for the entire thing. Would pricing for a product be made more competitive(vs dropbox) using such a solution? It takes somtimes years to fill it up with photos, vidoes and other documents. Sounds like one could build a great killer low amortized – pay as you fill it up – service for people to compete with dropbox.
On the other hand, something like Dropbox is actually a program running on your laptop that simulates a file system, and then does the synchronization at the file level as needed. I think that there's probably some latent demand for a similar product for developers to access their S3 buckets easily from their laptops, and it's something we might look into as we get farther along.
That is interesting, but I haven't read how it is implemented yet. The hard part is a cache layer with immediate consistency. It likely requires RAFT (or, otherwise, works incorrectly). Integration of this cache layer with S3 (offloading cold data to S3) is easy (not interesting). It should not be compared to s3fs, mountpoint, geesefs, etc., because they lack consistency and also slow and also don't support full file…
I'll be looking closely in what you're building!
[1] https://www.dell.com/en-us/blog/welcoming-spanning-maginatic...
[2] https://www.slideshare.net/slideshow/maginatics-sdcdwl/39257...
Earlier quoted context omitted.
All connected file system clients see read-after-write consistency, so you see the up to date file data!
I heard you about the "limited hands, infinite wishlist" but nowadays when I see someone making bold claims about transactions and consistency over the network, I grab my popcorn bucket and eagerly await the Jepsen report about it The good news is that you, personally, don't have to spend the time to create the Jepsen test harness, you can pay them to run the test but I have no idea what kind of O($) we're talking he…
I am the same, as distributed consensus is notoriously hard especially when it fronts distributed storage.
However, it is not imposssible.. Hunter and I were both in the EFS team at AWS (I am still there), and he was deeply involved in all aspects of our consensus and replication layers. So if anyone can do it, Hunter is!
I used the same approach based on Rclone for a long time. I wondered what makes Regatta Storage different than Rclone. Here is the answer: "When performing mutating operations on the file system (including writes, renames, and directory changes), Regatta first stages this data on its high-speed caching layer to provide strong consistency to other file clients." [0]. Rclone, on the contrary, has no layer that would gu…
I suppose rclone doesn't provide byte range file locking? Running sqlite over rclone would be a disaster.
In (March?) 2007 (correction 2008) myself and two other engineers in front of Bruce Chizen - Adobe's CEO in a small conference room in Bucharest demoed a photo taken with an iPhone automagically showing as a file on a Mac. I implemented the local FUSE talking to Ozzy - Adobe's distributed object store back then, using an equivalent of a Linux inode structure. It worked like a charm and if I remember correctly it took…