Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

cloudrail.com

Post reply on HN
Print view View original

41–50 of 69 posts

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#41

Post by blantonl »

okreallywtf wrote: post #31
tmm_ wrote: post #6

Earlier quoted context omitted.

This allows you to easily integrate multiple of these providers if necessary or easily switch. Of course you can integrate them one by one on your own but that takes a lot of time time. With CloudRail it's one API which works even cross platform. Our ultimate goal is to handle all your integrations and not only cloud storage. So unified APIs for fast API integrations and API Change Management to keep your integration…

Can you provide a use-case for when you would need to be able to easily switch (or potentially hot-swap) cloud storage? It sounds like some of the justification for ORM's; while it would be nice to have if it took no more time to integrate with that API its hard for me to see how it could justify extra effort in implementation/maintenance unless you have a concrete understanding of when you need it.

Pricing.

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#42

Post by kevan »

yandie wrote: post #38
kevan wrote: post #36

Earlier quoted context omitted.

>I can't help wonder if eventually consistent object stores will go the way of NoSQL databases, when a consistent, scalable hierarchical filesystem appears. Of course they will. Eventual consistency is a huge tradeoff that I don't think anyone would make if they weren't forced to.

Then you trade availability with consistency. No system can't escape CAP

Exactly. Availability and partition tolerance are often hard requirements, so we have to do all sorts of gymnastics to deal with eventual consistency. In personal projects where availability isn't a big concern and my most precious resource is my own time I tend to make different tradeoffs.

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#43

Post by Scaevolus »

jamesblonde wrote: post #30
Scaevolus wrote: post #17

Earlier quoted context omitted.

S3 only has read-after-write consistency for PUTs to new objects (not overwrites), and if you do a HEAD/GET before the PUT that degrades into being eventually consistent. http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction....

It's so bad that big companies, like Netflix, have to write a whole service to store a copy of the s3 metadata: http://techblog.netflix.com/2014/01/s3mper-consistency-in-cl... Netflix have turned the S3 metadata eventually consistent problem into s3mpr metadata eventually consistent problem. The difference is that they can now inspect and reason about s3mpr's metadata. Spotify have had to do the same thing for Google…

GCS is a bit better, since individual objects are strongly consistent, but I've been bitten by the eventually consistent listing.

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#45

Post by ejdyksen »

gaul wrote: post #25
jameshart wrote: post #15

Do all these services provide equivalent or abstractable guarantees? Amazon S3, for example, provides 'read-after-write' consistency, meaning once you've received a positive response to a put operation, you can expect to be immediately able to retrieve that object. But it used to be 'eventually consistent', meaning it was possible to receive a positive response to a put, but then not be able to read the object immedi…

You can compare these guarantees and results from testing: https://github.com/andrewgaul/are-we-consistent-yet

That is pretty old, considering what S3 has changed in the past two years. In particular, it talks about reading S3's us-standard region from the East and West coasts, and that has evolved to where us-standard (which is really us-east-1) is now the same as all other regions when it comes to consistency (presumably because all requests physically go to the East coast now, but I haven't tested that):

http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction....

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#46

Post by gaul »

ejdyksen wrote: post #45
gaul wrote: post #25

Earlier quoted context omitted.

You can compare these guarantees and results from testing: https://github.com/andrewgaul/are-we-consistent-yet

That is pretty old, considering what S3 has changed in the past two years. In particular, it talks about reading S3's us-standard region from the East and West coasts, and that has evolved to where us-standard (which is really us-east-1) is now the same as all other regions when it comes to consistency (presumably because all requests physically go to the East coast now, but I haven't tested that): http://docs.aws.am…

Fair point, I will update the AWS us-standard results and include Backblaze as well.

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#47

Post by bedros »

gaul wrote: post #21

S3Proxy offers similar multi-protocol access at the S3 API layer: https://github.com/andrewgaul/s3proxy It has compatibility with local filesystem, Backblaze B2, Google Cloud Storage, Microsoft Azure, and OpenStack Swift. Note that this is software and not a service. [S3Proxy author plug]

is this compatible with django-storages (which supports S3)

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#48

Post by bedros »

eriksssss wrote: post #8

Also check out libcloud by Apache. Pretty solid Python API wrapping various cloud services. :)

is this compatible with django-storages (which supports S3)

Update: actually it does http://django-storages.readthedocs.io/en/latest/backends/apa...

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#49

Post by 45h34jh53k4j »

Its nice but what incentive does cloudrail have to provide perpetual gratis access to someone elses API? Unlike an app that can still work when the original developer goes broke, an API requires that always on access.

Does my whole build break because cloudrail can't handle the traffic anymore? At least with AWS or GCS or Azure, etc, one has a business relationship with the API provider and so can keep the lights on.

Re: Show HN: Amazon S3, Microsoft Azure, Rackspace, Backblaze via a Single API

#50

Post by tmm_ »

45h34jh53k4j wrote: post #49

Its nice but what incentive does cloudrail have to provide perpetual gratis access to someone elses API? Unlike an app that can still work when the original developer goes broke, an API requires that always on access. Does my whole build break because cloudrail can't handle the traffic anymore? At least with AWS or GCS or Azure, etc, one has a business relationship with the API provider and so can keep the lights on.

CloudRail doesn't touch the data. Everything flows P2P between the app and the provider. It's like if you integrate the native AWS SDK. So even if CloudRail goes completely down, your integrations will keep running.
Post reply on HN