Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

chooseacloud.com

Post reply on HN
Print view View original

41–50 of 115 posts

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#41

Post by alexbilbie »

icebraining wrote: post #28
alexbilbie wrote: post #18

What is the durability objects stored with DO? S3 offers 11x9s of durability. Likewise what is the replication story? Can you get event notifications when objects are uploaded/deleted? Is there versioning? Static website hosting? Lifecycle management?

S3 offers 11x9s of durability. So they say, but the SLA doesn't make any promises about durability. https://aws.amazon.com/s3/sla/

The SLA governs availability.

Details of durability here https://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurabili...

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#42

Post by notacoward »

Lunatic666 wrote: post #6

Is there any open source S3 compatible software? I know Riak Cloud Storage ( http://docs.basho.com/riak/cs/2.1.1/ ), but I think it’s not maintained anymore.

Minio and Ceph both fit this description, with the latter also offering block and file access. Swift is close, but a slightly different native API (there's an adapter but I'm not sure of its status). Scality has open-sourced their implementation as Zenko. Also LeoFS (which despite the name is not a filesystem).

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#43

Post by truetraveller »

aclelland wrote: post #39
truetraveller wrote: post #4

I was really excited about DO spaces. I compared every major Object Storage (OVH,B2,Wasabi,S3,Azure). DO spaces came out much ahead. I did dozens of hours of research. I was a customer (and I still am). But I am less excited now. Basically, there are loads of issues with rejected requests because of rate limiting (returns a lot of 503 "slow down" responses). Basically, I don't recall ever receiving this from S3. You…

S3 will sometimes return a "503 temporary error" response if you start writing lots of files per second. From my understanding, if they see that your bucket has a constant high write rate they'll make some configuration changes in the background to accommodate the higher write rate. That being said, last month I wrote over 60 million files to S3 and the number of failed writes were tiny (solved by simply retrying the…

Yes, DO spaces is very strict about both GET and PUT. I did a benchmark requests-per-second (literally just fetching a URL a bench of times). I get about ~180 requests per second, after which all requests failed.

Amazon S3 is much better in the sense that there IS dynamic scaling if it notices spikes.

To the defense of DO, they are newer and their business model is "cheap,cheap,cheap", so they can't compete at the same level.

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#45

Post by dsacco »

This is great, but I’m disappointed Backblaze B2 isn’t included. That seems like an oversight unless someone can point out how B2 doesn’t hold its own with these options in a glaring way. There are tradeoffs, but B2 seems to be very competitive overall.

B2 is cheaper than every option here for both storage ($0.005/GB) and egress ($0.01/GB).[1] Their transaction pricing is also cheaper.[2] Despite being cheaper, it’s still hot storage, so you can immediately download buckets, in whole or in part. I’ve personally used it to backup (and restore) terabytes of data for over a year. I doubt it has an SLA like GCP or AWS, but DigitalOcean doesn’t either, yet it’s listed here. I find the B2 API documentation to be very readable as well.[3]

I’ve used AWS S3, Glacier and GCP Nearline, Coldline. I can’t think of a specific thing that has disappointed me about B2, and the reliability has been excellent. The nature of my work is that I have very large datasets, and B2 becomes extremely competitive when you’re backing up tens of terabytes or more.

_________________________________

1. https://www.backblaze.com/b2/cloud-storage-pricing.html

2. https://www.backblaze.com/b2/b2-transactions-price.html

3. https://www.backblaze.com/b2/docs/

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#47

Post by kryptkpr »

SlowBro wrote: post #38

Maybe I'm missing something, but where is the Azure data?

Also missing the SoftLayer/IBM S3-compatible offering. We use it very lightly (our IoT firmware builds from CI are going up there) and pay nothing, their free limits are quite generous.

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#48

Post by truetraveller »

dsacco wrote: post #45

This is great, but I’m disappointed Backblaze B2 isn’t included. That seems like an oversight unless someone can point out how B2 doesn’t hold its own with these options in a glaring way. There are tradeoffs, but B2 seems to be very competitive overall. B2 is cheaper than every option here for both storage ($0.005/GB) and egress ($0.01/GB).[1] Their transaction pricing is also cheaper.[2] Despite being cheaper, it’s…

I really wanted to like B2. I love the docs and super-clean interface. But B2 was just too "weird" when it came to uploading an object. You can't replace an object key, so you always get a new object key when uploading. This is unlike any normal object store, where you can just upload to an existing key.

Also, if you upload an object with the same name (e.g. myphoto.png) it creates a new version, and there's no way to stop this. I don't want or need a new version!

I have a feeling this restriction is in place because of the underlying "vault" implementation.

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#49

Post by beepbeepbeep1 »

truetraveller wrote: post #48
dsacco wrote: post #45

This is great, but I’m disappointed Backblaze B2 isn’t included. That seems like an oversight unless someone can point out how B2 doesn’t hold its own with these options in a glaring way. There are tradeoffs, but B2 seems to be very competitive overall. B2 is cheaper than every option here for both storage ($0.005/GB) and egress ($0.01/GB).[1] Their transaction pricing is also cheaper.[2] Despite being cheaper, it’s…

I really wanted to like B2. I love the docs and super-clean interface. But B2 was just too "weird" when it came to uploading an object. You can't replace an object key, so you always get a new object key when uploading. This is unlike any normal object store, where you can just upload to an existing key. Also, if you upload an object with the same name (e.g. myphoto.png) it creates a new version, and there's no way t…

B2 is great as a backup store. I use it for backup.

I wouldn't use it for non backup object storage, as it's in a single data center. S3 and Google Cloud are completely different, I'm not sure on the Google specifics but S3 has data replicated across three AZs.

Re: Object Storage: AWS vs Google Cloud Storage vs Azure Storage vs DigitalOcean

#50

Post by ngrilly »

alexbilbie wrote: post #18

What is the durability objects stored with DO? S3 offers 11x9s of durability. Likewise what is the replication story? Can you get event notifications when objects are uploaded/deleted? Is there versioning? Static website hosting? Lifecycle management?

I asked DO about this, and their answer was that 1) data are replicated on-site (in the same data center) but are not replicated to another data center; and 2) as for now Spaces does not provide lifecycle management.

As a side note, I think lifecycle management is very useful for backups. A server push its backups to the object storage, but cannot overwrite or delete previous backups. This is useful if the server is hacked...

Post reply on HN