Sia [1] could be even cheaper, coming out at $1.10 for a TB [2]. Although this price might not be forever, and I'm still not sure about the reliability. In theory it should be more reliable, as it's decentralised and your data gets split among multiple servers around the world. The question remains what if the Sia network itself stops being profitable and people all exit at the same time. Although the same could be s…
Curious as to why you edited your post to remove mention of the other service that you had listed originally?
Is there any open source S3 compatible software? I know Riak Cloud Storage ( http://docs.basho.com/riak/cs/2.1.1/ ), but I think it’s not maintained anymore.
OpenStack Swift or Ceph with Ceph Object Gateway. Don't use minio, it's a toy for testing.
That's libelously untrue. You might think Ceph or Swift are better, that's fine, but it's no toy. I've seen a few toy S3 implementations. I even called out Zettar on my blog back in the day. If it's not scalable or highly available it's a toy. If it's unstable it's a toy. If it's super-duper slow it's a toy. Minio is none of these things, plus it has features like erasure coding and encryption that are mature enough to be backed by real support. Best ever? I'm not saying that, but it's no toy.
Disclaimer: I have no affiliation with Minio. In fact, they're very nearly a competitor with the project I work on. However, Minio is full of ex-Gluster people and I consider many of them friends.
Sia [1] could be even cheaper, coming out at $1.10 for a TB [2]. Although this price might not be forever, and I'm still not sure about the reliability. In theory it should be more reliable, as it's decentralised and your data gets split among multiple servers around the world. The question remains what if the Sia network itself stops being profitable and people all exit at the same time. Although the same could be s…
Curious as to why you edited your post to remove mention of the other service that you had listed originally?
Sorry @cyberferret. I previously listed Storj [1] but did a bit of research and it didn't actually look cheaper when you consider bandwidth cost.
When you say way cheaper what do you mean? Like savings of ~$5-10 on a spend of $10-20? Or is it material? Like $50
Well, DO is a minimum of $5 per month (up to 250GB IIRC). 100GB on S3 with just uploads during the month will run under $2.
So $5 is way cheaper? if you need any of those data down the line, transferring out, any savings you have with AWS in the past year or years are going to be wiped out.
Curious as to why you edited your post to remove mention of the other service that you had listed originally?
Sorry @cyberferret. I previously listed Storj [1] but did a bit of research and it didn't actually look cheaper when you consider bandwidth cost. $0.015 per GB per month $0.05 per GB downloaded [1] https://storj.io/
Cool thanks! I presumed that it didn't meet some sort of criteria that was being discussed here. I just wanted to make sure there were no issues with the service itself per se.
The bandwidth egress charges on AWS (and GCP/Azure) are way too high. It almost seems cartel like. Bandwidth costs have dropped by a huge factor over the past few years; but none of this has been passed on. I really hope backblaze and/or DO manage to cause the big three some hurt on this and get them to reduce prices significantly; 7c/GB is really high these days.
What are bandwidth costs? This comes up again and again, people see HE.net / Cogent / Level3 / Hibernia / NTT whatever offering 10gig handoff IP transit at a colo neutral location for X per month and somehow determine the true cost. When I suspect the reality is that much of cost is invested in routing equipment both internally and externally, evolving SDN and all the goodness it affords, being able to offer SLA per instance in terms of mbit that can be pushed, being able to mitigate DDOS, hiring top network engineers and researchers, being fully redundant across multiple providers and protected circuits rather than just a lonely hand off - or in GCP case building out the network themselves with multiple SLA tiers, being able to mitagate network issues quicker than a PagerDuty alert at a banwidth blender. Granted I'd say there still probably cushioning in there but judging network costs by comparing to a 10gig handoff at a random colo neutral, or blender, or OVH/Hetznet commodity servers, or Linode/Vultr/DO seems unfair in my opinion.
(*) for a particular use case. The example is 200GB storage with 2000GB data transfer (out) every month. That's a LOT of data going out every month, so I am guessing the scenario is if you are hosting a photo library and lots of people are downloading every month. If however you are just using the service as an online storage to hold Not knocking either service - I actually use both, for different use cases.
I am not sure I find your argument particularly valid.
Yes, AWS S3 is cheaper if you have no egress traffic, and the monthly storage cost falls below $5, which is a minimum price for DO.
This scenario is very much a corner case. If you have egress traffic, or more than $5 worth of storage, then DO runs ahead quite fast.
I'm slightly disappointed that Backblaze B2 was not on the list, though.
I was really excited about DO spaces. I compared every major Object Storage (OVH,B2,Wasabi,S3,Azure). DO spaces came out much ahead. I did dozens of hours of research. I was a customer (and I still am). But I am less excited now. Basically, there are loads of issues with rejected requests because of rate limiting (returns a lot of 503 "slow down" responses). Basically, I don't recall ever receiving this from S3. You…
S3 will sometimes return a "503 temporary error" response if you start writing lots of files per second. From my understanding, if they see that your bucket has a constant high write rate they'll make some configuration changes in the background to accommodate the higher write rate.
That being said, last month I wrote over 60 million files to S3 and the number of failed writes were tiny (solved by simply retrying the write)
I've not used DOs spaces yet, I'd love to know if they guarantee read after write consistency. I know S3 has some issues with that depending on use case. Maybe it's time for another look at spaces.
The bandwidth egress charges on AWS (and GCP/Azure) are way too high. It almost seems cartel like. Bandwidth costs have dropped by a huge factor over the past few years; but none of this has been passed on. I really hope backblaze and/or DO manage to cause the big three some hurt on this and get them to reduce prices significantly; 7c/GB is really high these days.
What are bandwidth costs? This comes up again and again, people see HE.net / Cogent / Level3 / Hibernia / NTT whatever offering 10gig handoff IP transit at a colo neutral location for X per month and somehow determine the true cost. When I suspect the reality is that much of cost is invested in routing equipment both internally and externally, evolving SDN and all the goodness it affords, being able to offer SLA per…
I totally understand that but the price hasn't dropped in what, a decade? My home internet connection (which includes a shedload of last mile costs laying fibre to my apartment) has went from 2mbit/sec to 1gig/sec (for less money in real terms) in a similar timespan.
I'm not expecting it to be at the cost of a 10gig cogent connection at LINX, but some drop makes sense. DO/Backblaze pricing seems to be more on the money.
Keep in mind these provides also do charge on top for a lot of other networking services (VPCs, NATs etc) which often can really add up, so I'd expect some of the SDN capex to be absorbed by that.