Backblaze B2 Cloud Storage • Hacker News Board

Re: Backblaze B2 Cloud Storage

#51

Post by brianwski » Tue, Sep 22, 2015, 5:38 PM UTC

toomuchtodo wrote: post #26

jcreedon wrote: post #2

The biggest shortcoming I see compared to the other big players (AWS, Azure, Google), and it is something they don't mention, is that they only have one datacenter, compared to the several from the other big players. The pricing is quite incredible though. I suspect if enough people hop on board with this they will probably look into setting up another datacenter.

Something to note: Unless you're storing data in us-east-1, all other regions in AWS are "one datacenter". Yes, they have AZs, but those aren't datacenters, they're just compartmentalized segments of the same datacenter. So! If you can tolerate the loss of a datacenter, store in Blackblaze. If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 (which is geo-redundant between Virginia and Oregon)…

> If you need geo-redundancy until Backblaze can offer it? Store in us-east-1

Or store it in Amazon AND store another copy in Backblaze. This isn't necessarily an "either/or" question. Having two copies with two different vendors in two separate regions is probably more reliable than having two copies inside the same vendor. For example, if Amazon has a large outage that affects both your regions, you can still access the copy in Backblaze.

Re: Backblaze B2 Cloud Storage

#52

Post by toomuchtodo » Tue, Sep 22, 2015, 5:40 PM UTC

gamegoblin wrote: post #41

toomuchtodo wrote: post #26

Earlier quoted context omitted.

Something to note: Unless you're storing data in us-east-1, all other regions in AWS are "one datacenter". Yes, they have AZs, but those aren't datacenters, they're just compartmentalized segments of the same datacenter. So! If you can tolerate the loss of a datacenter, store in Blackblaze. If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 (which is geo-redundant between Virginia and Oregon)…

Do you have a source that all other regions are "one datacenter"? All I could find on the S3 FAQ says "your objects are redundantly stored on multiple devices across multiple facilities." which seems to contradict the "one datacenter" claim. Also, do you have a source that us-east-1 is geo-redundant between Virginia and Oregon? That was not my understanding of how it worked.

AWS considers multiple facilities to be separate AZs in the same region. If you want multi region durability (besides us-east-1), you need cross region replication enabled (from the same FAQ you read).

"You specify a region when you create your Amazon S3 bucket. Within that region, your objects are redundantly stored on multiple devices across multiple facilities. Please refer to Regional Products and Services for details of Amazon S3 service availability by region"

Note, "within that region". Separate AZs, same geographic location.

"CRR is an Amazon S3 feature that automatically replicates data across AWS regions. With CRR, every object uploaded to an S3 bucket is automatically replicated to a destination bucket in a different AWS region that you choose. You can use CRR to provide lower-latency data access in different geographic regions. CRR can also help if you have a compliance requirement to store copies of data hundreds of miles apart."

This post http://shlomoswidler.com/2009/12/read-after-write-consistenc... has a quote from Jeff Barr at AWS indicating that us-east-1 is bicoastal, which is also why its eventually consistent, instead of immediately after a write (EDIT: it appears this constraint no longer applies to the US standard region).

Re: Backblaze B2 Cloud Storage

#53

Post by toomuchtodo » Tue, Sep 22, 2015, 5:42 PM UTC

brianwski wrote: post #51

toomuchtodo wrote: post #26

Earlier quoted context omitted.

Something to note: Unless you're storing data in us-east-1, all other regions in AWS are "one datacenter". Yes, they have AZs, but those aren't datacenters, they're just compartmentalized segments of the same datacenter. So! If you can tolerate the loss of a datacenter, store in Blackblaze. If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 (which is geo-redundant between Virginia and Oregon)…

> If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 Or store it in Amazon AND store another copy in Backblaze. This isn't necessarily an "either/or" question. Having two copies with two different vendors in two separate regions is probably more reliable than having two copies inside the same vendor. For example, if Amazon has a large outage that affects both your regions, you can still acces…

If you're going to pick two providers, use Backblaze and Google. Google's Nearline Storage is still more reliable (AWS only offers a 98% SLA on a monthly basis for S3 IA storage class) and cheaper (if I recall properly) than AWS' Intermediate Availability offering.

Re: Backblaze B2 Cloud Storage

#54

Post by toomuchtodo » Tue, Sep 22, 2015, 5:44 PM UTC

DinkyG wrote: post #44

toomuchtodo wrote: post #26

Earlier quoted context omitted.

Something to note: Unless you're storing data in us-east-1, all other regions in AWS are "one datacenter". Yes, they have AZs, but those aren't datacenters, they're just compartmentalized segments of the same datacenter. So! If you can tolerate the loss of a datacenter, store in Blackblaze. If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 (which is geo-redundant between Virginia and Oregon)…

This is incorrect. If you look for news articles about Amazon constructing data centers or buying facilities you'll notice that they have multiple data center facilities in each region.

Have a citation? My data is directly from AWS docs and speaking with AWS staff.

Re: Backblaze B2 Cloud Storage

#55

Post by BillinghamJ » Tue, Sep 22, 2015, 5:44 PM UTC

toomuchtodo wrote: post #26

jcreedon wrote: post #2

The biggest shortcoming I see compared to the other big players (AWS, Azure, Google), and it is something they don't mention, is that they only have one datacenter, compared to the several from the other big players. The pricing is quite incredible though. I suspect if enough people hop on board with this they will probably look into setting up another datacenter.

Something to note: Unless you're storing data in us-east-1, all other regions in AWS are "one datacenter". Yes, they have AZs, but those aren't datacenters, they're just compartmentalized segments of the same datacenter. So! If you can tolerate the loss of a datacenter, store in Blackblaze. If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 (which is geo-redundant between Virginia and Oregon)…

This is incorrect on almost all points.

All AWS AZs are physically separated facilities with redundancy on all their infrastructure, although they're obviously in the same general area.

us-east-1 is not geo-redundant. It is entirely on the east-coast, as the name suggests. Although S3 does have geo-redundancy in all regions.

You may have been thinking of "US Standard", but it is the same as "us-east-1".

http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_r...

Re: Backblaze B2 Cloud Storage

#56

Post by devit » Tue, Sep 22, 2015, 5:44 PM UTC

Finally some more reasonable prices in this space.

Eventually it could make sense for Backblaze to partner with someone like DigitalOcean or Linode and offer low cost bulk storage and low cost virtualization colocated in the same datacenter: these services seem to be a perfect complement for each other.

Re: Backblaze B2 Cloud Storage

#57

Post by DinkyG » Tue, Sep 22, 2015, 5:45 PM UTC

toomuchtodo wrote: post #52

gamegoblin wrote: post #41

Earlier quoted context omitted.

Do you have a source that all other regions are "one datacenter"? All I could find on the S3 FAQ says "your objects are redundantly stored on multiple devices across multiple facilities." which seems to contradict the "one datacenter" claim. Also, do you have a source that us-east-1 is geo-redundant between Virginia and Oregon? That was not my understanding of how it worked.

AWS considers multiple facilities to be separate AZs in the same region. If you want multi region durability (besides us-east-1), you need cross region replication enabled (from the same FAQ you read). "You specify a region when you create your Amazon S3 bucket. Within that region, your objects are redundantly stored on multiple devices across multiple facilities. Please refer to Regional Products and Services for de…

Multiple facilities != "one datacenter"

Re: Backblaze B2 Cloud Storage

#58

Post by brianwski » Tue, Sep 22, 2015, 5:45 PM UTC

benjojo12 wrote: post #36

brianwski wrote: post #35

Earlier quoted context omitted.

Or a price savings! If you do the math, I think the break even is at 1 TByte. If you only need to backup 500 GBytes from your Linux server then you'll save 50%.

Not server, desktop, Hence my annoyance you don't have a client for it. I can understand your biz reasons for not having one though.

Now with B2 we immediately support Linux and provide a client for it out of the box (written in python). Granted, it is only a command line interface so give us a little time to polish it up and add some features.

Re: Backblaze B2 Cloud Storage

#59

Post by toomuchtodo » Tue, Sep 22, 2015, 5:45 PM UTC

DinkyG wrote: post #57

toomuchtodo wrote: post #52

Earlier quoted context omitted.

AWS considers multiple facilities to be separate AZs in the same region. If you want multi region durability (besides us-east-1), you need cross region replication enabled (from the same FAQ you read). "You specify a region when you create your Amazon S3 bucket. Within that region, your objects are redundantly stored on multiple devices across multiple facilities. Please refer to Regional Products and Services for de…

Multiple facilities != "one datacenter"

Like I asked below, do you have a citation?

Re: Backblaze B2 Cloud Storage

#60

Post by toomuchtodo » Tue, Sep 22, 2015, 5:46 PM UTC

BillinghamJ wrote: post #55

toomuchtodo wrote: post #26

Earlier quoted context omitted.

Something to note: Unless you're storing data in us-east-1, all other regions in AWS are "one datacenter". Yes, they have AZs, but those aren't datacenters, they're just compartmentalized segments of the same datacenter. So! If you can tolerate the loss of a datacenter, store in Blackblaze. If you need geo-redundancy until Backblaze can offer it? Store in us-east-1 (which is geo-redundant between Virginia and Oregon)…

This is incorrect on almost all points. All AWS AZs are physically separated facilities with redundancy on all their infrastructure, although they're obviously in the same general area. us-east-1 is not geo-redundant. It is entirely on the east-coast, as the name suggests. Although S3 does have geo-redundancy in all regions. You may have been thinking of "US Standard", but it is the same as "us-east-1". http://docs.a…

> This is a feature for EU and US-West. US Standard is bi-coastal and doesn’t have read-after-write consistency.

Quote from Jeff Barr @ AWS: http://shlomoswidler.com/2009/12/read-after-write-consistenc....