Introducing HTTP Load Balancing and SSD Persistent Disk

Introducing HTTP Load Balancing and SSD Persistent Disk
googlecloudplatform.blogspot.com

Softlayer has a similar product called Global IP where traffic enters their extensive network at the local POP. You can then reroute the IP to any of your servers but it’s not load balanced and is really just another IP. Still a good product which we use to power all our services for automated failover. But Google just one-upped them.

Think AWS elastic IP, but Softlayer allow you to route to any server in any DC. And Google allow you to automatically route to a pool of servers in a location closest to the user.

I'm going to consider 3 cases:

CDN: 1TB of storage, 1000 IOPS required. (a cache or "smart"/ACLed CDN)

DB: 20GB of storage, 5000 IOPS required. (a moderately sized database)

Hybrid: 500GB of storage, 2500 IOPS required. (some composite of the two)

A summary of pricing right now for storage IOPS:

  (US East)
                            $/GB  $/IOPS    CDN      DB    Hybrid
  Google Cloud Platform    0.325  $0.011   $ 325   $  54   $ 163
  Amazon EC2/EBS           0.125  $0.100   $ 225   $ 502   $ 313
  Microsoft Azure          0.050   n/a     $  50   $   1   $  25
  Microsoft Azure (w/vm)   0.050   n/a     $  85   $ 278   $ 164

Google: pay for the GB, get a fixed IOPS/GB.

Amazon: pay for the GB, pay for the IOPS.

Microsoft: pay for the GB, get 500 IOPS/disk, with a maximum of 16 disks. (but you can't attach that many disks unless you pay for a larger instance size, too)

Microsoft (w/VM): I've included the cost of the compute instance in this summary. I used an A1 VM for CDN, A4 for DB, and A3 for Hybrid, at $35, $277, and $139 monthly, respectively.

CDN

Google: $325 for 1TB/30,000 IOPS.

Amazon: $225 for precisely 1TB/1,000 IOPS.

Microsoft: $50 (using 2 500GB disks)

Microsoft (w/vm): $85 (an A1 VM is required to use two disks)

DB

Google: $54 for 167GB/5,000 IOPS.

Amazon: $502 for 20GB/5,000 IOPS.

Microsoft: $1 for 20GB/5,000 IOPS (using 10 2GB disks)

Microsoft (w/vm): $278 (an A4 VM is required to use 9-16 disks)

Hybrid

Google: $162.50 for 500GB/15,000 IOPS.

Amazon: $312.50 for 500GB/2,500 IOPS.

Microsoft: $25 for 500GB/2500 IOPS (using 5 100 GB disks).

Microsoft (w/vm): $164 (an A3 VM is required to use 5-8 disks)

EDIT: I forgot to add a disclaimer. I am a student at the University of Northern Iowa. I am not affiliated with Amazon or Google. I am a Microsoft Partner, however. I was not paid or asked to write this post, and have never been directly paid by Microsoft to produce any writing, benchmark, or whatever, etc, etc. IANAL, etc. I have received standard gifts from them from attending conferences.

Google probably has the highest server turnover rate, and my wager is that they recently switched to new SSDs to reduce failure rates in their datacenters. My guess is that their compute servers are now swimming in excess IOPS, and so they are clearly the cheapest if you want very fast storage. This reminds me of Facebook purchasing massive quantities of SSDs for their own datacenters.

Microsoft has very peculiar needs. Azure grew out of servicing internal Microsoft IT needs first, so it is stuck with some quirks. You don't provision IOPS, you create disks, which you attach to VMs. There are no IOPS tunable options for their PaaS, only IaaS VMs. They use software RAID (Linux or Windows) to turn multiple disks into higher speed storage. Their storage backend can only be described as weird[1], because whatever you think the actual disk layout looks like from your VM's perspective, it doesn't look anything like that on the storage layer. They partition data into chunks (disks) which get written to a distributed, log-structured file system on JBODs. Every 1GB extant is made immutable once full, and they use erasure encoding to distribute the data such that they can sustain 2 failures with only 33% disk overhead. The result is that writes are cheap (almost all writes are sequential) and the backend does a bunch of compute work to make reads fast (with caching of metadata and the erasure encoding, I'm guessing).

[1] - http://snia.org/sites/default/files2/SDC2013/presentations/G...

[2] - http://research.microsoft.com/pubs/179583/LRC12-cheng%20webp...