Netflix Billing Migration to AWS

Does anyone happen to know how MS actually charges per core these days? I found: https://www.microsoft.com/en-us/Licensing/learn-more/brief-l... which points to http://go.microsoft.com/fwlink/?LinkID=229882 -- taken together, it looks like AMD hex-core+cpus count at .75 cores, single-core cpus count as 4, and dual-cores count as 2. Given that you need to buy 2-packs, it appears you can get a single 2-pack for a single dual-core (hyper-treading appears to be ignored), 2 packs for a single single-core cpu, or 3 two-packs for 8 AMD cpus?

Really? Because Amazon, Netflix, Facebook, and Google all have stake in the MySQL ring (google least so). MySQL has a lot of good knowns at scale, too.

The usage of DRDB somewhat eliminates the need for built in master/master replication, since its handled at the block layer. The two disks are always in sync, thus you don't need mysql's master/master setup. One of the main features missing from Postgres is that feature. Since it's not being used in their Mysql setup many people might ask why they wouldn't just use Postgres.

For starters, your company is not the largest video provider in the world, like Netflix. As someone else mentioned here, diversification can be for "political" reasons as well. Especially if you're as large as Netflix; diversifying could keep your billing in check. Let's say you split between AWS, Azure and GCS. That gives you the ability to scale based on what is cheapest. You could also do public and private clouds in tandum. You can scale your private cloud/on-premise to the public cloud if need be with most large cloud providers (though I never suggested they use on-premise.)

As far as this not being Amazon's primary business, well, they are a lot bigger and diverse. But let's use a Wal-Mart analogy. Wal-Mart is so powerful (maybe not anymore, because of Amazon) that if it doesn't like your wholesale pricing to them, they can move your shelf space and practically destroy your business. They have the leverage in those relationships. Now, with AWS being the #1 cloud provider you have a similar lock-in, at the very least your switching cost would be quite high.

So, let's just for example sake say Amazon make a decision to give their own video streaming priority on their own cloud. That's not breaking net neutrality laws, because...it's their servers, this has nothing to do with telecom. So now Amazon Prime Video, streams 4K at a much better rate than Netflix (and Netflix would likely never know.) Then, perhaps another assumption, AWS decides to increase their pricing tier for their media servers for streaming. So suddenly they have squeezed you in two ways. Quality of Service and pricing.

Amazon could easily do a calculation to compare themselves to Azure and GCS to tell what is the proper amount they could get away with and still make it more expensive just to migrate to a competitor. You're locked-in, and just to make up for your switching, it would cost, let's say 1 years worth of AWS service. Hard to explain your sudden blip in earnings to short minded investors.

Anyhow, all I am trying to say is...it would make more sense, to me, from a business stand point to diversify across multiple providers instead of being all in on AWS. I never said you had to have an on-premise set-up. It would be much different if they were a small start-up, but Netflix is not. It can at times take up more internet traffic than torrenting.

I will point out though that services like Docker are making cloud lock-ins harder to do, but they don't solve reliance on APIs or just proprietary offerings. There's a reason why Google, Microsoft and Amazon all underline, push and constantly improve those offerings as a form of lock-in.

It is possible Netflix has some sort of pricing agreement with Amazon that locks in a rate for x amount of years. But either way, when you're as large as Netflix, I think diversification is the better long term strategy.

> That means commit returns before replication to peer is complete.

That's what I was talking about. It's just a matter of what aspect of it you are talking about, but I'll give you that it's in their official documentation, so there's no point in me pressing the issue.

> Committed data can be lost if master's disk is destroyed before a slave replicates it.

That is true. It's a trade-off you can make for slightly different CAP assurances, or nuances in the failure states at least (mostly in what you might expect to do in a split brain scenario).

> For complication, you need to find out and configure the binary log position.

Your backups should be logging the binary log position as well (--dump-slave or --maser-data). If they aren't, you aren't doing yourself any favors.

> Also because replication is on the binary log, if you ever truncate the log, you can't simply add a brand new master. You have to do a backup on the primary and restore to the new master, and then set up the log position to just prior to backup. Just lots of extra complication.

If you have to do another backup because it was truncated recently, you aren't much worse off than doing backups with DRBD replication (which even if you have a slave configured and do backups off that, you can truncate logs and need to backup from the master then as well). The downside is that you may not want to immediately do a backup of the master due to reasons of load, which will leave you without a failover for a short while. Whether an extra queryable resource available is worth that is up the the architect.

I remember a Percona training I was at a few years back there were a few more clustering options available I hadn't played with (and still haven't). Percona XtraDB Cluster was one, and it's supposed to support synchronous master/master replication. That might be the best of both worlds, if it lives up to its billing.