31–40 of 93 posts
So their AWS bill is $500k/year. Hire one person to buy, configure, manage hardware is what? $250k/year? And is this self-hosting more reliable than AWS? For an email service, you'd think that you want the most reliable host. Weird.
Could anyone expand on why it makes sense for Netflix to use AWS so extensively?
Earlier quoted context omitted.
The bulk of the media is served from appliances they give out to ISPs, so their AWS costs might be reasonable. It's also a huge company and a massive distributed system, so moving that off from AWS is an insane amount of effort and there might be legitimate benefits from AWS' management and autoscaling layer that aren't as valuable for 37Signals which runs simpler, monolithic systems.
Additionally, Netflix falls into one category of website DHH writes about: unpredictable, significant bursts of traffic. When they publish Tiger King during the pandemic there’s a spike in streaming no one could have predicted. When people finish the show it goes away. The arrival time of the next breakout show is likely uncertain, even if it can be modeled using a statistical distribution.
It still cost /far/ less to buy/rack/power etc. our servers 100% of the time than it would have done to use the cloud and scale up and down with demand.
There is definitely a situation when bursty requirements will make cloud the more obvious answer, but given the near order of magnitude cost increase for cloud it's not as clear cut as just seeing a wave on a graph :)
This is a good point. I really dont like the centralization on AWS.
> If one of the primary AWS regions go down, seemingly half the internet is offline along with it. This is not what DARPA designed! This is a good point. I really dont like the centralization on AWS.
So their AWS bill is $500k/year. Hire one person to buy, configure, manage hardware is what? $250k/year? And is this self-hosting more reliable than AWS? For an email service, you'd think that you want the most reliable host. Weird.
> Hire one person Go back and re-read :) They [believe that they] don't need to hire anyone. They already have a staff of ops folks that run their cloud operations. And those folks will run their on-prem operations.
If you're constantly installing stuff then you're scaling up a lot, and that's the level where it would make sense to have someone hired dedicated for it.
You can also rent hardware.
the thing is: cloud ops are also quite complex but it's a different kind of complexity. I don't know how many people are required.
The largest teams I've been on have been cloud deployments, and some of the smallest teams were bare-metal; but that's just an anecdote.
> If one of the primary AWS regions go down, seemingly half the internet is offline along with it. This is not what DARPA designed! This is a good point. I really dont like the centralization on AWS.
That's really only us-east-1. It's a special pet region, don't use it.
Curious to know how to which degree they are self hosting, seems they'll buy their own servers but I doubt they'll go all the way to buying land and building their own data centers?
When I was there, they rented space in a DC outside Chicago (near the airport), and later near NY for a backup location. The DC in Chicago was fantastic. We had a whole load of racks (about a row in the DC), and we bought our own servers. Running our own DC would have crossed the line, it's a whole huge thing you need to do and it only makes sense at scale if you want to do it well, Basecamp used a tiny fraction of t…
A whole load of Racks for running Basecamp? We are talking about 42U per Rack, and total of 420U of Servers?
The scale seems quite massive. At least to the idea / perception of what I had about Basecamp. Would be nice to see those specification and see how much of an improvement it is 10 years later and if we could fit those into 2-3 Racks.
Could anyone expand on why it makes sense for Netflix to use AWS so extensively?
2. Content Encoding are not done at a regular interval. And most of their work on ML and other quality requirements does not require fixed interval either. i.e They are schedule to work in Spot when AWS price are at its lowest.
3. Netflix get huge discount anyway.