Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effectively using AWS Reserved Instances

stripe.com

Post reply on HN
Print view View original

21–30 of 123 posts

Re: Effectively using AWS Reserved Instances

#21

Post by hueving »

jedberg wrote: post #10
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

How exactly does that solve the problem? Don't you have to do the same capacity planning to decide how many servers to buy for your datacenter? Except you get less flexibility because you can't buy servers and have them instantly available like you can for reserved instances? Also, what kind of workloads are you running that don't require databases? The biggest expense in any distributed system is moving data. If you…

Right, you have to do the same capacity planning, but you are getting the massive upside involved in that work instead of Amazon.

>What you might save in not paying AWS's profit margin you will probably spend in not being able to be as efficient as they are.

This isn't how I've seen the numbers work out for the huge chunk of workloads that require mostly static instances (a.k.a haven't been modernized into a serverless code base). You are right about Amazon having an efficiency edge, but you are wrong about that benefit being to the customer's bottom line instead of theirs.

We are nowhere near the real commoditized pricing of massive scale compute. Even with the inefficiency of smaller datacenters, you can easily best AWS prices.

Where did you get the impression that you have to move all of the data into the cloud for every bursted request? That's a lazy strawman architecture to attack.

Re: Effectively using AWS Reserved Instances

#23

Post by toomuchtodo »

eldavido wrote: post #20
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

This is terrible advice for all but the largest of organizations. Running your own hardware is AWFUL. Get ready to dedicate an entire team to network engineering, fixing broken hard disks, patching operating systems, screwing around with RAID controllers, upgrading switches, planning power and cooling, and endless vendor negotiation -- with ISPs, hardware manufacturers, datacenter operators, etc. Oh, and did I mentio…

Netflix runs a third of internet traffic off of their OpenConnect CDN appliances, not AWS (Netflix.com and other control plane-ish functions aside).

Once you’re spending a few million dollars a year on cloud (arguably even less in some cases), it behooves you to explore hosting yourself. Cloud provider margins are substantial for a reason.

Re: Effectively using AWS Reserved Instances

#24

Post by Johnny555 »

eldavido wrote: post #20
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

This is terrible advice for all but the largest of organizations. Running your own hardware is AWFUL. Get ready to dedicate an entire team to network engineering, fixing broken hard disks, patching operating systems, screwing around with RAID controllers, upgrading switches, planning power and cooling, and endless vendor negotiation -- with ISPs, hardware manufacturers, datacenter operators, etc. Oh, and did I mentio…

It's been a few years since doing the cost benefit analysis between AWS and self-hosting, but for around 50 racks worth of servers and storage, the numbers came in on AWS's side.

That didn't even take into account the "free" multi-region capability you get from Amazon. Splitting our physical servers into a second region with enough capacity to failover would have nearly doubled our costs.

Re: Effectively using AWS Reserved Instances

#25

Post by eldavido »

toomuchtodo wrote: post #23
eldavido wrote: post #20

Earlier quoted context omitted.

This is terrible advice for all but the largest of organizations. Running your own hardware is AWFUL. Get ready to dedicate an entire team to network engineering, fixing broken hard disks, patching operating systems, screwing around with RAID controllers, upgrading switches, planning power and cooling, and endless vendor negotiation -- with ISPs, hardware manufacturers, datacenter operators, etc. Oh, and did I mentio…

Netflix runs a third of internet traffic off of their OpenConnect CDN appliances, not AWS (Netflix.com and other control plane-ish functions aside). Once you’re spending a few million dollars a year on cloud (arguably even less in some cases), it behooves you to explore hosting yourself. Cloud provider margins are substantial for a reason.

If you're truly spending "a few million dollars"/year, then MAYBE.

200k/month would get you there. My personal breakpoints are 50k/mo PaaS->EC2/IaaS and then something like 100k or 150k start thinking about physical. Maybe. By the time you hire everyone, get all the planning right, etc. you might be there need-wise, but it's not a sure thing.

Re: Effectively using AWS Reserved Instances

#26

Post by niftich »

eldavido wrote: post #20
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

This is terrible advice for all but the largest of organizations. Running your own hardware is AWFUL. Get ready to dedicate an entire team to network engineering, fixing broken hard disks, patching operating systems, screwing around with RAID controllers, upgrading switches, planning power and cooling, and endless vendor negotiation -- with ISPs, hardware manufacturers, datacenter operators, etc. Oh, and did I mentio…

I get what you're saying about the benefits of PaaS or other managed services, but Netflix serves most content (by volume) off of appliances [1] they ship to ISPs that feed into the network at points-of-presence somewhere close to an edge. They use AWS for logic and compute (including transcoding [3]), but video streams from this globally-distributed CDN [2].

[1] https://openconnect.netflix.com/en/ [2] https://media.netflix.com/en/company-blog/how-netflix-works-... [3] https://medium.com/netflix-techblog/high-quality-video-encod...

Re: Effectively using AWS Reserved Instances

#27

Post by [deleted] »

eldavido wrote: post #25
toomuchtodo wrote: post #23

Earlier quoted context omitted.

Netflix runs a third of internet traffic off of their OpenConnect CDN appliances, not AWS (Netflix.com and other control plane-ish functions aside). Once you’re spending a few million dollars a year on cloud (arguably even less in some cases), it behooves you to explore hosting yourself. Cloud provider margins are substantial for a reason.

If you're truly spending "a few million dollars"/year, then MAYBE. 200k/month would get you there. My personal breakpoints are 50k/mo PaaS->EC2/IaaS and then something like 100k or 150k start thinking about physical. Maybe. By the time you hire everyone, get all the planning right, etc. you might be there need-wise, but it's not a sure thing.

[deleted]

Re: Effectively using AWS Reserved Instances

#28

Post by [deleted] »

eldavido wrote: post #20
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

This is terrible advice for all but the largest of organizations. Running your own hardware is AWFUL. Get ready to dedicate an entire team to network engineering, fixing broken hard disks, patching operating systems, screwing around with RAID controllers, upgrading switches, planning power and cooling, and endless vendor negotiation -- with ISPs, hardware manufacturers, datacenter operators, etc. Oh, and did I mentio…

[deleted]

Re: Effectively using AWS Reserved Instances

#29

Post by jedberg »

hueving wrote: post #21
jedberg wrote: post #10

Earlier quoted context omitted.

How exactly does that solve the problem? Don't you have to do the same capacity planning to decide how many servers to buy for your datacenter? Except you get less flexibility because you can't buy servers and have them instantly available like you can for reserved instances? Also, what kind of workloads are you running that don't require databases? The biggest expense in any distributed system is moving data. If you…

Right, you have to do the same capacity planning, but you are getting the massive upside involved in that work instead of Amazon. >What you might save in not paying AWS's profit margin you will probably spend in not being able to be as efficient as they are. This isn't how I've seen the numbers work out for the huge chunk of workloads that require mostly static instances (a.k.a haven't been modernized into a serverle…

You don't have to move all the data, but you have to constantly move the needed data back and forth, unless you store a second copy in the cloud. And then you have to start capacity planning again.

> This isn't how I've seen the numbers work out for the huge chunk of workloads that require mostly static instances.

Any time someone says this I have to question if they really looked at the "all in" number. Did you include the salary of the person in purchasing who orders the servers? Did you include the lost engineering time dealing with dead servers (instead of just shutting them off)? Did you include the cost of spare hardware sitting around for emergencies? Did you include the cost of downtime due to broken hardware while waiting for it to be repaired or replaced?

There are so many other costs to running your own datacenter besides the servers and the space, which Amazon gets to amortize over all their customers, but you have to bear 100% on your own.

Re: Effectively using AWS Reserved Instances

#30

Post by jjeaff »

Johnny555 wrote: post #24
eldavido wrote: post #20

Earlier quoted context omitted.

This is terrible advice for all but the largest of organizations. Running your own hardware is AWFUL. Get ready to dedicate an entire team to network engineering, fixing broken hard disks, patching operating systems, screwing around with RAID controllers, upgrading switches, planning power and cooling, and endless vendor negotiation -- with ISPs, hardware manufacturers, datacenter operators, etc. Oh, and did I mentio…

It's been a few years since doing the cost benefit analysis between AWS and self-hosting, but for around 50 racks worth of servers and storage, the numbers came in on AWS's side. That didn't even take into account the "free" multi-region capability you get from Amazon. Splitting our physical servers into a second region with enough capacity to failover would have nearly doubled our costs.

Why would you compare AWS vs managing your own data center?

You could also compare AWS vs building your own silicon.

I think it would be better to compare AWS vs renting dedicated servers from a large provider? I think you will find that the scales tip heavily in favor of renting bare metal as far as price is concerned.

Post reply on HN