Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effectively using AWS Reserved Instances

stripe.com

Post reply on HN
Print view View original

81–90 of 123 posts

Re: Effectively using AWS Reserved Instances

#81

Post by rohan404 »

mathattack wrote: post #42
alex_young wrote: post #15

The notion that the break even point is 70% is ignoring some really important stuff. If you reserve workload x on hardware y for n years, you're effectively strapping yourself into a sure-to-be-obsolete and more expensive platform which you'll have to then move off of at an arbitrary point n years in the future. If you don't move, you wind up paying a premium to be stuck with the obsolete / more expensive platform ju…

You’re also locking in to outdated prices. Moore’s law is at your back with cheaper prices every year.

Not quite - the outdated price issue certainly exists for standard AWS RIs, however if you purchase convertible RIs then price reductions are applied.

Also bear in mind that the savings you get from RIs will generally outweigh the reduction in price of the instance.

Re: Effectively using AWS Reserved Instances

#82

Post by hinkley »

alex_young wrote: post #15

The notion that the break even point is 70% is ignoring some really important stuff. If you reserve workload x on hardware y for n years, you're effectively strapping yourself into a sure-to-be-obsolete and more expensive platform which you'll have to then move off of at an arbitrary point n years in the future. If you don't move, you wind up paying a premium to be stuck with the obsolete / more expensive platform ju…

I wonder how this is better than buying hardware. Some of the same problems but more flexibility than RI.

Re: Effectively using AWS Reserved Instances

#83

Post by mmt »

hueving wrote: post #66
cortesoft wrote: post #37

Earlier quoted context omitted.

Are you factoring in the cost of the people to deploy and maintain that infrastructure?

Yep, these people are called sysadmins. With AWS they are the ones who manage firewall rules, patching, etc on your instances as well as API keys etc. Some people like to call them devops because they didn't realize sysadmins could write scripts. Maintaining 40 racks of hardware takes a surprisingly trivial amount of time aside from regular OS management. The only products that actually eliminate these folks are thin…

> Maintaining 40 racks of hardware takes a surprisingly trivial amount of time aside from regular OS management.

I think it's only surprising to people who have been listening to the "it's awful" mythology or believe in a much higher than reality hardware failure rate.

I think the worst I've ever seen was with spinning disks from the time of the Thailand flooding, and even those were only 10-15% AFR and only for certain models.

Absent that kind of black swan event, one can easily engineer around even disk failures, such that failed ones can just be spun down and left in place, and, of course, one can do the same with server-level redundancy.

If the labor burden were actually onerous, it would be easy enough to avoid it, but I think it's telling that such techniques are so rarely spoken of.

Re: Effectively using AWS Reserved Instances

#84

Post by patio11 »

meritt wrote: post #22

Tangential to the point of the article but when did writing a select query become an "ETL process"?

Suppose you have a data source and business logic which you want to run periodically on the data source. Here are two scenarios which you could reasonably implement this as:

Method one: You write a SQL query and some Python. You put a sticky note on your computer "Remember to run that biweekly."

Method two: You pull up your shop's documentation for how to add the (BIG_NUMBER)th entry into the data processing pipeline. This gets you automatic scheduling, retries, monitoring, audit trails, alerts to the right people in case of breakage, etc etc. You write a SQL query and some Python. You plug it into the existing infrastructure.

Re: Effectively using AWS Reserved Instances

#85

Post by boulos »

sonnyblarney wrote: post #73
boulos wrote: post #59

Earlier quoted context omitted.

Disclosure: I work for Google Cloud. > I think it would be nice to have Google's offer, but then also a longer term 'lock in' low price option as well, as frankly, this fits a lot of businesses. We hear you. That's why we offer Committed Use Discounts [1]. Are you saying that a 3-year commitment to a specific price (or lower, as we do price cuts) is insufficient though? (I want to understand) [1] https://cloud.google…

I'm only making a very general reference to the fact that long-term visibility and predictability entails lower cost and therefore lower price, and that business owners are likely more empowered to determine that outlook than the cloud provider, either AWS or Google. Ergo - some kind of customer oriented long term lock-in would likely, in the long run, produce the cheapest prices in the system. That's all.

Ahh, I misunderstood your point! Yes, it’s certainly easier for us providers if you provide a clear demand signal. RIs, CUDs, SUDs, and even general contracting terms each provide some measure of information between customer and provider.

We actually had an interesting debate about this topic at the NSF Workshop on Cloud Economics [1]. The AWS person sadly had to cancel, but both some Google people and MSFT people were present along with CS and Economics academics. There are lots of industries where similar behavior exists, e.g., airlines or hotels. If you book in advance, or commit to a room block, you get a discount in exchange for certainty. We had lots of amusing debate about how similar cloud actually is to say energy markets (which turn out to be incredibly distorted, regulated, and confusing). Hopefully David and the rest of the organizers will have their summary report out within a few months.

[1] https://umass-sustainablecomputinglab.github.io/nsfw/

Re: Effectively using AWS Reserved Instances

#86

Post by mmt »

spookthesunset wrote: post #70
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

Opportunity costs are very, very real and very often overlooked when people give this kind of advice. For 95% if all workloads encountered by the types of people who dwell on this forum, there is zero value add in running your own data center. “Saving money” by running your own servers is penny wise, pound foolish. You’ll never, ever compete with AWS for features or cost. (And “you” means almost everybody reading thi…

> For 95% if all workloads encountered by the types of people who dwell on this forum,

I call shennanigans on this as a made-up statistic. You may be able to convince that percentage of people of that, but they may well not be aware of what's possible with commodity hardware.

> there is zero value add in running your own data center.

Sure there is. Money and maximum (I/O) performance.

> “Saving money” by running your own servers is penny wise, pound foolish. You’ll never, ever compete with AWS for features or cost.

You may be right about (software) features, but certainly not about cost. It's not "saving money" with quotes. It's actually saving money. You just have to hire someone that happens to know how to save money.

AWS will never, ever compete with you for hardware features or cost.

Re: Effectively using AWS Reserved Instances

#87

Post by scrollaway »

njovin wrote: post #75
msravi wrote: post #60

My experience with AWS reserved instances has not been very good previously. 1. Once you buy a reserved instance, you're locked in to that type and price for the duration, even though newer types at lower prices may get introduced (as they almost definitely would over 1-3 yrs). 2. If you're from outside the US, you might not be able to resell your reserved instance. So you're stuck with an old instance type at an inf…

I’ve gotten proactive emails from our account manager when they release new/cheaper instances and they offer us the option to transition and get a credit for our existing RIs. We aren’t a huge account (less than 30k/month) so I thought this was a nice gesture on Amazon’s part.

Our 15k/month account did not get such an email when they introduced the dc2 redshift class, despite most of our spend being on Redshift.

Re: Effectively using AWS Reserved Instances

#88

Post by [deleted] »

eldavido wrote: post #20
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

This is terrible advice for all but the largest of organizations. Running your own hardware is AWFUL. Get ready to dedicate an entire team to network engineering, fixing broken hard disks, patching operating systems, screwing around with RAID controllers, upgrading switches, planning power and cooling, and endless vendor negotiation -- with ISPs, hardware manufacturers, datacenter operators, etc. Oh, and did I mentio…

[deleted]

Re: Effectively using AWS Reserved Instances

#89

Post by kakwa_ »

not_kurt_godel wrote: post #77
hueving wrote: post #58

Earlier quoted context omitted.

>This is terrible advice for all but the largest of organizations. Don't start a conversation with an opening generalization like that if you want something constructive. Especially when the rest of your post is clearly based on the single anecdote of your experience. >Running your own hardware is AWFUL. Maybe for you. Not for any sysadmin with even just a couple of years of experience. >patching operating systems We…

> Don't start a conversation with an opening generalization like that if you want something constructive. Especially when the rest of your post is clearly based on the single anecdote of your experience. You are doing as much, if not more generalization by way of the assumptions you're making. > >Running your own hardware is AWFUL. > Maybe for you. Not for any sysadmin with even just a couple of years of experience.…

> How many organizations have the millions of dollars of capex needed to get that off the ground and keep it all running?

I think you are exaggerating when talking about "millions". Companies running in old school DC will not build the actual DC. They will rent a 42U cabinet or part of it.

You can typically rent 1/4 of a cabinet for a few hundred bucks a month.

I you operate at a small scale, you typically don't have that many servers, maybe 4 or 5. A decent server is in the range of 4000 to 6000$ and will last generally around 5 years.

And these servers rarely breaks (we have a fleet of more 300 servers, and maybe 1 or 2 "wake the on call" crashes a year).

Throw in 1 or 2 decent switches at about 1000 to 2000$ each and you are mostly good to go.

You end up with a capex of ~50 000$ to get you started, with a depreciation over 5 years.

Re: Effectively using AWS Reserved Instances

#90

Post by krn »

jedberg wrote: post #10
hueving wrote: post #6

The best answer is of course not to at all. Burst into the cloud, static workloads in your own DC.

How exactly does that solve the problem? Don't you have to do the same capacity planning to decide how many servers to buy for your datacenter? Except you get less flexibility because you can't buy servers and have them instantly available like you can for reserved instances? Also, what kind of workloads are you running that don't require databases? The biggest expense in any distributed system is moving data. If you…

What about dedicated servers available in 120s, without any setup fees and long-term commitments[1]?

Auto-monitoring included[2]. Unmetered bandwidth, too[3].

[1] https://www.ovh.com/world/dedicated-servers/infra/

[2] https://www.ovh.com/world/dedicated-servers/monitoring.xml

[3] https://www.ovh.com/world/dedicated-servers/bandwidth-upgrad...

Post reply on HN