Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autoscaling on AWS

qzaidi.github.io

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Autoscaling on AWS

#11

Post by pestaa »

contingencies wrote: post #10

Not to detract from this article explicitly on the subject of AWS, but ... if you are relying on any single cloud provider then you are probably not going to scale beyond a certain point, since it means that reliability (ie. heterogeneity - re: cloud providers, physical/logical points of presence, legal jurisdictions, etc.) is not all that important to you. At a certain point, a cloud provider neutral approach is cal…

I'd like to hear more about this. When buying or renting physical machines is worthwhile, how would I actually go about it? Should I get them preassemblied, should I buy parts and then glue it together, or people just generally rent their infrastructure?

Re: Autoscaling on AWS

#12

Post by mathrawka »

I have found that using a ubuntu base AMI and salt (http://saltstack.com/community.html) makes things much easier to manage. His suggestion of having node.js and nginx baked into the AMI is a big pain when you need to upgrade node.js or nginx (hello security update!). Using something like salt (or puppet or chef) would make this much easier.

But in general, I agree with the article.

Re: Autoscaling on AWS

#13

Post by x0x0 »

contingencies wrote: post #10

Not to detract from this article explicitly on the subject of AWS, but ... if you are relying on any single cloud provider then you are probably not going to scale beyond a certain point, since it means that reliability (ie. heterogeneity - re: cloud providers, physical/logical points of presence, legal jurisdictions, etc.) is not all that important to you. At a certain point, a cloud provider neutral approach is cal…

at a certain point, real hardware is called for, if you wish to scale. my experience: 1/3 the price, 2x the performance in softlayer vs a $97k/mo ec2 spend. Obviously specific to my workload, but you should at least benchmark on real hardware.

Re: Autoscaling on AWS

#14

Post by mqzaidi »

mathrawka wrote: post #12

I have found that using a ubuntu base AMI and salt ( http://saltstack.com/community.html ) makes things much easier to manage. His suggestion of having node.js and nginx baked into the AMI is a big pain when you need to upgrade node.js or nginx (hello security update!). Using something like salt (or puppet or chef) would make this much easier. But in general, I agree with the article.

We enable auto applying security updates in the AMI, and the first thing the provisioning script does is update packages. That said, I would definitely want to explore alternatives - not needing to update while provisioning will take a few seconds off the provisioning time.

Re: Autoscaling on AWS

#15

Post by jahewson »

pestaa wrote: post #11
contingencies wrote: post #10

Not to detract from this article explicitly on the subject of AWS, but ... if you are relying on any single cloud provider then you are probably not going to scale beyond a certain point, since it means that reliability (ie. heterogeneity - re: cloud providers, physical/logical points of presence, legal jurisdictions, etc.) is not all that important to you. At a certain point, a cloud provider neutral approach is cal…

I'd like to hear more about this. When buying or renting physical machines is worthwhile, how would I actually go about it? Should I get them preassemblied, should I buy parts and then glue it together, or people just generally rent their infrastructure?

If your workload is totally standard then renting dedicated servers can make sense, it's certainly flexible if you expect to scale on a monthly basis. However, if you need something "premium" like an SSD or lots of RAM then you can end up paying $300/month. Buying a used server off eBay for $400, putting a new HDD in it, and getting a colocation for $100/month will be the most cost-efficient for those on a tight budget. The provider will usually do basic maintainance like replacing HDDs for a small fee.

That said, if money is not a problem then go with whatever lets you get stuff done with the least pain.

Re: Autoscaling on AWS

#16

Post by morganK »

contingencies wrote: post #10

Not to detract from this article explicitly on the subject of AWS, but ... if you are relying on any single cloud provider then you are probably not going to scale beyond a certain point, since it means that reliability (ie. heterogeneity - re: cloud providers, physical/logical points of presence, legal jurisdictions, etc.) is not all that important to you. At a certain point, a cloud provider neutral approach is cal…

Not that simple. Having only one cloud provider allow you to make the most of it.

Once you have two providers, you are forced to only use the features offered by each of them and use an abstract layer (API agnostic dev).

Plus, AWS offers a wide range of strategies to ensure the availability of its infrastructure (Availability Zone, Regions, CDN etc.)

Re: Autoscaling on AWS

#17

Post by contingencies »

morganK wrote: post #16
contingencies wrote: post #10

Not to detract from this article explicitly on the subject of AWS, but ... if you are relying on any single cloud provider then you are probably not going to scale beyond a certain point, since it means that reliability (ie. heterogeneity - re: cloud providers, physical/logical points of presence, legal jurisdictions, etc.) is not all that important to you. At a certain point, a cloud provider neutral approach is cal…

Not that simple. Having only one cloud provider allow you to make the most of it. Once you have two providers, you are forced to only use the features offered by each of them and use an abstract layer (API agnostic dev). Plus, AWS offers a wide range of strategies to ensure the availability of its infrastructure (Availability Zone, Regions, CDN etc.)

Once you have two providers, you are forced to only use the features offered by each of them and use an abstract layer (API agnostic dev).

Abstraction layers don't necessarily need to produce lowest common denominator results.

Plus, AWS offers a wide range of strategies to ensure the availability of its infrastructure (Availability Zone, Regions, CDN etc.)

You still wind up vulnerable to quirks of the single provider though! For example, account freeze for whatever reason (financial quirks, legal issues, regulatory change) any multi-site failures (eg. financial, operational, legal, security) at that provider.

Re: Autoscaling on AWS

#18

Post by robotmay »

shimms wrote: post #3
thejosh wrote: post #2

What CSS/Framework/whatever are you using for that metro-style dashboard?

Looks like Dashing from Shopify: http://shopify.github.io/dashing/

Dashing is a great thing to play around with in the evening. I've got a bunch of custom data sources I should really open source for other people, like Heroku dyno counts etc.

Re: Autoscaling on AWS

#19

Post by contingencies »

x0x0 wrote: post #13
contingencies wrote: post #10

Not to detract from this article explicitly on the subject of AWS, but ... if you are relying on any single cloud provider then you are probably not going to scale beyond a certain point, since it means that reliability (ie. heterogeneity - re: cloud providers, physical/logical points of presence, legal jurisdictions, etc.) is not all that important to you. At a certain point, a cloud provider neutral approach is cal…

at a certain point, real hardware is called for, if you wish to scale. my experience: 1/3 the price, 2x the performance in softlayer vs a $97k/mo ec2 spend. Obviously specific to my workload, but you should at least benchmark on real hardware.

Agreed.

However, cloud does have benefits that you lose with physical, eg. speed of initial deployment, speed of scale-out (no ordering/assembly/wiring/burn-in test/etc. lead times), maintenance and data center proximity issues (often translating to increased costs), masking of some of the legal/financial issues of multi-site presence (eg. if you want multi-jurisdictional points of presence), etc.

There are of course ways to use mixed strategies to combine cloud/third party services/CDN with physical infrastructure.

As always, "right tool for the job". I just felt it was worth mentioning there are limitations with single-provider cloud infrastructures as it's possible to get excited about the benefits of a particular approach without considering the full picture.

Post reply on HN