Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon EC2 gets statistics, auto scaling and elastic load balancing

aws.typepad.com

Post reply on HN
Print view View original

31–40 of 44 posts

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#31

Post by neilc »

WALoeIII wrote: post #29
neilc wrote: post #27

AWS CloudWatch seems remarkably expensive: 15 cents per instance per hour! Considering that all they are doing is aggregating and distributing per-VM statistics. Woops: Brainfade. Yes, 1.5 cents/instance/hour, not 10x that. Still, quite expensive I think.

From the Amazon page its $0.015/hr: With Amazon CloudWatch... at a rate of $0.015 per hour for each Amazon EC2 instance you choose to monitor... As an example, a developer may want to monitor 10 Amazon EC2 instances 24×7 for a 30-day period. The Amazon CloudWatch cost would be $108 (or $0.015 per Amazon EC2 instance hour x 10 Amazon EC2 instances x 24 hours per day x 30 days)... That is close to what it costs to run…

Yeah, my bad. Still, I think it's quite expensive. If all they are offering is some simple runtime stats, then a 15% surcharge on the cost of a small instance is pretty expensive. If you have 50 instances, you're paying $540/month just for monitoring.

That is close to what it costs to run a single instance to collect all your statistics

Assuming you need a completely separate instance which couldn't do anything else but monitoring, which is unlikely to be the case.

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#32

Post by wmf »

neilc wrote: post #31
WALoeIII wrote: post #29

Earlier quoted context omitted.

From the Amazon page its $0.015/hr: With Amazon CloudWatch... at a rate of $0.015 per hour for each Amazon EC2 instance you choose to monitor... As an example, a developer may want to monitor 10 Amazon EC2 instances 24×7 for a 30-day period. The Amazon CloudWatch cost would be $108 (or $0.015 per Amazon EC2 instance hour x 10 Amazon EC2 instances x 24 hours per day x 30 days)... That is close to what it costs to run…

Yeah, my bad. Still, I think it's quite expensive. If all they are offering is some simple runtime stats, then a 15% surcharge on the cost of a small instance is pretty expensive. If you have 50 instances, you're paying $540/month just for monitoring . That is close to what it costs to run a single instance to collect all your statistics Assuming you need a completely separate instance which couldn't do anything else…

Think of it as a bailout for the third-party cloud monitoring startups; at least they have a chance to be cheaper than Amazon.

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#33

Post by jayliew »

mattjung wrote: post #11
rjurney wrote: post #2

How many startups does this put out of business? This keeps happening repeatedly - so let there be no more startups offering missing features of EC2. Go back! Build something else! Amazon has made it clear that you should not build a general cloud offering on top of EC2 - it must be niche, or not an enhancement of their offering, or they will simply do the same thing and kill you. And I don't think its that they're c…

I Agree. But what's so special about this? Any startup must think about the defendability of its idea. Completing very obvious gaps in the Amazon-offering is surely hardly defendable.

IMHO, their strategy might have been "if we build what's missing in AMZN's portfolio, we might get acquired". Granted, not the best strategy, but this strategy has worked for some in the past.

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#34

Post by ujeezy »

jf wrote: post #25
erlanger wrote: post #22

The load balancer seems nice. We used HAProxy, but it seemed like the sort of thing that Amazon could easily take care of on their own. I see that it has health checks, but I wonder if it's generally as nice and configurable as HAProxy.

Elastic Load Balancing isn't yet a replacement for HAProxy, for now it only seems to handle HTTP traffic, and not HTTPS: http://docs.amazonwebservices.com/ElasticLoadBalancing/lates...

I don't think HAProxy supports SSL either, does it? I had to put an nginx layer in front of HAProxy to get SSL working.

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#35

Post by tve »

luigi wrote: post #19
rjurney wrote: post #2

How many startups does this put out of business? This keeps happening repeatedly - so let there be no more startups offering missing features of EC2. Go back! Build something else! Amazon has made it clear that you should not build a general cloud offering on top of EC2 - it must be niche, or not an enhancement of their offering, or they will simply do the same thing and kill you. And I don't think its that they're c…

Check out the marketing language for RightScale: http://www.rightscale.com/ Their "Key Features": (1) Auto-Scaling, (2) Monitoring & Alerting, (3) Load Balancing. Ouch. But you're right -- the opportunity to provide unique value is in harnessing EC2 for specific purposes and applications.

Yeah, our marketing web site has gotten old, and it'll change within a couple of days. We haven't been selling auto-scaling and assorted features for a while. As Amazon is moving up the stack so are we. We're integrating a lot of the various features and really provide a platform for all the config management, automation, access control, etc. Plus pre-configured stacks that you can fully customize. If you actually try to use Amazon's monitoring and auto-scaling you'll find you have a lot of wheels to reinvent...

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#36

Post by erlanger »

ujeezy wrote: post #34
jf wrote: post #25

Earlier quoted context omitted.

Elastic Load Balancing isn't yet a replacement for HAProxy, for now it only seems to handle HTTP traffic, and not HTTPS: http://docs.amazonwebservices.com/ElasticLoadBalancing/lates...

I don't think HAProxy supports SSL either, does it? I had to put an nginx layer in front of HAProxy to get SSL working.

HAProxy certainly supports SSL.

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#37

Post by troystribling »

luigi wrote: post #19
rjurney wrote: post #2

How many startups does this put out of business? This keeps happening repeatedly - so let there be no more startups offering missing features of EC2. Go back! Build something else! Amazon has made it clear that you should not build a general cloud offering on top of EC2 - it must be niche, or not an enhancement of their offering, or they will simply do the same thing and kill you. And I don't think its that they're c…

Check out the marketing language for RightScale: http://www.rightscale.com/ Their "Key Features": (1) Auto-Scaling, (2) Monitoring & Alerting, (3) Load Balancing. Ouch. But you're right -- the opportunity to provide unique value is in harnessing EC2 for specific purposes and applications.

RightScale can provide services that span service providers. Currently, AWS is the principal cloud service provider but likely later this year there will be others (i.e Slice Host, FlexiScale, GoGrid, Sun). Services spanning service providers may include migrations, load balancing and redundancy.

It appears to me that RightScale is positioning itself to be a market for cloud computing services. It will be interesting to see if they pull it off.

Also, from the description of the monitoring service (i.e. no agent required) it is likely the monitoring data is collected at the host not the guest OS. So there is probably no process monitoring it follows that application profiling and similar services will not be possible with the data.

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#38

Post by nethergoat »

polvi wrote: post #30

If anyone is willing to share... Who here does this solve a specific problem for, and are you actively/planning on implementing it ASAP? What is your use case? It's a sweet feature, and I can't wait to see how it manifests in the hands of hackers.

We have several application components that use software load balancers (HAProxy) so we can manually scale each component quickly. The load balancing service will let us shut down each of those instances for a decent savings. Our QA environment is a copy of prod (scaled-back, but identical, including LBs) - more nodes to shut down, more money saved.

This is easy to implement and is basically found money, so we'll be doing it ASAP.

My expectation is that an Amazon (LB) access point is more reliable than a generic EC2 instance. For these components, we'll be paying less for something that's both more reliable (hopefully) and easier to manage.

Auto-scaling is powerful and easy to set up (even in the private beta, before tools were available), but as we're heavily integrated with RightScale (specifically, using templates throughout) migrating to pre-configured AMIs would take some effort. We'll have to do some internal review before making a decision on rolling out auto-scaling to existing apps.

Monitoring is, as expected, also provided by RightScale. Amazon requires you have CloudWatch turned on for instances in auto-scaled pools (for obvious reasons), so that's one way we may end up using it. RightScale's blog post mentioned they'll be integrating their alerting and reporting with CloudWatch, but I'm not sure it's worth the extra $0.015/intance-hour to have the data come from Amazon instead of RightScale. I expect that, for now, CloudWatch is more relevant to: 1) users not on RightScale, Cloudkick, etc. 2) users who want direct access to their monitoring data (for archival, custom graphs, whatever - not currently possible with RightScale, not sure about other cloud management vendors)

Amazon's move up the application stack (Elastic MapReduce, for starters) may create additional use cases for CloudWatch, but that's entirely up to Amazon.

What I'd really like to see in the near-term is an open source tool to read in and graph CloudWatch data that's been archived somewhere (SimpleDB would be cool, but S3 would work too). Of course, I can see why people would be hesitant to start down that path given that Amazon has a console that's begging to include this functionality.

-Mike B @ ShareThis

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#39

Post by ezmobius »

Here's my take on these features...

Monitoring and autoscaling I'll pass on. You can get better monitoring with collectd for free and autoscaling based on just load age or disk util and not app specific stuff is plain crazy.

The load balancer is pretty nice but I'll stick with haproxy until you can do httpand https on the same lb and also currently you can only do CNAME dns so no foo.com only www.foo.com and wildcard subdomains don't work unless you use tinydns(bind can't do it)

So I feel like this is a good start but I won't be using any of these services until they get more fully baked. The load balancer is the most interesting part but is currently seriously flawed.

Re: Amazon EC2 gets statistics, auto scaling and elastic load balancing

#40

Post by antirez »

While in my company we are loving S3 since it's really cheap and good, we can't say the same for EC2: the small instances are too slow, and the bigger instances are too costly and still can't compare with a decent Linux box. Overall we made our math and to have our servers in the server farm is going to cost less compared to EC2. Of course it's not going to be as simple to manage as EC2 is, but it is an acceptable compromise for now.

I think that here there is space for new startups if they are able to provide better performances at the same cost. I wonder if a configuration where in a cloud real hardware is selected in order to run an instance could work, instead to use virtualization technologies. This does not allow to take advantage of the fact that most boxes are not under high load most of the time, but maybe could work if it will be possible to use very low energy in this condition.

Post reply on HN