Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When AWS Autoscale Doesn’t

segment.com

Post reply on HN
Print view View original

41–50 of 99 posts

Re: When AWS Autoscale Doesn’t

#41

Post by NathanKP »

eunoia wrote: post #11
avitzurel wrote: post #2

There are many limitations that you need to "read between the lines" with AWS auto scaling. For example, we have daemons reading messages from SQS, if you try to use auto scaling based on SQS metrics, you come to realize pretty quickly that CloudWatch is updated every 5 minutes. For most messages, this is simply too late. In a lot of cases, you are better off with updating CloudWatch yourself with your own interval u…

Out of curiosity, what’s the use case for running ECS on EC2 (instead of using Fargate) these days?

AWS employee here. If you are able to achieve consistent greater than 50% utilization of your EC2 instances or have a high percentage of spot or reserved instances then ECS on EC2 is still cheaper than Fargate. If your workload is very large, requiring many instances this may make the economics of ECS on EC2 more attractive than using Fargate. (Almost never the case for small workloads though).

Additionally, a major use case for ECS is machine learning workloads powered by GPU's and Fargate does not yet have this support. With ECS you can run p2 or p3 instances and orchestrate machine learning containers across them with even GPU reservation and GPU pinning.

Re: When AWS Autoscale Doesn’t

#42

Post by cdoxsey »

DVassallo wrote: post #16

The way I've been happiest using EC2 Auto Scaling was to have a single cron-job continuously calculating how many instances I should be running, and it sets the desired capacity manually with the Auto Scaling API[1]. This may seem to defeat the purpose of Auto Scaling, but it's actually much more convenient than spinning up/down EC2 instances with the EC2 API. You get to precisely control how to scale, and won't be a…

This is how the cluster-autoscaler works in kubernetes. It sets the desired capacity based on the number of pods needing to be scheduled.

Coupled with a horizontal pod autoscaler (which sets the replica count based on a metric) you get the best of both worlds.

Re: When AWS Autoscale Doesn’t

#43

Post by avip »

eunoia wrote: post #11
avitzurel wrote: post #2

There are many limitations that you need to "read between the lines" with AWS auto scaling. For example, we have daemons reading messages from SQS, if you try to use auto scaling based on SQS metrics, you come to realize pretty quickly that CloudWatch is updated every 5 minutes. For most messages, this is simply too late. In a lot of cases, you are better off with updating CloudWatch yourself with your own interval u…

Out of curiosity, what’s the use case for running ECS on EC2 (instead of using Fargate) these days?

Cost and legacy.

Re: When AWS Autoscale Doesn’t

#44

Post by NathanKP »

Legogris wrote: post #7

Having used ECS quite a bit, I do not recommend anyone building a new stack based on it. Kubernetes solves everything ECS solves, but usually better and without sveral of the issues mentioned here. Last time I checked, AWS was still lagging behind Azure and GCP on Kubernetes, but I have a strong feeling they're prioritizing improving EKS over ECS. If you're already invested in ECS it's a different story, of course.

AWS employee here. I don't think there is anything fundamental that makes Kubernetes avoid autoscaling issues. Just like with ECS if you don't setup the right horizontal pod autoscaling settings in Kubernetes you can easily end up with under or over scheduling of your Kubernetes pods. Ultimately no matter whether you use ECS or EKS you will need to do some finetuning and testing to make sure your autoscaling setup matches with your real world traffic patterns.

AWS is committed to improving both ECS and EKS. You can see our public roadmap with many in progress improvements for both container orchestration platforms here: https://github.com/aws/containers-roadmap/projects/1

Feel free to add your own additional suggestions, or vote on existing items to help us prioritize what we should work on faster!

Re: When AWS Autoscale Doesn’t

#45

Post by jedberg »

Auto-scaling is depending on startup time. If your startup time for a new instance/container is 5 seconds, then you need to predict what your traffic will be in 5 seconds. If your startup time is 10 minutes, then you need to predict your traffic in 10 minutes.

The choice of metric is important, but it needs to be a metric that predicts future traffic if you want to autoscale user facing services. CPU load is not that metric.

The best way to do autoscaling is to build a system that is unique to your business to predict your traffic, and then use AWS's autoscaling as your backup for when you get your prediction wrong.

Re: When AWS Autoscale Doesn’t

#46

Post by analogj »

xfitm3 wrote: post #26
eikenberry wrote: post #24

Earlier quoted context omitted.

> Having two resources (such as DB and app) scale in concert can be exceedingly difficult. This means your resources are too tightly coupled. If they are so tightly coupled that they need to scale together then they are not two resources and you should look into restructuring them into two actual resources or bind them more closely to make a single resource.

In my example of DB and backend: How could they be decoupled?

read only replicas, sharding, clustering, caching

Re: When AWS Autoscale Doesn’t

#47

Post by jedberg »

tyingq wrote: post #5

Vertical autoscale is on my wish list. Some way to automatically scale instance size for those things that don't scale well horizontally.

You could hack around this by creating two auto-scaling groups with different instance types and then have them follow the same metric such that the small group goes to 0 and the larger one spins up. Not a great solution but better than nothing.

Re: When AWS Autoscale Doesn’t

#48

Post by chrissnell »

NathanKP wrote: post #41
eunoia wrote: post #11

Earlier quoted context omitted.

Out of curiosity, what’s the use case for running ECS on EC2 (instead of using Fargate) these days?

AWS employee here. If you are able to achieve consistent greater than 50% utilization of your EC2 instances or have a high percentage of spot or reserved instances then ECS on EC2 is still cheaper than Fargate. If your workload is very large, requiring many instances this may make the economics of ECS on EC2 more attractive than using Fargate. (Almost never the case for small workloads though). Additionally, a major…

I'm not totally up to speed on ECS vs EKS economics but it seems like EKS with p2/p3 would be a sweet solution for this. Even better if you have a mixed workload and you want to easily target GPU-enabled instances by adding a taint to the podspec.

Re: When AWS Autoscale Doesn’t

#49

Post by NathanKP »

chrissnell wrote: post #48
NathanKP wrote: post #41

Earlier quoted context omitted.

AWS employee here. If you are able to achieve consistent greater than 50% utilization of your EC2 instances or have a high percentage of spot or reserved instances then ECS on EC2 is still cheaper than Fargate. If your workload is very large, requiring many instances this may make the economics of ECS on EC2 more attractive than using Fargate. (Almost never the case for small workloads though). Additionally, a major…

I'm not totally up to speed on ECS vs EKS economics but it seems like EKS with p2/p3 would be a sweet solution for this. Even better if you have a mixed workload and you want to easily target GPU-enabled instances by adding a taint to the podspec.

Kubernetes GPU scheduling is currently still marked as experimental: https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus...

ECS GPU scheduling is production ready, and streamlined quite a bit on the initial getting started workflow due to the fact that we provide a maintained GPU optimized AMI for ECS that already has your NVIDIA kernel drivers and Docker GPU runtime. ECS supports GPU pinning for maximum performance, as well as mixed CPU and GPU workloads in the same cluster: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...

Re: When AWS Autoscale Doesn’t

#50

Post by chrissnell »

cdoxsey wrote: post #42
DVassallo wrote: post #16

The way I've been happiest using EC2 Auto Scaling was to have a single cron-job continuously calculating how many instances I should be running, and it sets the desired capacity manually with the Auto Scaling API[1]. This may seem to defeat the purpose of Auto Scaling, but it's actually much more convenient than spinning up/down EC2 instances with the EC2 API. You get to precisely control how to scale, and won't be a…

This is how the cluster-autoscaler works in kubernetes. It sets the desired capacity based on the number of pods needing to be scheduled. Coupled with a horizontal pod autoscaler (which sets the replica count based on a metric) you get the best of both worlds.

We do this for all of our Kube clusters here. We have a nifty use of this in our CI cluster, where Job resources are scheduled by a service that monitors our build queue. As builds are initiated by git pushes, single-use Jobs (pods) are created and when there are not enough free capacity to scale them, the build cluster scales up. On the weekends when everyone is gone, it scales back to near-nothing. This is a huuuuge money-saver for us because we use beefy c5 instances for our builds. It also saves lots of time because devs are no longer waiting 45+ minutes for their build to start.
Post reply on HN