An incredible amount of software and infrastructure is written precisely for analytics data gathering workloads. I'm pretty confident AWS's product for this use case would be Lambda and the new on-demand DynamoDB. Is there actually a use case in analytics that requires a server that accepts connections from multiple clients, and then has to have Because if there isn't, if your goal is to scale, why have containers at…
Batching incoming requests, for one. Kinesis only allows 5 write requests per second per shard, for example. As well, Lambda have limits regarding concurrent executions and are very slow (10s) if needing VPC connectivity (in this case the default concurrent lambda limit is 350 due to ENIs)
When AWS Autoscale Doesn’t
21–30 of 99 posts
Re: When AWS Autoscale Doesn’t
#22AWS needs to glue EC2 and ECS scheduling together. Today the schedulers are separate. So basically the feet does not know what the arms are doing. That leaves fixing this scaling up to the client meaning duplicate code effort solving the same thing for each AWS customer.
I have been in Devops since it before it had a name and I see many companies solving the same problems.
Like ths auto-scaling post. That's not the first company to deal with it (nor the last). Providing a set of tools can be very beneficial but so hard to dial down.
I have a very big itch around solving this problem.
Re: When AWS Autoscale Doesn’t
#23Re: When AWS Autoscale Doesn’t
#24I'm surprised I didn't see application performance monitoring mentioned here. A lot of applications are complex and in those cases adding containers is only effective until you reach the next constraint. Having two resources (such as DB and app) scale in concert can be exceedingly difficult.
This means your resources are too tightly coupled. If they are so tightly coupled that they need to scale together then they are not two resources and you should look into restructuring them into two actual resources or bind them more closely to make a single resource.
Re: When AWS Autoscale Doesn’t
#25The way I've been happiest using EC2 Auto Scaling was to have a single cron-job continuously calculating how many instances I should be running, and it sets the desired capacity manually with the Auto Scaling API[1]. This may seem to defeat the purpose of Auto Scaling, but it's actually much more convenient than spinning up/down EC2 instances with the EC2 API. You get to precisely control how to scale, and won't be a…
Re: When AWS Autoscale Doesn’t
#26I'm surprised I didn't see application performance monitoring mentioned here. A lot of applications are complex and in those cases adding containers is only effective until you reach the next constraint. Having two resources (such as DB and app) scale in concert can be exceedingly difficult.
> Having two resources (such as DB and app) scale in concert can be exceedingly difficult. This means your resources are too tightly coupled. If they are so tightly coupled that they need to scale together then they are not two resources and you should look into restructuring them into two actual resources or bind them more closely to make a single resource.
Re: When AWS Autoscale Doesn’t
#27The biggest scaling issue I always run into is the database is the bottleneck and there’s not a lot of options for most databases to auto scale them.
Read capacity is easy to skill to infinity with caches. But if a DB can only write 1000 updates per second, nothing will change that.
In many cases - it's ok to not process EVERYTHING right away. Process the important stuff RIGHT AWAY. Slowly process the unimportant stuff in your spare time.
Re: When AWS Autoscale Doesn’t
#28The way I've been happiest using EC2 Auto Scaling was to have a single cron-job continuously calculating how many instances I should be running, and it sets the desired capacity manually with the Auto Scaling API[1]. This may seem to defeat the purpose of Auto Scaling, but it's actually much more convenient than spinning up/down EC2 instances with the EC2 API. You get to precisely control how to scale, and won't be a…
Re: When AWS Autoscale Doesn’t
#29There are many limitations that you need to "read between the lines" with AWS auto scaling. For example, we have daemons reading messages from SQS, if you try to use auto scaling based on SQS metrics, you come to realize pretty quickly that CloudWatch is updated every 5 minutes. For most messages, this is simply too late. In a lot of cases, you are better off with updating CloudWatch yourself with your own interval u…
For example, imagine that each machine has 20 available threads for processing messages received from SQS. Then I'd track a metric which is the percent of threads that are in use. If I'm trying to meet a message processing SLA, then my goal is to begin auto-scaling before that in-use percentage reaches 100%, e.g., we might scale up when the average thread utilization breaches 80%. (Or if you process messages with unlimited concurrent threads, then you could use CPU utilization instead.)
The benefit of this approach is that you can begin auto-scaling your system before it saturates and messages start to be delayed. Messages will only be delayed once the in-use percent reaches 100% -- as long as there are threads available (i.e., in-use If you were to auto-scale on SQS metrics like queue length, then the length will stay approximately zero until the system starts falling behind, and then it's too late. If you scale on queue size then you can't preemptively scale when load is increasing. By monitoring and scaling on thread capacity, you can track your effective utilization as it climbs from 50% to 80% to 100%; and you can begin scaling before it reaches 100%, before messages start to back up.
The other benefit of this approach is that it works equally well at many different scales; a threshold like 80% thread utilization works just as well with a single host, as with a fleet of 100 hosts. By comparison, thresholds on metrics like queue length need to be adjusted as the scale and throughput of the system changes.
Re: When AWS Autoscale Doesn’t
#30Vertical autoscale is on my wish list. Some way to automatically scale instance size for those things that don't scale well horizontally.
I didn't even realize this wasn't part of the offering yet.