AWS Introducing Provisioned Concurrency for Lambda Functions • Hacker News Board

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#51

Post by scarface74 » Wed, Dec 04, 2019, 5:22 PM UTC

scottndecker wrote: post #41

AWS 2006: "Run your workloads on our EC2 instances in the cloud 24/7." AWS 2014: "Run your work loads on serverless so you don't have to deal with those pesky EC2 instances 24/7 anymore." AWS 2019: "Click a checkbox and you can have your serverless workloads get dedicated EC2 instances 24/7!"

You wouldn’t use provisioned capacity for “workloads” where you don’t care about latency - like processing events.

It would only be used for user impacting APIs.

There are a few types of processes that I have had to create.

1. A Windows service that processed a queue. We have 20x more messages at peak. Of course since it was tied to Windows, lambda wasn’t an option. I had to create an autoscaling group based on queue length. That also involves CloudWatch alarms to trigger scaling and now we either have one instance running all the time (production) or we have a min of zero and only launch an instance when there is a message in the queue (non prod). Not only is the process slower to scale, but because it’s Windows AWS does hourly billing.

Of course the deployment process and Cloudformation template are a lot more complicated than lambda.

2. Same sort of process on lambda. The CloudFormation template using SAM is much simpler and the process is faster to scale in and out.

Also, you can configure everything on the web and export the template.

3. A Node/Express API using lambda proxy integration behind API Gateway.

Again this was easy to set up but cold start times were killing us and we knew that we were going to have to move it off of lambda because of the 6MB request/response limit.

4. The same API as above running in Fargate.

Since we knew advance that this was the direction we wanted to go, I opted to use Node/Express for the lambda. So we didn’t require any code changes. But creating a registry, Docker containers, services, clusters, load balancers, autoscaling groups, etc took a lot longer to get right and then automating everything with CloudFormation was more complicated.

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#52

Post by munns » Wed, Dec 04, 2019, 5:27 PM UTC

scarface74 wrote: post #50

munns wrote: post #48

Hey all, I lead developer advocacy for serverless at AWS and was part of this product launch since we started thinking about it(quite some time ago I should say). I'm running around re:Invent this week, but will try and pop in and answer any questions I can. Provisioned Concurrency (PC) is an interesting feature for us as we've gotten so much feedback over the years about the pain point of the service over head leadi…

The cold start from using lambda has a number of causes 1. the time to initialize the VM 2. the time to create an ENI if you are connecting to a VPC[1](until the NAT alternative rolls out globally) 3. the time to initialize your language runtime (Java seems to be the worse, scripting languages the best) 4. any program initialization done outside of your handler that runs once per cold start of your lambda runtime. A…

Fwiw new networking for VPC is completely rolled out for all public regions now. (#2) (and thank you, its called "Attached to a VPC" and not in :) )

This covers you straight through 4.

Now it's possible that your execution environment could be sitting for sometime waiting for any action and so pre-handler DB connections and things like that might need to be tweaked in this model.

Thanks, - munns

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#53

Post by munns » Wed, Dec 04, 2019, 5:43 PM UTC

munns wrote: post #48

Hey all, I lead developer advocacy for serverless at AWS and was part of this product launch since we started thinking about it(quite some time ago I should say). I'm running around re:Invent this week, but will try and pop in and answer any questions I can. Provisioned Concurrency (PC) is an interesting feature for us as we've gotten so much feedback over the years about the pain point of the service over head leadi…

We've just published a second post on this launch: https://aws.amazon.com/blogs/compute/new-for-aws-lambda-pred...

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#54

Post by munns » Wed, Dec 04, 2019, 5:47 PM UTC

reilly3000 wrote: post #23

With Fargate Savings Plans and Spot Instances, the cost of running workloads on Fargate is getting substantially cheaper, and with the exception of extremely bursty workloads, much more consistently performant vs Lambda. The cost of provisioning Lambda capacity as well as paying for the compute time on that capacity means Fargate is even more appealing for high volume workloads. The new pricing page for lambda ("Exam…

Dollar to dollar comparisons are one way to compare these two technologies but it leaves a lot not covered. The application programming model varies greatly (socket/port vs. event). There's also a lot more that Lambda brings to the table in terms of monitoring, logging, etc that you'd need to do work yourself to enable.

Fargate is a great product, but it doesn't completely remove all operational work to the degree that Lambda does.

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#55

Post by scarface74 » Wed, Dec 04, 2019, 5:55 PM UTC

munns wrote: post #52

scarface74 wrote: post #50

Earlier quoted context omitted.

The cold start from using lambda has a number of causes 1. the time to initialize the VM 2. the time to create an ENI if you are connecting to a VPC[1](until the NAT alternative rolls out globally) 3. the time to initialize your language runtime (Java seems to be the worse, scripting languages the best) 4. any program initialization done outside of your handler that runs once per cold start of your lambda runtime. A…

Fwiw new networking for VPC is completely rolled out for all public regions now. (#2) (and thank you, its called "Attached to a VPC" and not in :) ) This covers you straight through 4. Now it's possible that your execution environment could be sitting for sometime waiting for any action and so pre-handler DB connections and things like that might need to be tweaked in this model. Thanks, - munns

So I had to convert three lambda APIs using proxy integration to Fargate mostly because of the 6MB request/response limit but the cold starts caused us to make a rule that we weren’t going to convert our EC2 hosted APIs to lambda. We were going to host them on Fargate.

But since the APIs that I moved over to Fargate are now automatically being deployed to both lambda and Fargate with separate URLs, we can A/B test both and see if we will move to lambda in cases where the request/response limit isn’t a problem.

Btw, I didn’t think using a NAT instead of an ENI had rolled out completely. I tried to delete a stack recently and it still took awhile to “cleanup” resources. I thought that was caused when it was deleting the ENI. I’ll be on the look out for it next time I need to delete a stack.

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#56

Post by inopinatus » Wed, Dec 04, 2019, 7:44 PM UTC

munns wrote: post #48

Hey all, I lead developer advocacy for serverless at AWS and was part of this product launch since we started thinking about it(quite some time ago I should say). I'm running around re:Invent this week, but will try and pop in and answer any questions I can. Provisioned Concurrency (PC) is an interesting feature for us as we've gotten so much feedback over the years about the pain point of the service over head leadi…

I'm wondering how feasible it'd be to vary PC through the day.

The pricing examples include using PC on a limited duty cycle, and billing is defined to start from the moment it's enabled (rather than from when it's ready), so it'd be reasonable to expect there's some level of certainty that the concurrency level is ready within a defined timeframe. What might that timeframe be, and to what level of certainty?

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#57

Post by WNWceAJ9R9Ezc4 » Wed, Dec 04, 2019, 7:56 PM UTC

inopinatus wrote: post #56

munns wrote: post #48

Hey all, I lead developer advocacy for serverless at AWS and was part of this product launch since we started thinking about it(quite some time ago I should say). I'm running around re:Invent this week, but will try and pop in and answer any questions I can. Provisioned Concurrency (PC) is an interesting feature for us as we've gotten so much feedback over the years about the pain point of the service over head leadi…

I'm wondering how feasible it'd be to vary PC through the day. The pricing examples include using PC on a limited duty cycle, and billing is defined to start from the moment it's enabled (rather than from when it's ready), so it'd be reasonable to expect there's some level of certainty that the concurrency level is ready within a defined timeframe. What might that timeframe be, and to what level of certainty?

PC supports auto scaling.

https://docs.aws.amazon.com/autoscaling/application/userguid...

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#58

Post by [deleted] » Wed, Dec 04, 2019, 7:57 PM UTC

scarface74 wrote: post #50

munns wrote: post #48

Hey all, I lead developer advocacy for serverless at AWS and was part of this product launch since we started thinking about it(quite some time ago I should say). I'm running around re:Invent this week, but will try and pop in and answer any questions I can. Provisioned Concurrency (PC) is an interesting feature for us as we've gotten so much feedback over the years about the pain point of the service over head leadi…

The cold start from using lambda has a number of causes 1. the time to initialize the VM 2. the time to create an ENI if you are connecting to a VPC[1](until the NAT alternative rolls out globally) 3. the time to initialize your language runtime (Java seems to be the worse, scripting languages the best) 4. any program initialization done outside of your handler that runs once per cold start of your lambda runtime. A…

[deleted]

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#59

Post by inopinatus » Wed, Dec 04, 2019, 7:58 PM UTC

dillondoyle wrote: post #11

etaioinshrdlu wrote: post #8

This feels like a step backwards to me, nevermind how necessary it may be. The magic was paying only for what you use on super bursty workloads. Now this is like throwing your hands up and saying the users bursts are too big for AWS.

Honest question not being sarcastic: if this cold start latency is so important why chose function over elastic beanstalk or other auto-scaling type system? Answer could help us try something new. We currently use large google app engine 'apps' after failing to get functions to scale quick enough (and hit limits). we have SUPER bursty traffic that needs to scale up to hundreds of instances very fast.

Cold start of an instance is still slower and larger grained.

Re: AWS Introducing Provisioned Concurrency for Lambda Functions

#60

Post by reilly3000 » Wed, Dec 04, 2019, 8:08 PM UTC

munns wrote: post #54

reilly3000 wrote: post #23

With Fargate Savings Plans and Spot Instances, the cost of running workloads on Fargate is getting substantially cheaper, and with the exception of extremely bursty workloads, much more consistently performant vs Lambda. The cost of provisioning Lambda capacity as well as paying for the compute time on that capacity means Fargate is even more appealing for high volume workloads. The new pricing page for lambda ("Exam…

Dollar to dollar comparisons are one way to compare these two technologies but it leaves a lot not covered. The application programming model varies greatly (socket/port vs. event). There's also a lot more that Lambda brings to the table in terms of monitoring, logging, etc that you'd need to do work yourself to enable. Fargate is a great product, but it doesn't completely remove all operational work to the degree th…

Agreed. For the vast majority of cases, a Lambda function is easier to ship and maintain, and most likely dramatically cheaper. I really only think the value of using Fargate kicks in compared to Lambda at around 5M+ invocations/month. YMMV based on workflow and workload.