AWS Lambda Cold Start Times • Hacker News Board

Re: AWS Lambda Cold Start Times

#171

Post by gunnarmorling » Tue, Oct 12, 2021, 6:48 PM UTC

websap wrote: post #157

gunnarmorling wrote: post #117

The best cold starts are those which aren't noticed by the user. For my blog search (which runs on Lambda), I found a nice way of achieving that [1]: as soon as a user puts the focus to the input field for the search text, this will already submit a "ping" request to Lambda. Then, when they submit the actual query itself, they will hit the already running Lambda most of the times. And, as others said, assigning more…

Have you tried provision concurrency[1], this along with ARM based functions can help you with performance and reduce costs. 1 - https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...

PC doesn't make that much sense for a use case like mine, where absolute cost minimization is the goal. My request volume (including ping requests) easily fits into the Lambda free tier, whereas with PC I'd be looking at some cost.

More generally, I feel PC kinda defeats the purpose of Lambda to begin with. If I'm in that range of continuous load, I'd rather look at more traditional means of provisioning and hosting this application.

Re: AWS Lambda Cold Start Times

#172

Post by gunnarmorling » Tue, Oct 12, 2021, 6:52 PM UTC

yoava wrote: post #166

gunnarmorling wrote: post #117

The best cold starts are those which aren't noticed by the user. For my blog search (which runs on Lambda), I found a nice way of achieving that [1]: as soon as a user puts the focus to the input field for the search text, this will already submit a "ping" request to Lambda. Then, when they submit the actual query itself, they will hit the already running Lambda most of the times. And, as others said, assigning more…

Just use http functions in a Velo by Wix project - you will get 50ms cold start out of the box.

Thanks, will take a look. Can I run Java, or any kind of binary, there? That's why I'm not running this with CloudFlare workers, for instance. An alternative I'm actually considering is fly.io.

Re: AWS Lambda Cold Start Times

#173

Post by jacurtis » Tue, Oct 12, 2021, 6:53 PM UTC

joncrane wrote: post #138

jacurtis wrote: post #122

Earlier quoted context omitted.

There is actually a really awesome middle-ground that AWS offers that no one seems to talk about. That is using ECS + Fargate. This gives you (IMHO) the best of both worlds between Lambda and EC2. ECS is Elastic Container Service. Think docker/podman containers. You can even pull from Dockerhub or ECR (Elastic Container Registry - amazon's version of dockerhub). ECS can then deploy to either a traditional EC2 compute…

>It has automatic scaling, so it can scale up and down with traffic (all of which is configured in ECS) Doesn't scaling take time, though? Doesn't downloading a new docker container definition and starting it take at least as long as initializing a new lambda function? Also with lambda there's no configuring to do for scaling. If anything lambda gives you tools to limit the concurrency.

Thanks for pointing that out. I should have clarified because I agree that "Automatic" is a relative term.

Lambda is entirely automatic like you point out. You literally don't need to think about it. You upload your function and it scales to meet demand (within limits).

ECS however still requires configuration, but it is extremely simple to do. They actually call it "Service Auto-Scaling". Within there you choose a scaling strategy and set a few parameters. That is it. After that, it really is "automatic".

Most of the time you will be selecting the "Target Tracking" strategy. Then you select a Cloudwatch metric and it will deploy and terminate Fargate instances (called "tasks" in the docs) to stay within your specified range. So a good example would be selecting a CPUUsage metric and keeping the average CPUUsage between 40-70%. If the average CPU usage starts to get above 70% across your tasks (Fargate instances), then ECS will deploy more automatically. If it falls below 40% then it will terminate them until you get within your desired range. You get all this magic from a simple configuration in ECS. So that's what I mean by automatic. Its pretty easy. Depending on what you are doing, it can set scaling to any other metric. It could be bandwidth, users, memory usage, etc. Some of these (like memory) require you to configure a custom metric, but again it isn't bad.

You can also scale according to other strategies like scheduled. So if get lots of traffic during business hours you can scale up during business hours and scale down during the night. Again, just set your schedule in ECS. It is pretty simple.

Re: AWS Lambda Cold Start Times

#174

Post by rimutaka » Tue, Oct 12, 2021, 6:55 PM UTC

The articles states that it takes approximately 1000 requests to optimize / warm up. I suspect that they had concurrency set at the default 999, so the first 999 requests would spin up new instances.

Does that mean their 15,000 requests were actually 15 requests spread over 1000 instances?

Re: AWS Lambda Cold Start Times

#175

Post by kilburn » Tue, Oct 12, 2021, 6:55 PM UTC

jacurtis wrote: post #122

lostcolony wrote: post #91

Earlier quoted context omitted.

"To combat this" Did you actually need to? That's one of the things that always threw me with complaints about cold starts - how many apps/etc do I use daily where I interact, and there's a 10 second delay before something happens? The answer: quite a lot. Yeah, we can do better. And in fact, with Serverless, -most users will experience better-. It's only when load is increasing that you see those delays, and then it…

There is actually a really awesome middle-ground that AWS offers that no one seems to talk about. That is using ECS + Fargate. This gives you (IMHO) the best of both worlds between Lambda and EC2. ECS is Elastic Container Service. Think docker/podman containers. You can even pull from Dockerhub or ECR (Elastic Container Registry - amazon's version of dockerhub). ECS can then deploy to either a traditional EC2 compute…

The big issue with ECS+Fargate is the lack of CPU bursting capability. This means that if you want to run a small service that doesn't consume much, you have two options:

1. Use a 0.25cpu + 0.5gb ram configuration and accept that your responses are now 4 times slower because the 25% time is strictly enforced.

2. Use a 1cpu + 2gb ram (costing 4 times more) even though it is very under-utilized.

AWS is definitely in no rush to fix this, as they keep saying they are aware of the issue and "thinking about it". No commitment or solution on sight though:

https://github.com/aws/containers-roadmap/issues/163

Re: AWS Lambda Cold Start Times

#176

Post by idunno246 » Tue, Oct 12, 2021, 6:56 PM UTC

wmfiv wrote: post #76

VWWHFSfQ wrote: post #15

Earlier quoted context omitted.

Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely…

Insanely expensive is definitely a flexible term. I think numbers help here. Provisions Concurrency $8.64 / GB / month 256 MB per Lambda (Assuming Python, Ruby, NodeJS, or Rust) $2.16 per Lambda per month A lot of organizations can probably make a good business case for keeping 100s or even 1000s of Lambda's warm. You also don't need to keep them warm 24x7, can get an additional 12% discount using savings plans, and…

it ends up being cheaper overall if you have high utilization of your provisioning since the per second fee while a function is running is cheaper. using https://calculator.aws/#/createCalculator/Lambda, if you have a steady 1 request/s and each requests takes 1 second, 2592000 seconds in a month. at 1024mb, i get 36.52 for provisioned and 43.72 for on demand. With autoscaling...you wont get 100% utilization, but it probably ends up being close enough to a wash

Re: AWS Lambda Cold Start Times

#177

Post by wertgklrgh » Tue, Oct 12, 2021, 6:56 PM UTC

bezossucks wrote: post #165

If you can run your entire function in V8 and don't need node, Cloudflare Workers is MUCH faster, more affordable, more manageable, and more reliable. They get cloned all over the world and you're not region-locked. https://www.cloudflare.com/learning/serverless/serverless-pe...

yet nobody ever uses them

Re: AWS Lambda Cold Start Times

#178

Post by StanAngeloff » Tue, Oct 12, 2021, 6:56 PM UTC

bezossucks wrote: post #165

If you can run your entire function in V8 and don't need node, Cloudflare Workers is MUCH faster, more affordable, more manageable, and more reliable. They get cloned all over the world and you're not region-locked. https://www.cloudflare.com/learning/serverless/serverless-pe...

We've moved from AWS to Cloudflare and I can confirm Workers are awesome. The only downside is lots of work required to get npm packages running properly. Due to the lack of Node.js' runtime, bundling, shims and stubbing required to get stuff like "fs" and "net" compiling. AWS is more mature, has a better community support around it and generally the tooling is more stable. Workers are quickly catching up, though, with great new additions such as ES modules, custom builds, Miniflare, etc.

Re: AWS Lambda Cold Start Times

#179

Post by moonchrome » Tue, Oct 12, 2021, 7:15 PM UTC

ewjt wrote: post #100

moonchrome wrote: post #9

One thing I wish he included was .NET 5 - since that switched to using Docker images I would be very interested in the differences.

Lambda only has default support for the LTS releases of .NET, which is 2.1, 3.1 and the upcoming (November 2021) .NET 6. The only way to run .NET 5 in a Lambda that I know of would be custom runtimes or containers. Is that what do you mean by “switched to using Docker images”? This article describes the containerized performance of .NET in Lambda and the cold starts are dramatically (~4x) worse. https://www.kloia.com…

>Lambda only has default support for the LTS releases of .NET

Not true, https://aws.amazon.com/blogs/developer/net-5-aws-lambda-supp...

>Even though Lambda’s policy has always been to support LTS versions of language runtimes for managed runtimes, the new container image support makes .NET 5 a first class platform for Lambda functions.

Re: AWS Lambda Cold Start Times

#180

Post by kaishiro » Tue, Oct 12, 2021, 7:21 PM UTC

danielvaughn wrote: post #71

cmcconomy wrote: post #32

My experience with cold starts in Azure Functions Serverless is pretty awful. Like most other Azure services, their affordable consumer grade offerings are designed from the ground up not to be good enough for "serious" use. Cold start times compared to Lambda are worse, and in addition, we would get random 404s which do not appear in any logs; inspecting these 404s indicated they were emitted by nginx, leading me to…

Same experience with Firebase. I just joined a team that has been using it. I've never worked with serverless before, and it boggles my mind how anyone thought it would be a good idea. The cold starts are horrendous. In one case, it's consistently taking about 7 seconds to return ~10K of data. I investigated the actual runtime of the function and it completes in about 20ms, so the only real bottleneck is the fucking…

I'm assuming by Firebase you mean Firebase Functions? We have a fairly complex infrastructure running off a combination of Firebase's RTDB, Firestore, and Google Cloud Functions and have never seen anywhere near what you're describing. Are you sure you're experiencing a 7 second "cold start", or is the invocation simply taking 7 seconds to run? Because the latter is far more easily explained.