AWS Lambda Cold Start Times • Hacker News Board

Re: AWS Lambda Cold Start Times

#51

Post by k__ » Tue, Oct 12, 2021, 1:53 PM UTC

_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

If you build an API and are concerned about cold-starts you could look at Lambda@Edge and "CloudFront Functions".

I could imagine these to perform better.

If you aren't married to AWS, then Cloudflare Workers could also be worth a shot.

Re: AWS Lambda Cold Start Times

#52

Post by Shadonototra » Tue, Oct 12, 2021, 1:54 PM UTC

that is why the language for cloud native is GO, and languages like Java/C# are dead tech stack, they failed to reinvent themselves to stay relevant

Re: AWS Lambda Cold Start Times

#53

Post by kidsil » Tue, Oct 12, 2021, 1:54 PM UTC

I find it surprising that NodeJS is showing the worst cold start times. Most likely over 90% of AWS Lambda functions are written NodeJS.

Re: AWS Lambda Cold Start Times

#54

Post by Shadonototra » Tue, Oct 12, 2021, 1:57 PM UTC

moonchrome wrote: post #13

jabart wrote: post #5

We run a few .net core lambdas and a few things that make a big difference for latency. 1. pre-jit the package, this reduces cold start times as the JIT doesn't need to run on most items. Still does later to optimize some items. 2 is sticking to the new .net json seralizer. The reference code uses both the new and old newtsonsoft package. The old package has higher memory allocations as it doesn't make use of the Spa…

Great tip for running the JIT AOT, for anyone interested Microsoft calls this "ReadyToRun" compilation [1]. [1] https://docs.microsoft.com/en-us/dotnet/core/deploying/ready... I wonder did you test if the increased size results in an actual win for the startup time ?

and then you need to upgrade the plan because it is disk/memory hungry and doesn't fit within the 128mb plan, so you'll pay 4x more already

just use GO and be faster while saving money at the same time

Re: AWS Lambda Cold Start Times

#55

Post by d0gsg0w00f » Tue, Oct 12, 2021, 1:58 PM UTC

wongarsu wrote: post #21

rajin444 wrote: post #2

Cool article, but shouldn’t Amazon be providing this kind of info? Surely they have this data internally.

It makes Lambda look like a product with a much narrower niche than what AWS wants to sell it as. For many people knowing beforehand that cold start times are > 500ms with 256MB (quite extravagant for serving a single web request) would disqualify Lambda for any customer-serving endpoint. As it stands many get tricked into that choice if they don't perform these tests themselves.

In my experience, if you can't say "get over it" to your customers when they complain about performance then Lambda is not the right tool. Just use EC2.

Re: AWS Lambda Cold Start Times

#56

Post by projectileboy » Tue, Oct 12, 2021, 2:00 PM UTC

AWS Lambda is pretty cool, it just gets used a lot for applications that it was never really designed for. While I wish that Amazon would address the cold start times, if you try to grill your burgers with a cordless drill, you can’t really blame the drill manufacturer when the meat doesn’t cook.

Re: AWS Lambda Cold Start Times

#57

Post by kkielhofner » Tue, Oct 12, 2021, 2:00 PM UTC

haolez wrote: post #49

Slightly off topic, but what's the deal with Azure Functions cold start times in the Consumption (i.e. serverless) plan? I get cold start times in the multi seconds range (sometimes huge values, like 20s). Am I doing something wrong? Or is this expected?

I've experienced this as well. I gave up on optimizing it.

I get around it by using a load balancer (Cloudflare currently) that does periodic health checks. Keeps it alive and the charges are minimal (still well within the free tier).

Speaking of Cloudflare it's on my "nice to-do" list to move this to Workers as a primary anyway.

I also use a completely separate uptime monitor and alerting platform (Uptime Robot) so one way or another I'd be keeping at least one instance warm no matter what.

Re: AWS Lambda Cold Start Times

#58

Post by ctvo » Tue, Oct 12, 2021, 2:00 PM UTC

VWWHFSfQ wrote: post #15

jpdb wrote: post #8

Earlier quoted context omitted.

> To combat this we wrote a "runWarm" function that kept the API alive at all times. This doesn't really work like you'd expect and isn't recommended, as it only helps a particular use-case. The reason is that AWS Lambda will only keep a single instance of your function alive. That means if two requests come in at the same time, you'd see a cold start on one of those invocations. Instead, you want to look at somethin…

Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely…

This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. You can graph it against ECS / EC2 to see the point it stops becoming economical.

Taking all of this into account, Lambda is then useful for a very small niche:

- Tasks that don't care about low P99 latency. These tend to be asynchronous processing workflows, as APIs in the customer request path tend to care about low P99 latency.

- Tasks that have a low request per second. Again, these tend to be asynchronous processing workflows.

You talk to anyone on the AWS serverless team and the conversation eventually focuses on toil. If you can quantify engineering toil for your organization, and give it a number, the point at which Lambda stops being economical shifts right, but it doesn't change the overall shape of the graph.

Re: AWS Lambda Cold Start Times

#59

Post by cmcconomy » Tue, Oct 12, 2021, 2:05 PM UTC

secondaryacct wrote: post #46

cmcconomy wrote: post #32

My experience with cold starts in Azure Functions Serverless is pretty awful. Like most other Azure services, their affordable consumer grade offerings are designed from the ground up not to be good enough for "serious" use. Cold start times compared to Lambda are worse, and in addition, we would get random 404s which do not appear in any logs; inspecting these 404s indicated they were emitted by nginx, leading me to…

My company is moving away from datacenter and into Azure and I have to get the az900 this week, it doesnt bode well. And I was so happy to leave the clusterfuck of 300 aws lambda I was working with in my prev company. What an expensive fad, and no engineer is ever consulted ...

There are things I like, such as the consistent and pervasive security model with Azure AD. I don't even mind running stuff on App Services.

I just find their commodity serverless offering in particular is subpar

Re: AWS Lambda Cold Start Times

#60

Post by rcarmo » Tue, Oct 12, 2021, 2:05 PM UTC

I recently discovered that uWSGI has a "cheap mode" that will hold the socket open but only actually spawn workers when a connection comes in (and kill them automatically after a timeout without any requests).

Pertinent options: https://github.com/piku/piku/blob/master/piku.py#L908

If you already have 24/7 compute instances going and can spare the CPU/RAM headroom, you can co-host your "lambdas" there, and make them even cheaper :)