Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Lambda Cold Start Times

filia-aleks.medium.com

Post reply on HN
Print view View original

31–40 of 233 posts

Re: AWS Lambda Cold Start Times

#31

Post by ignoramous »

rajin444 wrote: post #2

Cool article, but shouldn’t Amazon be providing this kind of info? Surely they have this data internally.

afaik AWS doesn't publish benchmarks on runtimes; but if they did, I am sure it'd result in a lot of finger-pointing and wasted energy if they were not to normalize the process first (something like acidtests.org).

That said, they do publish plenty of guidance. For ex, see Chapter 9: Optimizing serverless application performance of their well-architected series: https://aws.amazon.com/ru/blogs/compute/building-well-archit...

Re: AWS Lambda Cold Start Times

#32

Post by cmcconomy »

My experience with cold starts in Azure Functions Serverless is pretty awful. Like most other Azure services, their affordable consumer grade offerings are designed from the ground up not to be good enough for "serious" use.

Cold start times compared to Lambda are worse, and in addition, we would get random 404s which do not appear in any logs; inspecting these 404s indicated they were emitted by nginx, leading me to believe that the ultimate container endpoint was killed for whatever reason but that fact didn't make it back to the router, which attempted and failed to reach the function.

Of course the cold start and 404 are mitigated if you pay for the premium serverless or just host their middleware on their own App Service plans (basically VMs)

Re: AWS Lambda Cold Start Times

#33

Post by tyingq »

berkes wrote: post #30
VWWHFSfQ wrote: post #15

Earlier quoted context omitted.

Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely…

> I'm sure it has some use-cases in some kind of backoffice task queue scenario, but Lambda is nearly unusable in a web context unless you have a very trivial amount of traffic. This has been the outcome for me on several projects too. Just use loadbalanced EC2 (or EB, for simplification) and pay for a few instances running 24/7. It's actually cheaper than having a busy lambda in all my cases. The only other case (ot…

I would guess this situation is maintained on purpose by AWS as the upsell reason for Fargate.

Re: AWS Lambda Cold Start Times

#34

Post by Unklejoe »

_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

That reminds me of a custom Linux device driver that I worked with in the past. It implemented "mmap" so that a user application could map a ring buffer into userspace for zero-copy transfers.

It used lazy mapping in the sense that it relied on the page fault handler to get triggered to map each page in as they were touched.

This resulted in a latency increase for the very first accesses, but then it was fast after that since the pages stayed mapped in.

The solution?

Read the entire ring buffer one time during startup to force all pages to get mapped in.

I eventually changed the driver to just map them all in at once.

Re: AWS Lambda Cold Start Times

#35

Post by jabart »

moonchrome wrote: post #13
jabart wrote: post #5

We run a few .net core lambdas and a few things that make a big difference for latency. 1. pre-jit the package, this reduces cold start times as the JIT doesn't need to run on most items. Still does later to optimize some items. 2 is sticking to the new .net json seralizer. The reference code uses both the new and old newtsonsoft package. The old package has higher memory allocations as it doesn't make use of the Spa…

Great tip for running the JIT AOT, for anyone interested Microsoft calls this "ReadyToRun" compilation [1]. [1] https://docs.microsoft.com/en-us/dotnet/core/deploying/ready... I wonder did you test if the increased size results in an actual win for the startup time ?

After like 256mb it is less of an impact when using readytorun. Some of the lambdas I have are webhooks so latency isn't as important and when it's user facing 512mb seems to be a sweet spot.

Re: AWS Lambda Cold Start Times

#36

Post by jSherz »

psanford wrote: post #16

Something I discovered recently, for my tiny Go Lambda functions it is basically always worth it to run them at least with 256mb of memory even if they don't need more than 128mb. This is because most of my functions run twice as fast at 256mb than they do at 128mb. Since lambda pricing is memory_limit times execution time, you get better performance for free. Test your lambda functions in different configurations to…

CPU allocated to Lambda functions scales with the memory allowance [1] and you can use power tuning [2] to find the optimal values.

[1] https://docs.aws.amazon.com/lambda/latest/dg/configuration-f...

[2] https://serverlessrepo.aws.amazon.com/applications/arn:aws:s...

Re: AWS Lambda Cold Start Times

#37

Post by x86_64Ubuntu »

jpdb wrote: post #8
_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

> To combat this we wrote a "runWarm" function that kept the API alive at all times. This doesn't really work like you'd expect and isn't recommended, as it only helps a particular use-case. The reason is that AWS Lambda will only keep a single instance of your function alive. That means if two requests come in at the same time, you'd see a cold start on one of those invocations. Instead, you want to look at somethin…

Here is a little AWS doc describing what parent is talking about. Personally, I had confused "provisioned concurrency" with "concurrency limit" since I don't work with cloud stuff outside of hobbying.

https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...

Re: AWS Lambda Cold Start Times

#38

Post by traspler »

muh_gradle wrote: post #14

Surprised to see such mediocre performance from Node. It was an engineering decision on our team to develop one of our Lambdas with Node and we were deciding between Python and Node. Looks like Go and Rust look very promising.

I was also quite surprised, other benchmarks I‘ve seen before have shown it to be just slightly behind Python.

Re: AWS Lambda Cold Start Times

#39

Post by nodejs_rulez_1 »

_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

> ...Sure kind of defeats the purpose of serverless but hey, enterprise software.

No, no, no. It's "hey, Amazon/Microsoft cloud engineering". They should be amazing with whiteboard interview exercises though.

Re: AWS Lambda Cold Start Times

#40

Post by nodejs_rulez_1 »

muh_gradle wrote: post #14

Surprised to see such mediocre performance from Node. It was an engineering decision on our team to develop one of our Lambdas with Node and we were deciding between Python and Node. Looks like Go and Rust look very promising.

I am not surprised that a re-purposed browser engine sucks for back-end.
Post reply on HN