Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Lambda Cold Start Times

filia-aleks.medium.com

Post reply on HN
Print view View original

91–100 of 233 posts

Re: AWS Lambda Cold Start Times

#91

Post by lostcolony »

_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

"To combat this"

Did you actually need to?

That's one of the things that always threw me with complaints about cold starts - how many apps/etc do I use daily where I interact, and there's a 10 second delay before something happens? The answer: quite a lot.

Yeah, we can do better. And in fact, with Serverless, -most users will experience better-. It's only when load is increasing that you see those delays, and then it's still only a delay. Not shed load.

The fact I can experience that delay easily in dev makes people think it's going to be a huge problem, but, A. In real use it probably isn't as common as it is in dev (since you have way more traffic) B. You can design to minimize it (different API endpoints can hit the same lambda and be routed to the right handler there, making it more likely to be hot), C. It forces you to test and plan for worst case from the beginning (rather than at the end where you've built something, and now have to load test it).

Not to say to use it all the time, of course; there are plenty of scenarios where the cost, the delay, etc, are non-starters. But there are also plenty of scenarios where an engineer's instinctual reaction would be "too slow", but in reality it's fine; your p95 is going to look great, and only your P99 is going to look bad (on that note, a slow API response accompanied with a spinner is a very different thing from a UX perspective than a slow page load with no indication of progress), and even then it's predictable when it happens, and it forces function scale out rather than tanking a service. Of course, it's often not obvious upfront which scenarios those would be until/unless you try it, and that's definitely a barrier.

Re: AWS Lambda Cold Start Times

#92

Post by ctvo »

nuerow wrote: post #85
ctvo wrote: post #58

Earlier quoted context omitted.

This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. You can graph it against ECS / EC2 to see the point it stops becoming economical. Taking all of this into account, Lambda is then useful for a very small niche: - Tasks that don't care about low P99 latency. These tend to be asynchronous processing workflows, as APIs in the customer re…

> This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. I feel this is a gross misrepresentation of AWS Lambdas. AWS lambdas are primarily tailored for background processes, event handlers, and infrequent invocations. This is how they are sold, including in AWS' serverless tutorials. Even though they can scale like crazy, and even though…

> I feel this is a gross misrepresentation of AWS Lambdas.

AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code as a ZIP file or container image, and Lambda automatically and precisely allocates compute execution power and runs your code based on the incoming request or event, for any scale of traffic. You can set up your code to automatically trigger from over 200 AWS services and SaaS applications or call it directly from any web or mobile app. You can write Lambda functions in your favorite language (Node.js, Python, Go, Java, and more) and use both serverless and container tools, such as AWS SAM or Docker CLI, to build, test, and deploy your functions.

https://aws.amazon.com/lambda/

Edit:

> it's widely known that if your API handles more more traffic than a few requests per second then you're better off putting together your own service.

How is it widely known? Is it on their documentation clearly or in their marketing materials to use another AWS product?

That's what's I mean by over marketing here. Requiring insider baseball knowledge because using it as described footguns your company at infection points isn't a great customer experience.

Re: AWS Lambda Cold Start Times

#93

Post by adflux »

VWWHFSfQ wrote: post #89
adflux wrote: post #69

Earlier quoted context omitted.

The way Azure Function scales out is different and is not entirely suited for the same goal as lambdas. Lambdas happily scale from 1 to 1000 instances in seconds* (EDIT: not A second), whereas Azure Functions just wont do that.

> Lambdas happily scale from 1 to 1000 instances in a second What on earth. No they don't...

https://docs.aws.amazon.com/lambda/latest/dg/invocation-scal...

Last time I tried this was a few years ago, but seems like its still the case.

For an initial burst of traffic, your functions' cumulative concurrency in a Region can reach an initial level of between 500 and 3000, which varies per Region. After the initial burst, your functions' concurrency can scale by an additional 500 instances each minute. This continues until there are enough instances to serve all requests, or until a concurrency limit is reached. When requests come in faster than your function can scale, or when your function is at maximum concurrency, additional requests fail with a throttling error (429 status code).

Re: AWS Lambda Cold Start Times

#94

Post by bmm6o »

mikesabbagh wrote: post #87
_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

> To combat this we wrote a "runWarm" function that kept the API alive at all times. maybe one should create 1 huge lambda instead of several hundred

Cold starts aren't just going from 0 to 1, you hit it anytime you scale. Putting everything in a single lambda likely means more scaling up.

Re: AWS Lambda Cold Start Times

#95

Post by berkes »

roland35 wrote: post #72
berkes wrote: post #30

Earlier quoted context omitted.

> I'm sure it has some use-cases in some kind of backoffice task queue scenario, but Lambda is nearly unusable in a web context unless you have a very trivial amount of traffic. This has been the outcome for me on several projects too. Just use loadbalanced EC2 (or EB, for simplification) and pay for a few instances running 24/7. It's actually cheaper than having a busy lambda in all my cases. The only other case (ot…

Have you run into issues with Lambda with complex tasks? I thought there was a 15 minute limit to tasks, plus a maximum storage size when importing large dependencies, etc?

The latex example did not run entirely on Lambda. Lambda would write a job into a queue (just Postgres), trigger a launch of a beefy ec2 instance, after which a worker on that ec2 picked up the job. Another lambda function would be called by the server itself to shut down the worker when all jobs were done.

Kludgy and slow. But it worked and did save some money, because the instance running this latex worker was big and chunky yet utilized maybe 10 hours a month.

Lambda was mostly acting as a kldugy load-balancer really.

Re: AWS Lambda Cold Start Times

#96

Post by tenaciousDaniel »

earthboundkid wrote: post #75
danielvaughn wrote: post #71

Earlier quoted context omitted.

Same experience with Firebase. I just joined a team that has been using it. I've never worked with serverless before, and it boggles my mind how anyone thought it would be a good idea. The cold starts are horrendous. In one case, it's consistently taking about 7 seconds to return ~10K of data. I investigated the actual runtime of the function and it completes in about 20ms, so the only real bottleneck is the fucking…

7s cold start is weird. I think there has to be more to the story than that.

That's what I thought. We've spent weeks investigating this one function and tried everything that Firebase recommends in their docs. Nothing has worked.

Re: AWS Lambda Cold Start Times

#97

Post by tenaciousDaniel »

alex_duf wrote: post #83
earthboundkid wrote: post #75

Earlier quoted context omitted.

7s cold start is weird. I think there has to be more to the story than that.

I've seen 30s on AWS, so it's not that surprising. They have now improved it greatly though. And yet I still believe it's a great technology, as always it's a matter of putting it on the right use case. Message consumption from a queue or topic, low traffic and low criticality API are two great use cases.

Yeah I could see serverless being useful for those scenarios.

Re: AWS Lambda Cold Start Times

#98

Post by VWWHFSfQ »

adflux wrote: post #93
VWWHFSfQ wrote: post #89

Earlier quoted context omitted.

> Lambdas happily scale from 1 to 1000 instances in a second What on earth. No they don't...

https://docs.aws.amazon.com/lambda/latest/dg/invocation-scal... Last time I tried this was a few years ago, but seems like its still the case. For an initial burst of traffic, your functions' cumulative concurrency in a Region can reach an initial level of between 500 and 3000, which varies per Region. After the initial burst, your functions' concurrency can scale by an additional 500 instances each minute. This cont…

That's just burst concurrency. It takes a lot longer that 1 second to actually reach the peak burstable limit. So while the Lambdas are scaling up your clients are either waiting several seconds, or getting 429s.

Re: AWS Lambda Cold Start Times

#99

Post by raro11 »

jpdb wrote: post #8
_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

> To combat this we wrote a "runWarm" function that kept the API alive at all times. This doesn't really work like you'd expect and isn't recommended, as it only helps a particular use-case. The reason is that AWS Lambda will only keep a single instance of your function alive. That means if two requests come in at the same time, you'd see a cold start on one of those invocations. Instead, you want to look at somethin…

Have your runWarm sleep for 500ms and execute 50 of them concurrently. As long as none of the functions are finished and you start a new one you get a new instance, at least that's what I think.

You can get 50 hot instances that way no?

I'd rather scale per connections. Have a lambda instance do 50 concurrent requests. Something like https://fly.io but cheaper.

Re: AWS Lambda Cold Start Times

#100

Post by ewjt »

moonchrome wrote: post #9

One thing I wish he included was .NET 5 - since that switched to using Docker images I would be very interested in the differences.

Lambda only has default support for the LTS releases of .NET, which is 2.1, 3.1 and the upcoming (November 2021) .NET 6.

The only way to run .NET 5 in a Lambda that I know of would be custom runtimes or containers. Is that what do you mean by “switched to using Docker images”?

This article describes the containerized performance of .NET in Lambda and the cold starts are dramatically (~4x) worse.

https://www.kloia.com/blog/aws-lambda-container-image-.net-b...

Post reply on HN