Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Lambda Cold Start Times

filia-aleks.medium.com

Post reply on HN
Print view View original

81–90 of 233 posts

Re: AWS Lambda Cold Start Times

#81

Post by earthboundkid »

mhaymo wrote: post #68
ctvo wrote: post #58

Earlier quoted context omitted.

This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. You can graph it against ECS / EC2 to see the point it stops becoming economical. Taking all of this into account, Lambda is then useful for a very small niche: - Tasks that don't care about low P99 latency. These tend to be asynchronous processing workflows, as APIs in the customer re…

In my experience cold starts don't affect the p99 if you have substantial traffic, because you have enough lambdas consistently running that cold start rate is ~0.1%. P99.9 also matters though!

If you have substantial traffic, the cost savings of Lambda are gone, and you can just use ECS or something.

Re: AWS Lambda Cold Start Times

#82

Post by VWWHFSfQ »

wmfiv wrote: post #76
VWWHFSfQ wrote: post #15

Earlier quoted context omitted.

Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely…

Insanely expensive is definitely a flexible term. I think numbers help here. Provisions Concurrency $8.64 / GB / month 256 MB per Lambda (Assuming Python, Ruby, NodeJS, or Rust) $2.16 per Lambda per month A lot of organizations can probably make a good business case for keeping 100s or even 1000s of Lambda's warm. You also don't need to keep them warm 24x7, can get an additional 12% discount using savings plans, and…

I'm sure there are plenty of companies that are happy to throw $5,000/mo at a problem that can be solved better for $250/mo. Not mine though.

Re: AWS Lambda Cold Start Times

#83

Post by alex_duf »

earthboundkid wrote: post #75
danielvaughn wrote: post #71

Earlier quoted context omitted.

Same experience with Firebase. I just joined a team that has been using it. I've never worked with serverless before, and it boggles my mind how anyone thought it would be a good idea. The cold starts are horrendous. In one case, it's consistently taking about 7 seconds to return ~10K of data. I investigated the actual runtime of the function and it completes in about 20ms, so the only real bottleneck is the fucking…

7s cold start is weird. I think there has to be more to the story than that.

I've seen 30s on AWS, so it's not that surprising. They have now improved it greatly though.

And yet I still believe it's a great technology, as always it's a matter of putting it on the right use case. Message consumption from a queue or topic, low traffic and low criticality API are two great use cases.

Re: AWS Lambda Cold Start Times

#84

Post by ctvo »

_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

What you and others are doing is attempting to predict your peak traffic when you take this approach. It may work for some companies, but more commonly in my experience, it hides P99+ tail latency from companies that may not instrument deeply (and they think the problem is solved).

The rate at which you execute `runWarm` is the peak traffic you're expecting. A request comes in over that threshold and you'll still experience cold start latency.

Provisioned concurrency doesn't change this, but it does move the complexity of `runWarm` to the Lambda team and gives you more control (give me a pool of 50 warmed Lambdas vs. me trying too run `runWarm` enough to keep 50 warmed myself). That's valuable in a lot of use cases. At the end of the day you're still in the game of predicting peak traffic and paying (a lot) for it.

We're almost always trying to predict peak traffic though! The difference is using a course grain computing platform, like EC2 for example, where a single box can handle hundreds++ requests per second, gives you more room for error, and is cheaper.

There are a lot of other trade-offs to consider. My biggest issue is this isn't enumerated clearly by AWS, and I run into way too many people who have footgun themselves unnecessarily with Lambda.

Re: AWS Lambda Cold Start Times

#85

Post by nuerow »

ctvo wrote: post #58
VWWHFSfQ wrote: post #15

Earlier quoted context omitted.

Provisioned concurrency is insanely expensive. If you have any kind of a thundering herd access pattern then Lambda is a complete non-starter because of the warm-up and scaling characteristics. We eventually just put an nginx/openresty server on a regular medium EC2 instance and got rid of Lambda from our stack completely and now we're paying about 1/300th the cost we were previously and the performance is infinitely…

This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second. You can graph it against ECS / EC2 to see the point it stops becoming economical. Taking all of this into account, Lambda is then useful for a very small niche: - Tasks that don't care about low P99 latency. These tend to be asynchronous processing workflows, as APIs in the customer re…

> This is another example of AWS over marketing Lambda. Lambda is horrendously expensive when requests pass a certain level per second.

I feel this is a gross misrepresentation of AWS Lambdas.

AWS lambdas are primarily tailored for background processes, event handlers, and infrequent invocations. This is how they are sold, including in AWS' serverless tutorials.

Even though they can scale like crazy, and even though you can put together an API with API Gateway or even Application Load Balancer, it's widely known that if your API handles more more traffic than a few requests per second then you're better off putting together your own service.

The rationale is that if you don't need to do much with a handler, or you don't expect to handle a lot of traffic on a small number of endpoints, AWS lambdas offer a cheaper solution to develop and operate. In some cases (most happy path cases?), they are actually free to use. Beyond a certain threshold, you're better off getting your own service to run on EC2/Fargate/ECS/whatever, specially given that once you have a service up and running then adding a controller is trivial.

Re: AWS Lambda Cold Start Times

#87

Post by mikesabbagh »

_fat_santa wrote: post #4

At my last job we built an entire API on top of serverless. One of the things we had to figure out was this cold start time. If a user were to hit an endpoint for the first time, it would take 2x as long as it normally would at first. To combat this we wrote a "runWarm" function that kept the API alive at all times. Sure kind of defeats the purpose of serverless but hey, enterprise software.

> To combat this we wrote a "runWarm" function that kept the API alive at all times.

maybe one should create 1 huge lambda instead of several hundred

Re: AWS Lambda Cold Start Times

#88

Post by FpUser »

Shadonototra wrote: post #52

that is why the language for cloud native is GO, and languages like Java/C# are dead tech stack, they failed to reinvent themselves to stay relevant

>"they failed to reinvent themselves to stay relevant"

This sounds like it came straight out of "The Corporate BS generator" - https://www.atrixnet.com/bs-generator.html.

Re: AWS Lambda Cold Start Times

#89

Post by VWWHFSfQ »

adflux wrote: post #69
cmcconomy wrote: post #32

My experience with cold starts in Azure Functions Serverless is pretty awful. Like most other Azure services, their affordable consumer grade offerings are designed from the ground up not to be good enough for "serious" use. Cold start times compared to Lambda are worse, and in addition, we would get random 404s which do not appear in any logs; inspecting these 404s indicated they were emitted by nginx, leading me to…

The way Azure Function scales out is different and is not entirely suited for the same goal as lambdas. Lambdas happily scale from 1 to 1000 instances in seconds* (EDIT: not A second), whereas Azure Functions just wont do that.

> Lambdas happily scale from 1 to 1000 instances in a second

What on earth. No they don't...

Re: AWS Lambda Cold Start Times

#90

Post by capableweb »

jmnicolas wrote: post #66

I just exploded in laughter when I read Java OOM at 128MB. I can't explain to my coworkers, they wouldn't understand.

Agree, it's always funny to see people using runtimes they are not used to using, so they don't know how to configure it properly, like assigning how much memory it can use. Wouldn't trust these results too much as a consequence.
Post reply on HN