Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Lambda Cold Start Times

filia-aleks.medium.com

Post reply on HN
Print view View original

191–200 of 233 posts

Re: AWS Lambda Cold Start Times

#191

Post by tsimionescu »

nuerow wrote: post #101
alex_duf wrote: post #83

Earlier quoted context omitted.

I've seen 30s on AWS, so it's not that surprising. They have now improved it greatly though. And yet I still believe it's a great technology, as always it's a matter of putting it on the right use case. Message consumption from a queue or topic, low traffic and low criticality API are two great use cases.

> I've seen 30s on AWS No, you've seen 30s on a random implementation running on AWS. If you write your lambdas without knowing what you're doing then you can't blame the technology for being grossly misused by you. Case in point: developing AWS Lambdas for the JDK runtime. The bulk of the startup time is not the lambda at all but the way the lambda code is initialized. This means clients and auth and stuff. I've wor…

That sort of time could be easily reached with lambdas that require VPC access, where a new ENI needs to be provisioned for each lambda container. I don't think alive seen 30s, but could easily see 5-10s for this case. And since this is required to run an isolated DB that is not some other AWS service, it isn't that uncommon. I believe they have since improved start times in this scenario significantly more recently.

Re: AWS Lambda Cold Start Times

#192

Post by 015a »

websap wrote: post #184
015a wrote: post #170

Earlier quoted context omitted.

Provisioned concurrency is a bit of a non-starter for cold start latency reduction in user-facing applications. Its a tool in the toolbox, but the problem is: Let's say you set the provisioned concurrency at 10, then you have 11 concurrent requests come in. That 11th request will still get a cold start. PC doesn't scale automatically. The ping architecture of warming up functions does scale (better) in this setup. It…

At least in the OP’s case it seems the requests are infrequent. I understand your point about the 11th request, but at that point what % of your customers are impacted. The tail latency is always an issue, PC is a good way to solve that.

Yeah, I'd say it just depends on your needs.

PC is definitely "better". It does require you to opt-in to function versioning, but if your ops automation is good enough to make that easy, its just better. The biggest advantage over warming pings is, really, that it keeps your metrics and logs clean and focused just on user invocations, not on weird faux-invokes that are extremely difficult to filter out.

But, its downside is that its ~100x more expensive than warming pings for infrequently invoked functions (defined as: average concurrent invocations over a month timeframe between 0 and 1).

If your company has the money, go for PC. But don't feel bad about going for warming pings; it ain't a bad setup at all.

Re: AWS Lambda Cold Start Times

#193

Post by gjhr »

zxcvbn4038 wrote: post #190
gunnarmorling wrote: post #171

Earlier quoted context omitted.

PC doesn't make that much sense for a use case like mine, where absolute cost minimization is the goal. My request volume (including ping requests) easily fits into the Lambda free tier, whereas with PC I'd be looking at some cost. More generally, I feel PC kinda defeats the purpose of Lambda to begin with. If I'm in that range of continuous load, I'd rather look at more traditional means of provisioning and hosting…

I think you are thinking of reserved concurrency - with reserved concurrency you can only run as many instances as you have reserved. With provisioned concurrency you can run as many as your quota allows but you are guaranteed to be able to handle as many concurrent instances have provisioned. Neither one of these are going to help you with cold start times, it is not defining the number of instances running, its jus…

Provisioned concurreny does decrease cold starts.

Reserved concurrency both guarantees a portion of your concurrency limit be allocated to a lambda as well as capping concurrency of that lambda to that portion. Reserved concurrency has no cost associated.

Provisioned concurrency keeps a certain number of execution environments warm for your use. Provisioned concurrency costs money.

Re: AWS Lambda Cold Start Times

#194

Post by gbourne »

tenaciousDaniel wrote: post #121
ceras wrote: post #114

Earlier quoted context omitted.

Not a fit for every scenario, but you can set a minimum number of instances for Firebase to keep ready. Should help reduce cold starts: https://firebase.google.com/docs/functions/tips#min

Yeah I tried that, it doesn't work. Ultimately the provisioning algorithm is a black box.

Why didn't the Firebase minInstances work for you? I found amazing performance benefits, but at a $ cost. I actually forgot[1] a function to set a minInstance and a user complained that this particular functionality was slow (compare to the rest of the site). However, it isn't cheap.

You also want to be sure your code is optimized[2]. For example, don't require module packages unless the function needs it, else you're loading unnecessary packages. I usually set a global variable `let fetch;` and in the function that requires fetch initialize it `fetch = fetch ?? require("node-fetch);`.

[1] https://www.ayrshare.com/a-firebase-cloud-functions-cold-sta... [2] https://firebase.google.com/docs/functions/tips

Re: AWS Lambda Cold Start Times

#195

Post by [deleted] »

muh_gradle wrote: post #14

Surprised to see such mediocre performance from Node. It was an engineering decision on our team to develop one of our Lambdas with Node and we were deciding between Python and Node. Looks like Go and Rust look very promising.

[deleted]

Re: AWS Lambda Cold Start Times

#196

Post by itisit »

gjhr wrote: post #193
zxcvbn4038 wrote: post #190

Earlier quoted context omitted.

I think you are thinking of reserved concurrency - with reserved concurrency you can only run as many instances as you have reserved. With provisioned concurrency you can run as many as your quota allows but you are guaranteed to be able to handle as many concurrent instances have provisioned. Neither one of these are going to help you with cold start times, it is not defining the number of instances running, its jus…

Provisioned concurreny does decrease cold starts. Reserved concurrency both guarantees a portion of your concurrency limit be allocated to a lambda as well as capping concurrency of that lambda to that portion. Reserved concurrency has no cost associated. Provisioned concurrency keeps a certain number of execution environments warm for your use. Provisioned concurrency costs money.

> Provisioned concurreny does decrease cold starts.

Indeed it does.

https://aws.amazon.com/blogs/aws/new-provisioned-concurrency...

https://docs.aws.amazon.com/lambda/latest/dg/provisioned-con...

Re: AWS Lambda Cold Start Times

#197

Post by paxys »

davewritescode wrote: post #125

The main downside of Lambda, in particular for user facing applications is that the incentives of the cloud provider and you are completely opposed. You (the developer) want a bunch of warm lambdas ready to serve user requests and the cloud provider is looking to minimize costs by keeping the number of running lambdas as low as possible. It's the incentive model that fundamentally makes Lambda a poor choice for these…

Disagree with "completely opposed". Cloud providers want to make money, sure, but in general everyone in the ecosystem benefits if every CPU cycle is used efficiently. Any overhead goes out of both AWS's and your pockets and instead to the electricity provider, server manufacturer, cooling service.

Re: AWS Lambda Cold Start Times

#198

Post by paxys »

losvedir wrote: post #43

I'm surprised Node has cold-start issues. I had it in my mind that JS was Lambda's "native" language and wouldn't have cold start issues at all. Did it used to be like that? Didn't Lambda launch with only support for JS, and maybe a couple other languages that could compile to it?

Cold start has been a problem with Lambda since day 1, and in fact has massively improved in recent years.

Node.js is optimized for request throughput rather than startup time. The assumption is that you will have a "hot" server running indefinitely. The Lambda pattern is in general a very recent invention, and not something that languages/rutimes have specifically considered in their design yet.

Re: AWS Lambda Cold Start Times

#199

Post by rough-sea »

Zababa wrote: post #169
ignoramous wrote: post #20

Earlier quoted context omitted.

OP should have tested Deno ( https://github.com/denoland/deno ) too along side Node. Alas...

Deno uses v8 too, that would be just comparing libuv to Tokio.

> just

As if everything else is the same. It is a completely independent code base.

Re: AWS Lambda Cold Start Times

#200

Post by Zababa »

rough-sea wrote: post #199
Zababa wrote: post #169

Earlier quoted context omitted.

Deno uses v8 too, that would be just comparing libuv to Tokio.

> just As if everything else is the same. It is a completely independent code base.

Well, the performance seems to be mostly the same https://mayankchoubey.github.io/Deno-vs-Node-Performance/.

> It is a completely independent code base.

Not exactly, the heavy lifting is done by v8 on both sides. Deno can do lots of things around the ergonomics, and switch around the event loop (though libuv is already pretty good), but outside of that they are mostly equivalent.

Post reply on HN