AWS Lambda Cold Start Times

How does a single metric from a highly specialized runtime environment indicate a tech stack is dead?

There are things you can do right now [1] to mitigate these cold start issues.

Going forward, ahead-of-time compilation will be an option. [2]

Aside from cold starts, note that the improvements in .NET make ASP.NET Core one of the fastest web frameworks. [3]

The article: > “.Net has almost the same performance as Golang and Rust, but only after 1k iterations(after JIT).”

Additions like async/await and nullable reference types make it easier to write bug-free code, which for a lot of folks is a better trade off than “speaking to the hardware directly”.

.NET also runs natively on a bunch of platforms now, including ARM.

I’d call all of that continuous improvement. Perhaps even reinvention?

[1] https://docs.microsoft.com/en-us/dotnet/core/deploying/ready...

[2] https://github.com/dotnet/runtimelab/tree/feature/NativeAOT

[3] https://www.techempower.com/benchmarks/#section=test&runid=5...

The article states that 600ms is a low cold start figure. However, 600ms cold start is still unacceptable for usage by WebApps.

For webapp, the figure should be 100ms. The only platform that meets that figure (that I know of) is Velo by Wix with around 50ms cold start for node.js

Just use http functions in a Velo by Wix project - you will get 50ms cold start out of the box.

No, but I think that it's super common to discount to $0 all the work that using lambda saves you from maintenance and operations. And if you can do any of that at scale for $250/mo you're lightyears ahead of nearly everyone.

Provisioned concurrency is a bit of a non-starter for cold start latency reduction in user-facing applications. Its a tool in the toolbox, but the problem is: Let's say you set the provisioned concurrency at 10, then you have 11 concurrent requests come in. That 11th request will still get a cold start. PC doesn't scale automatically.

The ping architecture of warming up functions does scale (better) in this setup. Its not perfect, but nothing on lambda is; its possible that the ping request for User2 could get routed to a live function, which finishes, then starts handling a real request for User1, User2's real request comes in but this function is busy, so Lambda cold starts another one.

That being said, these ping functions can be cheap relative to PC. PC is not cheap; its ~$10/mo/gb/function.

With the ping architecture; you'd generally just invoke then immediately exit, so there's very little billed execution time. For sparsely executed functions, the ping architecture is better because PC is billed 24/7/365, whereas ping is billed per invoke (that being said: PC is a much cleaner solution). For high volume functions, PC may be cheaper, but it also won't work as well; if your concurrent executions are pretty stable, then its a great choice, but if its spiky high volume (as most setups are!) then it's hard to find a number which balances both cost and efficacy.

A setup which also includes Application Autoscaling, to automatically adjust the PC level with time of day or system load or whatever, would approach a better setup. But, of course, be more work.

The other frustrating thing about PC is its price relative to a cloudwatch events cron warming setup. Most sources say Lambdas stay warm for ~5 minutes, which means you only need ~15,000 invocations/month per "faux-Provisioned Concurrency" level. Theorycrafting at 100ms/invocation and 512mb: that's ~$0.012/month. PC is ~500 times more expensive than this. This kind of cron-based ping setup is very easy when PC=1; at 1+, it gets harder, but not impossible: You have a cron configured to execute a "proxy" lambda, which then concurrently invokes the real lambda N times depending on what level of PC you more-or-less want.

It's also worth noting that PC cannot be maintained on the $LATEST function version. It can only be maintained on an explicit Function Version or Alias (which does not point to $LATEST). So there's just more management overhead there; you can't just have a CI deploy functions and let AWS do everything else, you have to configure the CI to publish new versions, delete the existing PC config, and create a new PC config, maybe also adjust autoscaling settings as well if you configured it. All automatable, but definitely in the domain of "CloudOps Engineer Job Security"; anyone who tells you AWS will make your life easier is probably working for AWS.