AWS Lambda Cold Start Times

I mean, there's GraalVM. Not sure how much more reinventing you think is warranted not to be "dead"

> In my experience, Lambdas are best used as glue between AWS components, message processors and cron style tasks.

i think lambda is the defacto standard now for integration on aws.

but they are pushing lambda hard for other workload too

C++. 200ms cold start. provided.al2 runtime environment.

Lambda success story:

Started with a .NET Core API about a year ago. Monolith-first. Mix of clients across mobile and React. Async/await is one of the better things about C# (the language used for ASP.NET Core) and as a result, we were able to do things you'd never consider doing in-process on a system like Ruby on Rails (right on the thread serving the HTTP request), like transcoding a 12 megapixel HEIC upload into JPEG. We just did it, left the connection open, and when it was done, returned an HTTP 200 OK.

That worked well for a while and let us serve tons of clients on a single Heroku dyno. The problem: memory. Resizing images takes tens/hundreds of MB when you're doing it into three different formats.

Over the last two weeks, I extracted the HEIC->JPEG transcode/resize out of our monolith into a Lambda. I'm extremely happy with how it turned out. We went with C++ because the whole idea was performance, we're going to be doing point cloud processing and other heavyweight stuff, and wanted fine-grained control of memory. Our process has 28MB of dynamic libraries (.so files), starts in 200ms, and runs comfortably on a 512MB instance. We moved to 1024 to provide a margin of safety just in case we get a really large image. The system has progressed into "I don't even think about it"-level reliably. It just works and I pay something like $1 for 40-50k transcode operations. No EC2 instances to manage, no queues, no task runners, no Ruby OOM, no running RabbitMQ, none of that (former ops engineer at a very high-scale analytics company).

As a general comment, I don't see many cloud services written in C/C++. This is no doubt partly because those skills just aren't widespread. But I think the bigger lesson is that it might be worth adding a little bit of development complexity to save 10x as much ops overhead. When I explained this setup to my friend, his first reaction was, "Why didn't you just put ImageMagick (the binary) into a container?" Once I explained that actually, I need to get images from S3, and write them into several formats, and manipulate their S3 keys in somewhat complex ways, fire off an HTTP request to a server, and pass a JWT around...sure, I could write this in a shell script, with wget, and curl, and everthing else. But at some point you just have to write the right code for the job using the right tools.

I think hybrid approaches like this make the most sense. .NET and Java are great high-productivity tools for running server apps where memory is relatively abundant. I wouldn't try to move a system like that onto Lambda any more than I'd try to do something that more naturally fits with a queue/worker pattern on a webserver. This seems kind of obvious but if I'm being honest, it's probably experience talking a bit.

It's also neat to just get back to the metal a bit. Drop the containers, runtime environments, multi-hundred-MB deployment packages, just ship a xx MB package up to the cloud, deploy it, and have it run as a standalone linux binary with all the speed and simplicity that brings. Modern C++ is a totally different animal than 90s C++, I'd encourage giving it a try if you haven't in a while.

Lambda@Edge helps with latency, definitely not with cold start times. You also can't buy provisioned Lambda@Edge, so for low traffic scenarios it's even worse than typical Lambda (where you can easily provision capacity, or keep on-demand capacity warm, which is not so cheap or easy when that must be done across every CloudFront cache region). For a low traffic environment, running e.g. 3-5 regular provisioned Lambda functions in different regions will produce a much more sensible latency distribution for end users than Edge would.

CloudFront Functions have no cold start, but their execution time is sorely restricted (1ms IIRC). You can't do much with them except origin selection, header tweaks or generating redirects, and there is no network or filesystem IO whatsoever.

> My company ran into this issue using lambdas to process some data where the 99% of requests were fine running in 256mb but a few required more.

Well, you could have deployed 2 lambdas: one as a coordinator with 256MB and another worker with the Ram you need called in those 1% of cases.

And remember that if you had to do that on a server with a virtual machine you still needed all the ram deployed 24/7.