Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eliminating JavaScript cold starts on AWS Lambda

goose.icu

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Eliminating JavaScript cold starts on AWS Lambda

#61

Post by no_wizard »

eliseumds wrote: post #18
bapak wrote: post #16

Earlier quoted context omitted.

I did try and it just stalls, taking down my CPU with it. I had moved onto other tasks and found it later still killing a few cores. It's in a monorepo with maybe 3000 files. I have no trust in the project.

Yeah I understand the frustration, I had issues setting up ESLint/Prettier years ago and spent a few hours getting the Biome configuration right. Also a monorepo with ~4000 TS/TSX files. If you ever feel like trying it again, make sure that you have "files.maxSize" and "files.ignoreUnknown" set. And be very careful with the "files.includes" list.

It sure would have been nice if biome used a better include syntax. I dislike the new way of doing it in biome 2, because it creates ambiguity and works differently than damn nearly every other tool I use

Re: Eliminating JavaScript cold starts on AWS Lambda

#62

Post by LtWorf »

thdhhghgbhy wrote: post #39

No big corporate will ever use this, they'd be too worried about the compiler being compromised in some way. Llrt will never go primetime either, so we're stuck with the full Node runtime for a while.

Corporations use pypi and npm, I think they don't care about being compromised.

Re: Eliminating JavaScript cold starts on AWS Lambda

#63

Post by anon7000 »

unfunco wrote: post #11

I really don't struggle that much with cold starts on Node.js/Lambda, and I don't do anything special, my build commands look like: esbuild src/handler.ts --bundle --external:@aws-lambda-powertools --external:@aws-sdk --minify --outfile=dist/handler.js --platform=node --sourcemap --target=es2022 --tree-shaking=true Maybe I'm not doing as much as others in my functions and I tend to stick within the AWS ecosystem, so…

Sure, but the approach mentioned here benchmarks with median performance of 16ms. 100ms isn’t great especially if it’s only one part of everything that needs to happen

Re: Eliminating JavaScript cold starts on AWS Lambda

#64

Post by hnlmorg »

ttoinou wrote: post #23
moqizhengz wrote: post #22

Awesome work, but I am genuinely curios about the use cases where the 200ms init time being a problem?

Maybe its a problem when you’re paying 200ms per small request and you have millions of them ?

The hypothetical 200ms is for cold starts. That means when the docker container that runs behind the lambda first starts up.

When a lambda is invocated, that docker container isn’t immediately destroyed. It’s kept around for reuse.

So there isn’t a realistic scenario where you would get a million cold starts in a month.

Re: Eliminating JavaScript cold starts on AWS Lambda

#66

Post by hvb2 »

bastawhiz wrote: post #34
ericmcer wrote: post #27

Earlier quoted context omitted.

Serverless implementations where lambdas are being used for APIs so +200ms on a single call is not good.

It's not 200ms per call. It's 200ms cold start. Once a lambda is started, you don't pay that cost again. In most cases, you pay this a few times after a deployment or when traffic is increasing and not again.

> you pay this a few times after a deployment or when traffic is increasing and not again.

This is now how lambda works. A lambda instance sticks around for 10s of minutes at most. So you'll have cold starts every day no matter how often you deploy.

If you have very little traffic you might actually have cold starts often as the lifetime of a lambda is determined based on its use.

Unfortunately AWS has always been rather unspecific about this behavior but here's a link to a pluralsight blog https://www.pluralsight.com/resources/blog/cloud/how-long-do...

Re: Eliminating JavaScript cold starts on AWS Lambda

#67

Post by hvb2 »

jfengel wrote: post #24
bastawhiz wrote: post #21

Tl;dr Use an experimental (as in, 60% of ECMA tests passing, "currently no good I/O or Node compat") AOT compiler for JS. You remove the cold start by removing the runtime, at the cost of maybe your JavaScript working and not having a garbage collector.

It might be reasonable to go without a garbage collector if your whole lifetime is so short. GC by decapitation. Kinda clever. But other than that it's impossible to assess performance with such a tiny toy.

> It might be reasonable to go without a garbage collector if your whole lifetime is so short.

That isn't how lambda works. A single lambda instance can run for 10s of minutes handling thousands or requests. So not cleaning up memory sounds like a problem to me since you're billed by the GB/second

Re: Eliminating JavaScript cold starts on AWS Lambda

#68

Post by crabmusket »

no_wizard wrote: post #59
samwillis wrote: post #41

Earlier quoted context omitted.

Web workers don't share memory (other than SAB) with the main thread, they are far from traditional threads. These APIs are designed the way they are to protect end users, stop sites from consuming resources or bad code blocking the main thread. None of that is needed to be that way on the server. There is zero reason that a JS implementation cannot implement proper threads within the same memory space. The issue is…

NodeJS has worker threads[0] already [0]: https://nodejs.org/docs/latest/api/worker_threads.html

These are very similar to web workers, they don't share memory other than via SharedArrayBuffer instances. For anything else you use message passing.

Re: Eliminating JavaScript cold starts on AWS Lambda

#69

Post by torginus »

I don't want to take away from the appreciation of this awesome technical achievement, but in practice I have noticed that:

- Cold starts are kinda rare, sure it sucks that your request takes 600ms, but that means you are the first user. If you would've been served by a container that was just scaled up, you'd have been waiting for much longer

- Microservices and AWS lambda are inherently stateless, and do a ton of things to make themselves useful - get credentials, establish db connections, query configuration endpoints, all of which take time, usually more than your runtime spends booting up.

As much as I like lambdas for their deployment and operational simplicity, if you want the best UX, they have inherent technical limitations which make them the wrong choice.

Post reply on HN