Live data from Hacker News

Eliminating JavaScript cold starts on AWS Lambda

goose.icu

31–40 of 72 posts

Re: Eliminating JavaScript cold starts on AWS Lambda

#31

Oliver is doing awesome work here. A few interesting points: - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. - There's no GC yet, and likely will be a while before it gets any. But you can get very far with no GC, particularly if you are doing something like serving web requests. You can fork a process per request and throw it away each ti…

> many of the restrictions that people associate with JS are due to VMs being designed to run untrusted code. If you compile your trusted TS/JS to native you can do many new things, such as use traditional threads, fork, and have proper low level memory access. Separating the concept of TS/JS from the runtime is long overdue.

This is just outright wrong. JS limitations come from lots of things:

1. The language has almost zero undefined behavior by design. Code will essentially never behave differently on different platforms.

2. JS has traditional threads in the form of web workers. This interface exists not for untrusted code but because of thread safety. That's a language design, like channels in Go, rather than a sandboxing consideration.

3. Pretty much every non-browser JS runtime has the ability to fork.

4. JS is fully garbage collected, of course you don't get your own memory management. You can use buffers to manage your own memory if you really want to. WASM lets you manage your own memory and it can run "untrusted" code in the browser with the WASM runtime; your example just doesn't hold water. There's no way you could fiddle with the stack or heap in JS without making it not JS.

5. The language comes with thirty years of baggage, and the language spec almost never breaks backwards compatibility.

Ironically Porffor has no IO at the moment, which is present in literally every JS runtime. It really has nothing to do with untrusted code like you're suggesting.

> You can fork a process per request and throw it away each time reclaiming all memory, or have a very simple arena allocator that works at the request level. It would be incredibly performant and not have the overhead of a full GC implementation.

You also must admit that this would make Porffor incompatible with existing runtimes. Code today can modify the global state, and that state can and does persist across requests. It's a common pattern to keep in-memory caches or to lazily initialize libraries. If every request is fully isolated in the future but not now, you can end up with performance cliffs or a system where a series of requests on Node return different results than a series of requests on Porffor.

As for arena allocation, this makes it even less compatible with Node (if not intractable). If means you can't write (in JS) any code that mutates memory that was initialized during startup. If you store a reference to an object in an arena in an object initialized during startup, at the end of the request when the arena is freed you now have a pointer into uninitialized memory.

How do you tell the developer what they can and cannot mutate? You can't, because any existing variable might be a reference to memory initialized during startup. Your function might receive an object as an argument that was initialized during startup or one that's wasn't, and there's no way to know whether it's safe to mutate it.

Long story short, JS must have a garage collector to free memory, or it's not JS.

> It is unlikely that many people would run something compiled with Porffor in a WASM runtime, but the portability it brings is very compelling.

Node (via SEA in v20), bun, and deno all have built in tooling for generating a self-contained binary. Granted, the runtime needs to work for your OS and CPU, but the exact same thing could be said about a WASM runtime.

And of course there are hundreds of mature bundlers that can compile JS into a single file that runs in various runtimes without ever thinking about platform. It's weird to even consider portability of JS as a benefit because JS is already almost maximally portable.

> This experiment from Oliver doesn't show that Porffor is ready for production, but it does validate that he is on the right track, and that the ideas he is exploring are correct.

It validates that the approach to building a compiler is correct, but it says little about whether the project will eventually be usable and good. It's unlikely it'll get faster, because robust JS compatibility will require more edge cases to be handled than it currently does, and as Porffor's own README says, it's still slower than most JITted runtimes. A stable release might not yield much.

Re: Eliminating JavaScript cold starts on AWS Lambda

#32
post #9
post #8

Earlier quoted context omitted.

> If you think something is slow then make it faster and open a pull request. But I want someone else to make it faster without me having to do anything.

In my experience the type of developer using JS for their Web apps rarely overlaps with someone comfortable doing high performance C++ runtime work.

In my experience, the type of developer complaining about someone using JS is most of the time is some native C++/C/Rust (neck|grey)-beard. Most of the time they deflect it with "I don't support Chromium/Google" (they never elaborate why they don't contribute to Mozilla).

Re: Eliminating JavaScript cold starts on AWS Lambda

#33

Oliver is doing awesome work here. A few interesting points: - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. - There's no GC yet, and likely will be a while before it gets any. But you can get very far with no GC, particularly if you are doing something like serving web requests. You can fork a process per request and throw it away each ti…

Based on how much imported libraries are relied upon, it makes sense to treat everything as untrusted. Unless you write every line yourself/in-house, code should be considered untrusted.

I would be curious which attack vectors change or become safe after compiling though.

Re: Eliminating JavaScript cold starts on AWS Lambda

#34

Awesome work, but I am genuinely curios about the use cases where the 200ms init time being a problem?

Serverless implementations where lambdas are being used for APIs so +200ms on a single call is not good.

It's not 200ms per call. It's 200ms cold start. Once a lambda is started, you don't pay that cost again. In most cases, you pay this a few times after a deployment or when traffic is increasing and not again.

Re: Eliminating JavaScript cold starts on AWS Lambda

#35
post #23

Awesome work, but I am genuinely curios about the use cases where the 200ms init time being a problem?

Maybe its a problem when you’re paying 200ms per small request and you have millions of them ?

If you have cold starts on all of your millions of requests, something is very wrong with your code.

Re: Eliminating JavaScript cold starts on AWS Lambda

#36
post #15

It'd be good if AWS Lambda provided a wasm runtime option. Cold start times for WebAssembly can be sub-millisecond. It'd also be interesting to see comparisons to the Java and .NET runtimes on AWS Lambda.

Java startup times would almost certainly be worse, depending on what's going on. A previous job I worked at ran Java on AWS Lambda. We ran our busiest Java lambda in a docker layer as our whole build system was designed around docker and from a compute performance point of view it was just as fast. The main issues were: * Longer init times for the JRE (including before the JIT kicks in). Our metrics had a noticeable…

You can use graalvm AoT. It mitigates the issues for some use cases when you need to use Java. Standard Java on Lambda has not been feasible for me either

Re: Eliminating JavaScript cold starts on AWS Lambda

#37
post #24

Tl;dr Use an experimental (as in, 60% of ECMA tests passing, "currently no good I/O or Node compat") AOT compiler for JS. You remove the cold start by removing the runtime, at the cost of maybe your JavaScript working and not having a garbage collector.

It might be reasonable to go without a garbage collector if your whole lifetime is so short. GC by decapitation. Kinda clever. But other than that it's impossible to assess performance with such a tiny toy.

If you throw away your memory on every request, every request is a cold start. You've reduced your one-time startup cost but now you pay that reduced cost on every request.

Re: Eliminating JavaScript cold starts on AWS Lambda

#38
post #33

Oliver is doing awesome work here. A few interesting points: - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. - There's no GC yet, and likely will be a while before it gets any. But you can get very far with no GC, particularly if you are doing something like serving web requests. You can fork a process per request and throw it away each ti…

Based on how much imported libraries are relied upon, it makes sense to treat everything as untrusted. Unless you write every line yourself/in-house, code should be considered untrusted. I would be curious which attack vectors change or become safe after compiling though.

The point of the js engine sandbox is to protect the user in the browser - it's completely redundant on the server. Supply chain attacks are real, but only Deno has tried to fix that through permissions/rules.

I don't think anything changes with compile to native on the server.

Re: Eliminating JavaScript cold starts on AWS Lambda

#40

Oliver is doing awesome work here. A few interesting points: - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. - There's no GC yet, and likely will be a while before it gets any. But you can get very far with no GC, particularly if you are doing something like serving web requests. You can fork a process per request and throw it away each ti…

> many of the restrictions that people associate with JS are due to VMs being designed to run untrusted code. If you compile your trusted TS/JS to native you can do many new things, such as use traditional threads, fork, and have proper low level memory access. Separating the concept of TS/JS from the runtime is long overdue. This is just outright wrong. JS limitations come from lots of things: 1. The language has al…

What a strange (and strangely adversarial) comment.

Almost none of your criticisms connects with anything that the other person wrote.

Post reply on HN