Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eliminating JavaScript cold starts on AWS Lambda

goose.icu

Post reply on HN
Print view View original

51–60 of 72 posts

Re: Eliminating JavaScript cold starts on AWS Lambda

#51

Post by samwillis »

bastawhiz wrote: post #50
samwillis wrote: post #41

Earlier quoted context omitted.

Web workers don't share memory (other than SAB) with the main thread, they are far from traditional threads. These APIs are designed the way they are to protect end users, stop sites from consuming resources or bad code blocking the main thread. None of that is needed to be that way on the server. There is zero reason that a JS implementation cannot implement proper threads within the same memory space. The issue is…

> Web workers don't share memory (other than SAB) with the main thread, they are far from traditional threads. These APIs are designed the way they are to protect end users, stop sites from consuming resources or bad code blocking the main thread. None of that is needed to be that way on the server. It doesn't protect end users any more than it protects servers. Node could easily expose raw threading, but they don't…

> ECMAScript specifies all of the things you're talking about, and that is JavaScript, irrespective of the runtime.

I suggest you go and read the EMCAScript standard: https://ecma-international.org/publications-and-standards/st...

There is nothing in there about browser APIs, and in fact it explicitly states that the browser runtime, or any other runtime + api are not EMCAScript.

Re: Eliminating JavaScript cold starts on AWS Lambda

#52

Post by tealpod »

alserio wrote: post #36
hylaride wrote: post #20

Earlier quoted context omitted.

Java startup times would almost certainly be worse, depending on what's going on. A previous job I worked at ran Java on AWS Lambda. We ran our busiest Java lambda in a docker layer as our whole build system was designed around docker and from a compute performance point of view it was just as fast. The main issues were: * Longer init times for the JRE (including before the JIT kicks in). Our metrics had a noticeable…

You can use graalvm AoT. It mitigates the issues for some use cases when you need to use Java. Standard Java on Lambda has not been feasible for me either

Yes, We used GraalVM for data encryption project, side-effect is super fast performance. The number of req/sec also 3x.

Re: Eliminating JavaScript cold starts on AWS Lambda

#53

Post by spankalee »

samwillis wrote: post #13

Oliver is doing awesome work here. A few interesting points: - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. - There's no GC yet, and likely will be a while before it gets any. But you can get very far with no GC, particularly if you are doing something like serving web requests. You can fork a process per request and throw it away each ti…

I'm very excited by Porffor too, but a lot of what you've said here isn't correct.

> - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler.

Proffor could use types, but TypeScript's type system is very unsound and doing so could lead to serious bugs and security vulnerabilities. I haven't kept track of what Oliver's doing here lately, but I think the best and still safe thing you could do is compile an optimistic, optimized version of functions (and maybe basic blocks) based on the declared argument types, but you'd still need a type guard to fall back to the general version when the types aren't as expected.

This isn't far from what a multi-tier JIT does, and the JIT has a lot more flexibility to generate functions for the actual observed types, not just the declared types. This can be a big help when the declared types are interfaces, but in an execution you only see specific concrete types.

> or have a very simple arena allocator that works at the request level.

This isn't viable. JS semantics mean that the request handling path can generate objects that are held from outside the request's arena. You can't free them or you'd get use-after-free problems.

> - many of the restrictions that people associate with JS are due to VMs being designed to run untrusted code

This is true to some extent, but most of the restrictions are baked into the language design. JS is a single-threaded non-shared memory language by design. The lack of threads has nothing to do with security. Other sandboxed languages, famously Java, have threads. Apple experimented with multithreaded JS and it hasn't moved forward not because of security but because it breaks JS semantics. Fork is possible in JS already, because it's a VM concept, not a language concept. Low-level memory access would completely break the memory model of JS and open up even trusted code to serious bugs and security vulnerabilities.

> It is unlikely that many people would run something compiled with Porffor in a WASM runtime

Running JS in WASM is actually the thing I'm most excited about from Porffor. There are a more and more WASM runtimes, and JS is handicapped there compared to Rust. Being able to intermix JS, Rust, and Go in a single portable, secure runtime is a killer feature.

Re: Eliminating JavaScript cold starts on AWS Lambda

#54

Post by spankalee »

This is the first time that I've heard of LLRT, so here's a link for anyone else interested: https://github.com/awslabs/llrt

I still like Proffor's approach because by compiling to WASM we could have latency-optimized WASM runtimes (though I'm unsure what that might entail) that would benefit other languages as well.

Re: Eliminating JavaScript cold starts on AWS Lambda

#55

Post by ahmedhawas123 »

I've approached this historically by committing persistent/reserved instances so you always have a few instances running. This is nice on paper but feels like you're omitting what a more production-appropriate solution is. "Cold starts" aren't just slow because of init, they're also slow because that's when lots of database connections, state starts, etc happen and managing init speed won't solve that.

Re: Eliminating JavaScript cold starts on AWS Lambda

#57

Post by samwillis »

spankalee wrote: post #53
samwillis wrote: post #13

Oliver is doing awesome work here. A few interesting points: - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. - There's no GC yet, and likely will be a while before it gets any. But you can get very far with no GC, particularly if you are doing something like serving web requests. You can fork a process per request and throw it away each ti…

I'm very excited by Porffor too, but a lot of what you've said here isn't correct. > - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. Proffor could use types, but TypeScript's type system is very unsound and doing so could lead to serious bugs and security vulnerabilities. I haven't kept track of what Oliver's doing here lately, but I think…

> I haven't kept track of what Oliver's doing here lately

Please do go and check up what the state of using types to inform the compiler is (I'm not incorrect)

On the area allocator, I wasn't clear enough, as stated elsewhere this was in relation to having something similar to isolates - each having a memory space that's cleaned up on exit.

Python has almost identical semantics to JS, and has threads - there is nothing in the EMCAScript standard that would prevent them.

Re: Eliminating JavaScript cold starts on AWS Lambda

#58

Post by spankalee »

samwillis wrote: post #57
spankalee wrote: post #53

Earlier quoted context omitted.

I'm very excited by Porffor too, but a lot of what you've said here isn't correct. > - Porffor can use typescript types to significantly improve the compilation. It's in many ways more exciting as a TS compiler. Proffor could use types, but TypeScript's type system is very unsound and doing so could lead to serious bugs and security vulnerabilities. I haven't kept track of what Oliver's doing here lately, but I think…

> I haven't kept track of what Oliver's doing here lately Please do go and check up what the state of using types to inform the compiler is (I'm not incorrect) On the area allocator, I wasn't clear enough, as stated elsewhere this was in relation to having something similar to isolates - each having a memory space that's cleaned up on exit. Python has almost identical semantics to JS, and has threads - there is nothi…

It is absolutely true that it is unsafe to trust TypeScript types. I've chatted briefly with Oliver on socials before and he knows this. So I am a bit confused by this issue: https://github.com/CanadaHonk/porffor/issues/234 which says "presume the types are good and have been validated by the user before compiling". This is just not a thing that's possible. Types are often wrong in subtle ways. Casts throw everything out the window.

Dart had very similar issues and constraints and they couldn't do a proper AOT compiler that considered types until they made the type system sound. TypeScript can never do that and maintain compatibility with JS.

Isolates are already available as workers. The key thing is that you can't have shared memory, other wise you can get cross-Isolate references and have all the synchronization problems of threads.

And ECMAScript is simply just specified as a single-threaded language. You break it with shared-memory threads.

In JS, this always logs '4'. With threads that's not always the case.

    let x = 4;
    console.log(x);

Re: Eliminating JavaScript cold starts on AWS Lambda

#59

Post by no_wizard »

samwillis wrote: post #41
bastawhiz wrote: post #31

Earlier quoted context omitted.

> many of the restrictions that people associate with JS are due to VMs being designed to run untrusted code. If you compile your trusted TS/JS to native you can do many new things, such as use traditional threads, fork, and have proper low level memory access. Separating the concept of TS/JS from the runtime is long overdue. This is just outright wrong. JS limitations come from lots of things: 1. The language has al…

Web workers don't share memory (other than SAB) with the main thread, they are far from traditional threads. These APIs are designed the way they are to protect end users, stop sites from consuming resources or bad code blocking the main thread. None of that is needed to be that way on the server. There is zero reason that a JS implementation cannot implement proper threads within the same memory space. The issue is…

NodeJS has worker threads[0] already

[0]: https://nodejs.org/docs/latest/api/worker_threads.html

Re: Eliminating JavaScript cold starts on AWS Lambda

#60

Post by no_wizard »

eliseumds wrote: post #14
bapak wrote: post #4

I seriously dislike this kind of comparisons. We're faster! (please disregard the fact that we're barely more than a demo) Everyone knows about 80:20, the slowdowns will come after you start doing everything your competition does. Look at Biome. We're 15x as fast as ESLint (but disregard the fact that we don't do typeaware linting). Then comes typeaware linting and suddenly they have huge performance issues that kill…

Idk, I'm having a good experience with Biome 2 in a large codebase. 4s to do a full-check including floating promises, undeclared and cyclic dependencies, and sorting imports. Our ESLint setup used to take almost a minute. The Biome team has been fixing bugs on a daily basis. Version 2.2.0 (released 3 days) ago addressed a common high-CPU-usage bug, try it out. Edit: it's not 4s anymore, I just measured with the late…

Are simply running eslint from the root of the project? And is this a monorepo?

I have used eslint in very large projects (far more than 3000 files) and running multiple instances via a task runner makes it a breeze to keep it under 30s or less, especially if you use the cache

Post reply on HN