Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A better streams API is possible for JavaScript

blog.cloudflare.com

Post reply on HN
Print view View original

111–120 of 167 posts

Re: A better streams API is possible for JavaScript

#111

Post by softfalcon »

conartist6 wrote: post #71
softfalcon wrote: post #62

Earlier quoted context omitted.

What happens when I send an extremely high throughput of data and the scheduler decides to pause garbage collection due to there being too many interrupts to my process sending network events? (a common way network data is handed off to an application in many linux distros) Are there any concerns that the extra array overhead will make the application even more vulnerable to out of memory errors while it holds off on…

Yeah I don't think that's generally a problem for JS engines because of the incremental garbage collector. If you make all your memory usage patterns possible for the incremental collector to collect, you won't experience noticeable hangups because the incremental collector doesn't stop the world. This was already pretty important for JS since full collections would (do) show up as hiccups in the responsiveness of th…

Interesting, thanks for the info, I'll do some reading on what you're saying. I agree, you're right about JS having issues with hiccups in the UI due to scheduling on a single process thread.

Makes a lot of sense, cool that the garbage collector can run independently of the call stack and function scheduler.

Re: A better streams API is possible for JavaScript

#112

Post by conartist6 »

hinkley wrote: post #106
rrr_oh_man wrote: post #104

Earlier quoted context omitted.

> I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case. I dabble in JS and… what?! Any idea why?

Any await runs the logic that attempts to release the main message pump to check for other tasks or incoming IO events. And it looks like that takes around 90 instructions to loop back around to running the next line of the code, when the process is running nothing else. If you’re doing real work, 90 instructions ain’t much but it’s not free either. If you’ve got an async accumulator (eg, otel, Prometheus) that could…

How did you come up with 90? Can you shed any might on the difference between the cost of promise resolution and the cost of await? Is there any cost component with how deep in the call stack you are when an await happens?

Re: A better streams API is possible for JavaScript

#113

Post by kaoD »

mlhpdx wrote: post #50
afavour wrote: post #4

Earlier quoted context omitted.

UDP is a protocol, not an API

True. But it’s also true that trying to shoehorn every use case into TCP streams is counter productive. A stream API can layer over UDP as well (reading in order of arrival with packet level framing), but such a stream would a bit weird and incompatible with many stream consumers (e.g. [de]compression). A UDP API is simpler and more naturally event (packet) oriented. The concepts don’t mix well. Still, it would be ni…

TCP or UDP are orthogonal to this, so the original comment feels like a non sequitur. These streams are not network streams and could be a file, chunks of procedural audio, or whatever.

Re: A better streams API is possible for JavaScript

#114

Post by szmarczak »

> This pattern has caused connection pool exhaustion in Node.js applications using undici (the fetch() implementation built into Node.js), and similar issues have appeared in other runtimes.

That's an inherent flaw of garbage collected languages. Requiring to explicitly close a resource feels like writing C. Otherwise you have a memory leak or resource exhaustion, because the garbage collector may or may not free the resource. Even C++ is better at this, because it does reference counting instead.

Re: A better streams API is possible for JavaScript

#115

Post by pcthrowaway »

bikeshaving wrote: post #34

A long time ago, I wrote an abstraction called a Repeater. Essentially, the idea behind it is, what would the Promise constructor look like if it was translated to async iterables. import { Repeater } from "@repeaterjs/repeater"; const keys = new Repeater(async (push, stop) => { const listener = (ev) => { if (ev.key === "Escape") { stop(); } else { push(ev.key); } }; window.addEventListener("keyup", listener); await…

In the repeater callback, you're both calling the stop argument and awaiting it. Is it somehow both a function and a promise? Is this possible in JS?

edit: I found where stop is created[1]. I can't say I've seen this pattern before, and the traditionalist in me wants to dislike the API for contradicting conventions, but I'm wondering if this was designed carefully for ergonomic benefits that outweigh the cost of violating conventions. Or if this was just toy code to try out new patterns, which is totally legit also

[1]: https://github.com/repeaterjs/repeater/blob/638a53f2729f5197...

Re: A better streams API is possible for JavaScript

#116

Post by bikeshaving »

pcthrowaway wrote: post #115
bikeshaving wrote: post #34

A long time ago, I wrote an abstraction called a Repeater. Essentially, the idea behind it is, what would the Promise constructor look like if it was translated to async iterables. import { Repeater } from "@repeaterjs/repeater"; const keys = new Repeater(async (push, stop) => { const listener = (ev) => { if (ev.key === "Escape") { stop(); } else { push(ev.key); } }; window.addEventListener("keyup", listener); await…

In the repeater callback, you're both calling the stop argument and awaiting it. Is it somehow both a function and a promise? Is this possible in JS? edit: I found where stop is created[1]. I can't say I've seen this pattern before, and the traditionalist in me wants to dislike the API for contradicting conventions, but I'm wondering if this was designed carefully for ergonomic benefits that outweigh the cost of viol…

Yes, the callable promise abstraction is just a bit of effort:

  let resolveRef;
  const promise = new Promise((res) => { resolveRef = res; });
  
  const callback = (data) => {
    // Do work...
    resolveRef(data); // This "triggers" the await
  };

  Object.assign(callback, promise);
There’s a real performance cost to awaiting a fake Promise though, like `await regularPromise` bypasses the actual thenable stuff.

Re: A better streams API is possible for JavaScript

#117

Post by hinkley »

conartist6 wrote: post #112
hinkley wrote: post #106

Earlier quoted context omitted.

Any await runs the logic that attempts to release the main message pump to check for other tasks or incoming IO events. And it looks like that takes around 90 instructions to loop back around to running the next line of the code, when the process is running nothing else. If you’re doing real work, 90 instructions ain’t much but it’s not free either. If you’ve got an async accumulator (eg, otel, Prometheus) that could…

How did you come up with 90? Can you shed any might on the difference between the cost of promise resolution and the cost of await? Is there any cost component with how deep in the call stack you are when an await happens?

Essentially for loop of 10k iterations comparing `fn()` versus `await fn()` fed into a microbenchmark tool, with some fiddling to detect if elimination was happening or ordering was changing things.

I was bumping into PRs trying to eliminate awaits in long loops and thinking surely the overhead can’t be so high to warrant doing this, especially after node ~16. I was wrong.

Re: A better streams API is possible for JavaScript

#118

Post by tprgoreturnon »

teaearlgraycold wrote: post #98
shevy-java wrote: post #8

We deserve a better language than JavaScript. Sadly it will never happen. WebAssembly failed to keep some of its promises here.

As wonky as JS is I really like it. Typescript has done such a good job at making it fun to use.

[dead]

Re: A better streams API is possible for JavaScript

#119

Post by whaleofatw2022 »

softfalcon wrote: post #109
conartist6 wrote: post #70

Earlier quoted context omitted.

My concern isn't with how you write your network layer. Use buffers in there, of course. But what if you just want to do a simple decoding transform to get a stream of Unicode code points from a steam of bytes? If your definition of a stream is that it has UInt8 values, that simply isn't possible. And there's still gonna be waaay too many code points to fall back to an async iterator of code points.

I think we're having a completely different conversation now. The parent comment I originally replied has been edited so much that I think the context of what I was referring to is now gone. Also, I wasn't talking about building network layers, I was explicitly referring to things that use a network layer... That is, an application receiving streams of enumerable network data. I also agree with what you're saying, we…

Not the person originally replying, but as someone who avoids JS I have to ask whether the abstraction you provide may have additional baggage as far as framing/etc.

Ironically, naively, I'd expect something more like a callback where you would specify how your input gets written to a buffer, but again im definitely losing a lot of nuance from not doing JS in a long while.

Re: A better streams API is possible for JavaScript

#120

Post by hackrmn »

jasnell wrote: post #26
dilap wrote: post #7

> The problems aren't bugs; they're consequences of design decisions that may have made sense a decade ago, but don't align with how JavaScript developers write code today. > I'm not here to disparage the work that came before — I'm here to start a conversation about what can potentially come next. Terrible LLM-slop style. Is Mr Snell letting an LLM write the article for him or has he just appropriated the style?

Heh, I was using emdashes and tricolons long before LLMs appropriated the style but I did let the agent handle some of the details on this. Honestly, it really is just easier sometimes... Especially for blogs posts like this when I've also got a book I'm writing, code to maintain etc. Use tools available to make life easier.

Just want to raise my hand and say I too have been using em dashes for considerably longer than LLM has been on every hacker's lips. It's obviously not great being accused of being an AI just because one has a particular style of writing...
Post reply on HN