Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A better streams API is possible for JavaScript

blog.cloudflare.com

Post reply on HN
Print view View original

131–140 of 167 posts

Re: A better streams API is possible for JavaScript

#131

Post by mlhpdx »

kaoD wrote: post #113
mlhpdx wrote: post #50

Earlier quoted context omitted.

True. But it’s also true that trying to shoehorn every use case into TCP streams is counter productive. A stream API can layer over UDP as well (reading in order of arrival with packet level framing), but such a stream would a bit weird and incompatible with many stream consumers (e.g. [de]compression). A UDP API is simpler and more naturally event (packet) oriented. The concepts don’t mix well. Still, it would be ni…

TCP or UDP are orthogonal to this, so the original comment feels like a non sequitur. These streams are not network streams and could be a file, chunks of procedural audio, or whatever.

I agree, the stream concept should be (and is) very general and ideally cover all these cases - any “bytes producing” source.

I was trying to be open minded about that and conceive a stream API over a UDP socket. It’d work IMHO, but be a little odd compared to an event-like API.

Re: A better streams API is possible for JavaScript

#132

Post by [deleted] »

hinkley wrote: post #79
conartist6 wrote: post #12

As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…

I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case. If you go back a few versions, that number goes up to around 105x. I don’t recall now if I tested back to 14. There was an optimization to async handling in 16 that I recall breaking a few tests that depended on nextTick() behavior…

[deleted]

Re: A better streams API is possible for JavaScript

#133

Post by billywhizz »

hinkley wrote: post #79
conartist6 wrote: post #12

As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…

I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case. If you go back a few versions, that number goes up to around 105x. I don’t recall now if I tested back to 14. There was an optimization to async handling in 16 that I recall breaking a few tests that depended on nextTick() behavior…

that sounds way off. there is a big perf hit to async, but it appears to be roughly 100 nanoseconds overhead per call. when benchmarking you have to ensure your function is not going to be optimized away if it doesn't do anything or inputs/outputs never change.

you can run this to see the overhead for node.js Bun and Deno: https://gist.github.com/billywhizz/e8275a3a90504b0549de3c075...

Re: A better streams API is possible for JavaScript

#134

Post by paulddraper »

fwip wrote: post #60
paulddraper wrote: post #35

Earlier quoted context omitted.

Your idea is flatten the UInt8Array into the stream. While I understand the logic, that's a terrible idea. * The overhead is massive . Now every 1KiB turns into 1024 objects. And terrible locality. * Raw byte APIs...network, fs, etc fundamentally operate on byte arrays anyway. In the most respectful way possible...this idea would only be appealing to someone who's not used to optimizing systems for efficiency.

I agree with your post, but in practice, couldn't you get back that efficiency by setting T = UInt8Array? That is, write your stream to send / receive arrays. My reference point is from a noob experience with Golang - where I was losing a bunch of efficiency to channel overhead from sending millions of small items. Sending batches of ~1000 instead cut that down to a negligible amount. It is a little less ergonomic to…

Yes, then you are back to Cloudflare's suggested interface.

An async iterator of buffers.

Re: A better streams API is possible for JavaScript

#135

Post by paulddraper »

conartist6 wrote: post #44
paulddraper wrote: post #35

Earlier quoted context omitted.

Your idea is flatten the UInt8Array into the stream. While I understand the logic, that's a terrible idea. * The overhead is massive . Now every 1KiB turns into 1024 objects. And terrible locality. * Raw byte APIs...network, fs, etc fundamentally operate on byte arrays anyway. In the most respectful way possible...this idea would only be appealing to someone who's not used to optimizing systems for efficiency.

JS engines actually are optimized to make that usage pattern fast. Small, short-lived objects with known key ordering (monomorphism) are not a major cost in JS because the GC design is generational. The smallest, youngest generation of objects can be quickly collected with an incremental GC because the perf assumption is that most of the items in the youngest generation will be garbage. This allows collection to be o…

Brother, you are talking about one object for every byte.

That is a madness. And often for no reason...you're copying or arranging bytes in lists anyway.

Re: A better streams API is possible for JavaScript

#136

Post by pornel »

socalgal2 wrote: post #83

Promises should not be a big overhead. If they are, that seems like a bug in JS engines. At a native level (C++/rust), a Promise is just a closure added to a list of callbacks for the event loop. Yes, if you did 1 per streamed byte then it would be huge but if you're doing 1 promise per megabyte, (1000 per gig), it really shouldn't add up 1% of perf.

In Rust, a Future can have only exactly one listener awaiting it, which means it doesn't need dynamic allocation and looping for an arbitrary number of .then() callbacks. This allows merging a chain of `.await`ed futures into a single state machine. You could get away with awaiting even on every byte.

Re: A better streams API is possible for JavaScript

#138

Post by vlovich123 »

softfalcon wrote: post #111
conartist6 wrote: post #71

Earlier quoted context omitted.

Yeah I don't think that's generally a problem for JS engines because of the incremental garbage collector. If you make all your memory usage patterns possible for the incremental collector to collect, you won't experience noticeable hangups because the incremental collector doesn't stop the world. This was already pretty important for JS since full collections would (do) show up as hiccups in the responsiveness of th…

Interesting, thanks for the info, I'll do some reading on what you're saying. I agree, you're right about JS having issues with hiccups in the UI due to scheduling on a single process thread. Makes a lot of sense, cool that the garbage collector can run independently of the call stack and function scheduler.

OP doesn’t know what he’s talking about. Creating an object per byte is insane to do if you care about performance. It’ll be fine if you do 1000 objects once or this isn’t particularly performance sensitive. That’s fine. But the GC running concurrently doesn’t change anything about that, not to mention that he’s wrong and the scavenger phase for the young generation (which is typically where you find byte arrays being processed like this) is stop the world. Certain phases of the old generation collection are concurrent but notably finalization (deleting all the objects) is also stop the world as is compaction (rearranging where the objects live).

This whole approach is going to be orders of magnitude of overhead and the GC can’t do anything because you’d still be allocating the object, setting it up, etc. Your only hope would be the JIT seeing through this kind of insanity and rewriting to elide those objects but that’s not something I’m aware AOT optimizer can do let alone a JIT engine that needs to balance generating code over fully optimal behavior.

Don’t take my word for it - write a simple benchmark to illustrate the problem. You can also look throughout the comment thread that OP is just completely combative with people who clearly know something and point out problems with his reasoning.

Re: A better streams API is possible for JavaScript

#139

Post by esprehn »

conartist6 wrote: post #97
socalgal2 wrote: post #83

Promises should not be a big overhead. If they are, that seems like a bug in JS engines. At a native level (C++/rust), a Promise is just a closure added to a list of callbacks for the event loop. Yes, if you did 1 per streamed byte then it would be huge but if you're doing 1 promise per megabyte, (1000 per gig), it really shouldn't add up 1% of perf.

I'm fairly sure it's not Promises that are actually the heavy part but the `await` keyword as used in the `for await` loop. That's because await tries to preserve the call stack for debugging, making it a relatively high-level expensive construct from a perf perspective where a promise is a relatively low-level cheap one. So if you're going to flatten everything into one stream then you can't have a for loop implemen…

Async call stacks is an optional feature when the devtools is open. There shouldn't be overhead from await like that?

Re: A better streams API is possible for JavaScript

#140

Post by esprehn »

Web Streams do feel rather painful compared to other languages. The author ends up basically describing kotlin flows which are great and I wish the web would adopt that model (Observables wanted to be that but the API is much worse than flows in practice).

Fwiw the original Streams API could have been simpler even without async iterators.

  interface Stream {
    // Return false from the callback to stop early.
    // Result is if the stream was completed.
    forEach(callback: (chunk: T) => Promise): Promise
  }
Similarly adding a recycleBuffer(chunk) method would have gone a long way towards BYOB without all the ceremony.

If we're optimizing allocations we can also avoid all the {done,value} records and return a semaphore value for the end in the proposed API.

(Web) API design is really difficult and without a voice in the room pushing really hard on ergonomics and simplicity it's easy to solve all the use cases but end up with lots of awkward corners and costs later.

Post reply on HN