121–130 of 167 posts
Ideally splitting out the use cases would allow both implementations to be simpler, but that ship has probably sailed.
I tried several implementations, tweaked settings, but ultimately couldn't get around it. In some cases I had bizarre drops in activity when the consumer was below capacity.
It could have been related to the other issue they mention, which is the cost of using promises. My streams were initiating HEAPS of promises. The cost is immense when you're operating on a ton of data.
Eventually I had to implement some complex logic to accomplish batching to reduce the number of promises, then figure out some clever concurrency strategies to manage backpressure more manually. It worked well.
Once I was happy with what I had, I ported it from Deno to Go and the result was so stunningly different. The performance improvement was several orders of magnitude.
I also built my custom/native solution using the Effect library, and although some people claim it's inefficient and slow, it out-performed mine by something like 15% off the shelf, with no fine-tuning or clever ideas. I wished I'd used it from the start.
The difference is likely in that it uses a fiber-based model rather than promises at the execution layer, but I'm not sure.
Here's the WritableConsumableStream module:
https://github.com/SocketCluster/writable-consumable-stream
SocketCluster solves the problem of maintaining message order with async processing.
This feature is even more useful now with LLMs as you can process data live, transform streams with AI with no risk of mangling the message order.
I may have been the first person to use a for-await-of loop in this way with backpressure. At least on an open source project.
Earlier quoted context omitted.
How did you come up with 90? Can you shed any might on the difference between the cost of promise resolution and the cost of await? Is there any cost component with how deep in the call stack you are when an await happens?
Essentially for loop of 10k iterations comparing `fn()` versus `await fn()` fed into a microbenchmark tool, with some fiddling to detect if elimination was happening or ordering was changing things. I was bumping into PRs trying to eliminate awaits in long loops and thinking surely the overhead can’t be so high to warrant doing this, especially after node ~16. I was wrong.
‘fn()’ is the exact same function call in both cases. Only the benchmark itself was async versus sync.
Because the case under test is what’s the cost of making an async api when 90% of the calls are sync? And the answer is a lot higher than I thought.
I've only used them in dotnet, I would be very interested to read any strong opinions about the use of Streams both in practice and as an abstract point in API design.
const buffer = new UInt8Array(256)
const bytesRead = await reader.read(buffer)
if (bytesRead === 0) {
// Done
return
}Earlier quoted context omitted.
Your idea is flatten the UInt8Array into the stream. While I understand the logic, that's a terrible idea. * The overhead is massive . Now every 1KiB turns into 1024 objects. And terrible locality. * Raw byte APIs...network, fs, etc fundamentally operate on byte arrays anyway. In the most respectful way possible...this idea would only be appealing to someone who's not used to optimizing systems for efficiency.
JS engines actually are optimized to make that usage pattern fast. Small, short-lived objects with known key ordering (monomorphism) are not a major cost in JS because the GC design is generational. The smallest, youngest generation of objects can be quickly collected with an incremental GC because the perf assumption is that most of the items in the youngest generation will be garbage. This allows collection to be o…
We have run into many problems with web streams over the years and solving them has always proven to be hairy, including the unbounded memory growth from response.clone().
The Deno team implemented a stream API inspired by Go, which I was happy with, until they ultimately acquiesced to web streams.
This proposal shares some of those principles as well.
Earlier quoted context omitted.
True. But it’s also true that trying to shoehorn every use case into TCP streams is counter productive. A stream API can layer over UDP as well (reading in order of arrival with packet level framing), but such a stream would a bit weird and incompatible with many stream consumers (e.g. [de]compression). A UDP API is simpler and more naturally event (packet) oriented. The concepts don’t mix well. Still, it would be ni…
The browser does have a UDP data stream available for applications to send arbitrary bytes over UDP; it's part of WebRTC.