Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A better streams API is possible for JavaScript

blog.cloudflare.com

Post reply on HN
Print view View original

101–110 of 167 posts

Re: A better streams API is possible for JavaScript

#101

Post by 85392_school »

paxys wrote: post #33
conartist6 wrote: post #12

As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…

There is no such thing as Uint8Array . Uint8Array is a primitive for a bunch of bytes, because that is what data is in a stream. Adding types on top of that isn't a protocol concern but an application-level one.

A Uint8Array can be backed by buffers other than ArrayBuffer, which is where the types [0] come from.

[0] https://github.com/microsoft/TypeScript/blob/924810c077dd410...

Re: A better streams API is possible for JavaScript

#102

Post by [deleted] »

conartist6 wrote: post #12

As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…

[deleted]

Re: A better streams API is possible for JavaScript

#103

Post by sfink »

boilerupnc wrote: post #41
bikeshaving wrote: post #34

A long time ago, I wrote an abstraction called a Repeater. Essentially, the idea behind it is, what would the Promise constructor look like if it was translated to async iterables. import { Repeater } from "@repeaterjs/repeater"; const keys = new Repeater(async (push, stop) => { const listener = (ev) => { if (ev.key === "Escape") { stop(); } else { push(ev.key); } }; window.addEventListener("keyup", listener); await…

Off topic - But just wanna say - Love the cheat code! 30 Lives added :-) Nostalgia runs deep with that code. So deep - in fact, that I sign many of my emails off with "Sent by hitting Up, Up, Down, Down, Left, Right, Left, Right, B, A"

Off topic to the off topic, but that logic doesn't look right. It seems like if up is pressed, you might need to reset i to 1 or 2, not 0.

Re: A better streams API is possible for JavaScript

#104

Post by rrr_oh_man »

hinkley wrote: post #79
conartist6 wrote: post #12

As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…

I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case. If you go back a few versions, that number goes up to around 105x. I don’t recall now if I tested back to 14. There was an optimization to async handling in 16 that I recall breaking a few tests that depended on nextTick() behavior…

> I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case.

I dabble in JS and… what?! Any idea why?

Re: A better streams API is possible for JavaScript

#105

Post by amelius »

conartist6 wrote: post #12

As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…

How do you send multiple sub-streams in parallel?

Re: A better streams API is possible for JavaScript

#106

Post by hinkley »

rrr_oh_man wrote: post #104
hinkley wrote: post #79

Earlier quoted context omitted.

I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case. If you go back a few versions, that number goes up to around 105x. I don’t recall now if I tested back to 14. There was an optimization to async handling in 16 that I recall breaking a few tests that depended on nextTick() behavior…

> I did a microbenchmark recently and found that on node 24, awaiting a sync function is about 90 times slower than just calling it. If the function is trivial, which can often be the case. I dabble in JS and… what?! Any idea why?

Any await runs the logic that attempts to release the main message pump to check for other tasks or incoming IO events. And it looks like that takes around 90 instructions to loop back around to running the next line of the code, when the process is running nothing else.

If you’re doing real work, 90 instructions ain’t much but it’s not free either. If you’ve got an async accumulator (eg, otel, Prometheus) that could be a cost you care about.

Re: A better streams API is possible for JavaScript

#107

Post by bikeshaving »

sfink wrote: post #103
boilerupnc wrote: post #41

Earlier quoted context omitted.

Off topic - But just wanna say - Love the cheat code! 30 Lives added :-) Nostalgia runs deep with that code. So deep - in fact, that I sign many of my emails off with "Sent by hitting Up, Up, Down, Down, Left, Right, Left, Right, B, A"

Off topic to the off topic, but that logic doesn't look right. It seems like if up is pressed, you might need to reset i to 1 or 2, not 0.

Not accepting PRs for this. I think the logic is sound (Easter Eggs should be difficult to trigger).

Re: A better streams API is possible for JavaScript

#108

Post by apitman »

zarzavat wrote: post #69
matheus-rr wrote: post #64

The practical pain with Web Streams in Node.js is that they feel like they were designed for the browser use case first and backported to the server. Any time I need to process large files or pipe data between services, I end up fighting with the API instead of just getting work done. The async iterable approach makes so much more sense because it composes naturally with for-await-of and plays well with the rest of t…

It's news to me that anyone actually uses the web streams in node. I thought they were just for interoperability, for code that needs to run on both client and server.

You need to use them for things like Cloudflare and Denos HTTP servers, which is actually a fairly common (and nice) pattern:

https://blog.val.town/blog/the-api-we-forgot-to-name/

Re: A better streams API is possible for JavaScript

#109

Post by softfalcon »

conartist6 wrote: post #70
softfalcon wrote: post #58

Earlier quoted context omitted.

> Adding types on top of that isn't a protocol concern but an application-level one. I agree with this. I have had to handle raw byte streams at lower levels for a lot of use-cases (usually optimization, or when developing libs for special purposes). It is quite helpful to have the choice of how I handle the raw chunks of data that get queued up and out of the network layer to my application. Maybe this is because I…

My concern isn't with how you write your network layer. Use buffers in there, of course. But what if you just want to do a simple decoding transform to get a stream of Unicode code points from a steam of bytes? If your definition of a stream is that it has UInt8 values, that simply isn't possible. And there's still gonna be waaay too many code points to fall back to an async iterator of code points.

I think we're having a completely different conversation now. The parent comment I originally replied has been edited so much that I think the context of what I was referring to is now gone.

Also, I wasn't talking about building network layers, I was explicitly referring to things that use a network layer... That is, an application receiving streams of enumerable network data.

I also agree with what you're saying, we don't want UInt8, we want bits and bytes.

I'm really confused as to why the parent comment was edited so heavily. Oh well, that's social media for you.

Post reply on HN