51–60 of 167 posts
Earlier quoted context omitted.
Browsers
Since when are browsers themselves built in JavaScript? Mainstream, fast ones?
To your question, I was about to point out Firefox[1], but realized you clarified 'mainstream'[2]...
As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…
As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…
I think the more generic stream concept is interesting, but their proposal is based on different underlying assumptions. From what it looks like, they want their streams to be compatible with AsyncIterator so it'd fit into existing ecosystem of iterators. And I believe the Uint8Array is there for matching OS streams as they tend to move batches of bytes without having knowledge about the data inside. It's probably no…
And yes, because it's a new abstraction the compat story is interesting. We can easily wrap any source so we'll have loads of working sources. The fight will be getting official data sinks that support a new kind of stream
We deserve a better language than JavaScript. Sadly it will never happen. WebAssembly failed to keep some of its promises here.
classic case of not using an await before your promise
Earlier quoted context omitted.
I’m building everything from first principles, I’m not climbing the exponential curve with some billionaire that has to finance it.
I really doubt you are. you're not visiting the transistor shop every time you want to build a react component
As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…
There is no such thing as Uint8Array . Uint8Array is a primitive for a bunch of bytes, because that is what data is in a stream. Adding types on top of that isn't a protocol concern but an application-level one.
I agree with this.
I have had to handle raw byte streams at lower levels for a lot of use-cases (usually optimization, or when developing libs for special purposes).
It is quite helpful to have the choice of how I handle the raw chunks of data that get queued up and out of the network layer to my application.
Maybe this is because I do everything from C++ to Javascript, but I feel like the abstractions of cleanly getting a stream of byte arrays is already so many steps away from actual network packet retrieval, serializing, and parsing that I am a bit baffled folks want to abstract this concern away even more than we already do.
I get it, we all have our focuses (and they're ever growing in Software these days), but maybe it's okay to still see some of the bits and bytes in our systems?
If you're dealing with small objects at the production side, like individual tag names, attributes, bindings, etc. during SSR., the natural thing to do is to just write() each string. But then you see that performance is terrible compared to sync iterables, and you face a choice:
1. Buffer to produce larger chunks and less stack switching. This is the exact same thing you need to do with Streams. or
2. Use sync iterables and forgo being able to support async components.
We faced this problem in Lit-SSR and our solution was to move to sync iterables that can contain thunks. If the producer needs to do something async it sends a thunk, and if the consumer receives a thunk it must call and await the thunk before getting the next value. If the consumer doesn't even support async values (like in a sync renderToString() context) then it can throw if it receives one.
This produced a 12-18x speedup in SSR benchmarks over components extracted from a real-world website.
I don't think a Streams API could adopt such a fragile contract (ie, you call next() too soon it will break), but having some kind of way where a consumer can pull as many values as possible in one microtask and then await only if an async value is encountered would be really valuable, IMO. Something like `write()` and `writeAsync()`.
The sad thing here is that generators are really the right shape for a lot of these streaming APIs that work over tree-like data, but generators are far too slow.
As it happens i have an even better API than this article proposes! They propose just using an async iterator of UInt8Array. I almost like this idea, but it's not quite all the way there. They propose this: type Stream = { next(): Promise }> } I propose this, which I call a stream iterator! type Stream = { next(): { done, value: T } | Promise } Obviously I'm gonna be biased, but I'm pretty sure my version is also obj…
Your idea is flatten the UInt8Array into the stream. While I understand the logic, that's a terrible idea. * The overhead is massive . Now every 1KiB turns into 1024 objects. And terrible locality. * Raw byte APIs...network, fs, etc fundamentally operate on byte arrays anyway. In the most respectful way possible...this idea would only be appealing to someone who's not used to optimizing systems for efficiency.
My reference point is from a noob experience with Golang - where I was losing a bunch of efficiency to channel overhead from sending millions of small items. Sending batches of ~1000 instead cut that down to a negligible amount. It is a little less ergonomic to work with (adding a nesting level to your loop).