201–210 of 244 posts
Earlier quoted context omitted.
One way is to eagerly call JSON.parse as fragments are coming in. If you also split on json semantic boundaries like quotes/closing braces/closing brackets, you can detect valid objects and start processing them while the stream continues.
Interesting approach! thanks for sharing
Earlier quoted context omitted.
What stops you from parsing tokens from a stream like a SAX parser for JSON? [ ["aaa", "bbb"], { "name", "foo" } ] Start array Start array String aaa String bbb End array Start object Key name String foo End object End array
Nothing, really, but I don’t have the bandwidth to write JSAX. I wonder why it hasn’t already been done by someone more qualified than I am. I suspect that I’d find out, if I started doing it. You can do that, in a specialized manner, with PHP, and Streaming JSON Parser[0]. I use that, in one of my server projects[1]. It claims to be JSON SAX, but I haven’t really done an objective comparison, and it specializes for…
In response to "Json is just a packing format that does have that [streaming] limitation".
Earlier quoted context omitted.
> Closer to the end of the article I slightly pivot to talk about RSC. Not again, please.
The best part about someone else's writing is you can just ignore it.
> I’d like to challenge more tools to adopt progressive streaming of data. It's a solved problem. Use HTTP/2 and keep the connection open. You now have effectively a stream. Get the top-level response: { header: "/posts/1/header", post: "/posts/1/body", footer: "/posts/1/footer" } Now reuse the same connection to request the nested data, which can all have more nested links in them, and so on.
> Now reuse the same connection to request the nested data, which can all have more nested links in them, and so on. This still involves multiple round-trips though. The approach laid out in the article lets you request exactly the data you need up-front and the server streams it in as it becomes available, e.g. cached data first, then data from the DB, then data from other services, etc.
The thing I have seem in performance is people trying to shave ms loading a page, while they fetch several mbs and do complex operations in the FE, when in the reality writing a BFF, improving the architecture and leaner APIs would be a more productive solution. We tried to do that with GraphQL, http2,... And arguably failed. Until we can properly evolve web standards we won't be able to fix the main issue. Novel fra…
Doesn't that depend on what you mean by "shave ms loading a page"? If you're optimizing for time to first render, or time to visually complete, then you need to render the page using as little logic as possible - sending an empty skeleton that then gets hydrated with user data over APIs is fastest for a user's perception of loading speed. If you want to speed up time to first input or time to interactive you need to…
It’s only fastest to get the loading skeleton onto the page.
My personal experience with basically any site that has to go through this 2-stage loading exercise is that:
- content may or may not load properly.
- I will probably be waiting well over 30 seconds for the actually-useful-content.
- when it does all load, it _will_ be laggy and glitchy. Navigation won’t work properly. The site may self-initiate a reload, button clicks are…50/50 success rate for “did it register, or is it just heinously slow”.
I’d honestly give up a lot of fanciness just to have “sites that work _reasonably_” back.
I have a filter I wrote that just reformats JSON into line-delimited text that can be processed immediately by line-oriented UNIX utilities. No waiting.
"The client can't do anything with JSON until the server sends the last byte."
"Would you call [JSON] good engineering?"
I would not call it "engineering". I would call it design.
IMO, djb's netstrings^1 is better design. It inspired similar designs such as bencode.^2
1. https://cr.yp.to/proto/netstrings.txt (1997)
2. https://wiki.theory.org/BitTorrentSpecification (2001)
"And yet [JSON's] the status quo-that's how 99.9999%^* of apps send and process JSON."
Perhaps "good" does not necessarily correlate with status quo and popularity.
Also, it is worth considering that JSON was created for certain popular www browsers. It could piggyback on the popularity of that software.
Also, not a single note about error handling?
There is already a common practice around streaming JSON content. One JSON document per line. This also breaks JSON (removal of newline whitespace), but the resulting documents are backwards compatible (a JSON parser can read them).
Here's a simpler protocol:
Upon connecting, the first line sent by the server is a JavaScript function that accepts 2 nullable parameters (a, b) followed by two new lines. All the remaining lines are complete JSON documents, one per line.
The consuming end should read the JavaScript function followed by two new lines and execute it once passing a=null, b=null.
If that succeeds, it stores the return value and moves to the next line. Upon reading a complete JSON, it executes the function passing a=previousReturn, b=newDocument. Do this for every line consumed.
The server can indicate the end of a stream by sending an extra new line after a document. It can reuse the socket (send another function, indicating new streamed content).
Any line that is not a JavaScript function, JSON document or empty is considered an error. When one is found by the consuming end, it should read at most 1024 bytes from the server socket and close the connection.
--
TL;DR just send one JSON per line and agree on a reduce function between the producer and consumer of objects.
Earlier quoted context omitted.
The best part about someone else's writing is you can just ignore it.
And an important part about publishing your own thoughts to a broader audience is evaluating feedback from that audience rather than dismissing it :)
Earlier quoted context omitted.
What do you mean by technical debt here?
Everywhere I worked with GraphQL it was always a pain for the backend team to keep the graphql server updated and also a pain to use in the frontend, simple REST apis or JSON-RPC are much better.