Progressive JSON

The GraphQL payload has redundant copies of the objects within. Relay has an experimental (dead) exploration of normalising data on the server[1] in order to avoid that transport cost.

For @defer, the transport is a stream[2] where subsequent payloads are tracked ("pending") and as those payloads arrive they are "patched" into the existing data at the appropriate path.

Both of these are effectively implementations of this Progressive JSON concept.

I can picture a future where both the JSX and object payloads are delivered progressively over the wire in the same stream, and hydrate React and Relay stores alike. That could potentially simplify and eliminate the need for Relay to synchronise it's own updates with React. It would also potentially be a suitable compile target for decoupling, React taking ownership of a normalised reactive object store (with state garbage collection which is a hard problem for external stores to solve) with external tools like Relay and TanStack Query providing the type-generation & fetching implementation details.

Using `new Promise()` in the client-side store would also mean you could potentially shrink the component API to this. `useFragment(fragment IssueAssignee on Issue { assignee { name }}, issue)` could instead be `const assignee = use(issue.assignee)` and would suspend (or not) appropriately.

[1]: https://github.com/facebook/relay/blob/main/packages/relay-r...

[2]: https://github.com/graphql/graphql-wg/blob/main/rfcs/DeferSt...

Or here is a different approach:

We acknowledge that streaming data is not a problem that JSON was intended, or designed, to solve, and ... not do that.

If an application has a usecase that necessitates sending truly gigantic JSON objects across the wire, to the point where such a scheme seems like a good idea, the much better question to ask is "why is my application sending ginormeous JSON objects again?"

And the answer is usually this:

Fat clients using bloated libraries and ignoring REST, trying to shoehorn JSON into a "one size fits all" solution, sending first data, then data + metadata, then data + metadata + metadata describing the interface, because finally we came full circle and re-invented a really really bad version of REST that requires several MB of minified JS for the browser to use.

Again, the solution is not to change JSON, the solution is to not do the thing that causes the problem. Most pages don't need a giant SPA framework.

Seems like 95% of businesses are not willing to pay the web dev who created the problem in the first place to also fix the problem, and instead want more features released last week.

The number of websites needlessly forced into being SPAs without working navigation like back and forth buttons is appalling.

It is a terrible idea organizationally. It puts backend devs at the whims of often hype train and CV driven development of frontend devs. What often happens is, that complexity is moved from the frontend to the backend. But that complexity is not necessarily implicit, but often self inflicted accidental complexity by choices in frontend. The backend API should facilitate getting the required data to render pages and perform required operations to interact with that data. Everything else is optimization that one may or may not need.

Oh I see. Maybe a way to restate this is "how do I communicate the costs of data to the client", that is the cost of returning top-level user data is let's just say 1; the cost of returning the last 10 comments is 2, and the cost of returning older comments is 2000. Because otherwise pushing that set of decisions back to the server doesn't exactly solve it, it just means you now actually can make that decision server side, even though you're still waiting a long time on comment 11 no matter what.

Re your "JSX Over The Wire" post, I think we've gone totally around the bend. A piece of code that takes 0 or more responses from a data backend and returns some kind of HTML is a web service. Like, that's CGI, that's PHP, that's Rails, Node, Django, whatever. If the argument here is "the browser should have some kind of state tracking/reactivity built in, and until that day we have a shim like jQuery or the old school thin React or the new school htmx" then OK, but this is so, so much engineering to elide `onclick` et al.

---

I kind of worry that we've spent way, way too much time in these weeds. There's millions and millions of lines of React out there, and certainly the majority of it is "stitch the responses of these N API calls together into a view/document, maybe poll them for updates from time to time", to the degree that AI just does it now. If it's so predictable that a couple of video cards can do it in a few seconds, why have we spent gazillions of engineering years polishing this?