Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Progressive JSON

overreacted.io

Post reply on HN
Print view View original

141–150 of 244 posts

Re: Progressive JSON

#141

Post by atombender »

You could stream incrementally like this without explicitly demarcating the "holes". You can simply send the unfinished JSON (with empty arrays as the holes), then compute the next iteration and send a delta, then compute the next and send a delta, and so on.

A good delta format is Mendoza [1] (full disclosure: I work at Sanity where we developed this), which has Go and JS/TypeScript [2] implementations. It expresses diffs and patches as very compact operations.

Another way is to use binary digging. For example, zstd has some nifty built-in support for diffing where you can use the previous version as a dictionary and then produce a diff that can be applied to that version, although we found Mendoza to often be as small as zstd. This approach also requires treating the JSON as bytes and keeping the previous binary snapshot in memory for the next delta, whereas a Mendoza patch can be applied to a JavaScript value, so you only need the deserialized data.

This scheme would force you to compare the new version for what's changed rather than plug in exactly what's changed, but I believe React already needs to do that? Also, I suppose the Mendoza applier could be extended to return a list of keys that were affected by a patch application.

[1] https://github.com/sanity-io/mendoza

[2] https://github.com/sanity-io/mendoza-js

Re: Progressive JSON

#142

Post by Bjartr »

MrJohz wrote: post #112
onion2k wrote: post #79

Earlier quoted context omitted.

Doesn't that depend on what you mean by "shave ms loading a page"? If you're optimizing for time to first render, or time to visually complete, then you need to render the page using as little logic as possible - sending an empty skeleton that then gets hydrated with user data over APIs is fastest for a user's perception of loading speed. If you want to speed up time to first input or time to interactive you need to…

> sending an empty skeleton that then gets hydrated with user data over APIs is fastest for a user's perception of loading speed This is often repeated, but my own experience is the opposite: when I see a bunch of skeleton loaders on a page, I generally expect to be in for a bad experience, because the site is probably going to be slow and janky and cause problems. And the more the of the site is being skeleton-loade…

> the experience should be improving

I think it's more the bounce rate is improving. People may recall a worse experience later, but more will stick around for that experience if they see something happen sooner.

Re: Progressive JSON

#143

Post by __mattya »

atombender wrote: post #141

You could stream incrementally like this without explicitly demarcating the "holes". You can simply send the unfinished JSON (with empty arrays as the holes), then compute the next iteration and send a delta, then compute the next and send a delta, and so on. A good delta format is Mendoza [1] (full disclosure: I work at Sanity where we developed this), which has Go and JS/TypeScript [2] implementations. It expresses…

They want to know where the holes are so that they can show a loading state.

Re: Progressive JSON

#144

Post by andrewingram »

atombender wrote: post #141

You could stream incrementally like this without explicitly demarcating the "holes". You can simply send the unfinished JSON (with empty arrays as the holes), then compute the next iteration and send a delta, then compute the next and send a delta, and so on. A good delta format is Mendoza [1] (full disclosure: I work at Sanity where we developed this), which has Go and JS/TypeScript [2] implementations. It expresses…

For the use case of streaming data for UI, I don’t think empty arrays and nulls are sufficient information. At any moment during the stream, you need the ability to tell what data is pending.

If pending arrays are just returned as empty arrays, how do I know if it’s empty because it’s actually empty, or empty because it’s pending?

GraphQL’s streaming payloads try to get the best of both worlds, at any point in time you have a valid payload according the GraphQL schema - so it’s possible to render some valid UI, but it also communicates what paths contain pending data, and then subsequent payloads act as patches (though not as sophisticated as Mendoza’s).

Re: Progressive JSON

#145

Post by atombender »

__mattya wrote: post #143
atombender wrote: post #141

You could stream incrementally like this without explicitly demarcating the "holes". You can simply send the unfinished JSON (with empty arrays as the holes), then compute the next iteration and send a delta, then compute the next and send a delta, and so on. A good delta format is Mendoza [1] (full disclosure: I work at Sanity where we developed this), which has Go and JS/TypeScript [2] implementations. It expresses…

They want to know where the holes are so that they can show a loading state.

You don't need templating ($1 etc.) for that as long as you can describe the holes somehow, which can be done out-of-band.

If we imagine a streaming protocol of key/value pairs that are either snapshots or deltas:

    event: snapshot
    data: {"topPost":[], "user": {"comments": []}}
    pending: topPosts,user.comments

    event: delta
    data: [17,{"comments":[{"body":"hello world"}]},"user"]
    pending: topPosts

Re: Progressive JSON

#146

Post by gavinray »

owebmaster wrote: post #137
gavinray wrote: post #135

Earlier quoted context omitted.

> React has become a behemoth requiring vendor specific hosting This is one of the silliest things I've read in a while. React is sub-3kB minified + gzip'ed [0], and the grand majority of React apps I've deployed are served as static assets from a fileserver. My blog runs off of Github Pages, for instance. People will always find a way to invent problems for themselves, but this is a silly example. [0] https://bundle…

> This is one of the silliest things I've read in a while. You know that the author of this post is the creator of React and that he's been pushing for RSC/Vercel relentlessly, right? btw reactdom is ~30kb gzipped so React minimal bundle is around 35kb

If you really want to bikeshed over size, you can use Preact which is a genuine 3kB full drop-in for React.

Re: Progressive JSON

#147

Post by sdeframond »

krzat wrote: post #86
goranmoomin wrote: post #29

Seems like some people here are taking this post literally, as in the author (Dan Abramov) is proposing a format called Progressive JSON — it is not. This is more of a post on explaining the idea of React Server Components where they represent component trees as javascript objects, and then stream them on the wire with a format similar to the blog post (with similar features, though AFAIK it’s bundler/framework speci…

Am I the only person that dislikes progressive loading? Especially if it involves content jumping around. And the most annoying antipattern is showing empty state UI during loading phase.

You might be interested in the "remote data" pattern (for lack of a better name)

https://www.haskellpreneur.com/articles/slaying-a-ui-antipat...

Re: Progressive JSON

#148

Post by atombender »

andrewingram wrote: post #144
atombender wrote: post #141

You could stream incrementally like this without explicitly demarcating the "holes". You can simply send the unfinished JSON (with empty arrays as the holes), then compute the next iteration and send a delta, then compute the next and send a delta, and so on. A good delta format is Mendoza [1] (full disclosure: I work at Sanity where we developed this), which has Go and JS/TypeScript [2] implementations. It expresses…

For the use case of streaming data for UI, I don’t think empty arrays and nulls are sufficient information. At any moment during the stream, you need the ability to tell what data is pending. If pending arrays are just returned as empty arrays, how do I know if it’s empty because it’s actually empty, or empty because it’s pending? GraphQL’s streaming payloads try to get the best of both worlds, at any point in time y…

As I commented in https://news.ycombinator.com/item?id=44150238, all you need is a way to express what is pending, which can be done using JSON key paths.

Of course, you could do it in-band, too:

    {"comments": {"state": "pending", "values": []}}
…at the cost of needing your data model to be explicit about it. But this has the benefit of being diffable, of course, so once the data is available, the diff is just the new state and the new values.

Re: Progressive JSON

#149

Post by andrewingram »

danabramov wrote: post #108
jatins wrote: post #77

I have seen Dan's "2 computers" talk and read some of his recent posts trying to explore RSC and their benefits. Dan is one of the best explainers in React ecosystem but IMO if one has to work this hard to sell/explain a tech there's 2 possibilities 1/ there is no real need of tech 2/ it's a flawed abstraction #2 seems somewhat true because most frontend devs I know still don't "get" RSC. Vercel has been aggressively…

You’re wrong about the historical aspects and motivations but I don’t have the energy to argue about it now and will save it for another post. (Vercel isn’t setting React’s direction; rather, they’re the ones who funded person-decades of work under the direction set by the React team.) I’ll just correct the allegation about the Vite — it’s being worked on but the ball is largely in the Vite team’s court because it ca…

I also appreciate that you’re doing these explainers so that people don’t have to go the long way round understand what problems exists that call for certain shapes of solutions — especially when those solutions can feel contrived or complicated.

As someone who’s been building web UI for nearly 30 years (scary…), I’ve generally been fortunate enough that when some framework I use introduces a new feature or pattern, I know what they’re trying to do. But the only reason I know what they’re trying to do is because I’ve spent some amount of time running into the problems they’re solving. The first time I saw GraphQL back in 2015, I “got” it; 10 years later most people using GraphQL don’t really get it because they’ve had it forced upon them or chose it because it was the new shiny thing. Same was true of Suspense, server functions, etc.

Re: Progressive JSON

#150

Post by pas »

metalrain wrote: post #114
jatins wrote: post #77

I have seen Dan's "2 computers" talk and read some of his recent posts trying to explore RSC and their benefits. Dan is one of the best explainers in React ecosystem but IMO if one has to work this hard to sell/explain a tech there's 2 possibilities 1/ there is no real need of tech 2/ it's a flawed abstraction #2 seems somewhat true because most frontend devs I know still don't "get" RSC. Vercel has been aggressively…

While RSC as technology is interesting, I don't think it makes much sense in practice. I don't want to have a fleet of Node/Bun backend servers that have to render complex components. I'd rather have static pages and/or React SPA with Go API server. You get similar result with much smaller resources.

It's convenient for integrating with backends. You can use async/await on the server, no need for hooks (callbacks) for data loading.

It allows for dynamism (user only sees the menus that they have permissions for), you can already show those parts that are already loaded while other parts are still loading.

(And while I prefer the elegance and clean separation of concerns that come with a good REST API, it's definitely more work to maintain both the frontend and the backend for it. Especially in caes where the backend-for-frontend integrates with more backends.)

So it's the new PHP (with ob_flush), good for dashboards and big complex high-traffic webshop-like sites, where you want to spare no effort to be able to present the best options to the dear customer as soon as possible. (And also it should be crawlable, and it should work on even the lowest powered devices.)

Post reply on HN