Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Progressive JSON

overreacted.io

Post reply on HN
Print view View original

231–240 of 244 posts

Re: Progressive JSON

#231

Post by chamomeal »

jatins wrote: post #77

I have seen Dan's "2 computers" talk and read some of his recent posts trying to explore RSC and their benefits. Dan is one of the best explainers in React ecosystem but IMO if one has to work this hard to sell/explain a tech there's 2 possibilities 1/ there is no real need of tech 2/ it's a flawed abstraction #2 seems somewhat true because most frontend devs I know still don't "get" RSC. Vercel has been aggressively…

Tangent: next.js is pretty amazing but it’s still surprising to me that it’s become to default way to write react. I just don’t enjoy writing next.js apps even though typescript is my absolute favorite language, and I generally love react as well.

Re: Progressive JSON

#232

Post by rictic »

jumploops wrote: post #220
rictic wrote: post #182

If you've got some client side code and want to parse and render JSON progressively, try out jsonriver: https://github.com/rictic/jsonriver Very simple API, takes a stream of string chunks and returns a stream of increasingly complete values. Helpful for parsing large JSON, and JSON being emitted by LLMs. Extensively tested and performance optimized. Guaranteed that the final value emitted is identical to passing the…

What’s the benefit of `jsonriver` over one of the myriad of “best effort” parsers[0][1][2] in a try/catch loop while streaming? [0] https://github.com/beenotung/best-effort-json-parser [1] https://github.com/unjs/destr [2] https://www.npmjs.com/package/json-parse-even-better-errors

Good question. jsonriver is well optimized, exhaustively tested (tens of thousands of test cases), and provides a number of potentially useful invariants[0] about its parsing.

jsonriver's performance comes primarily from simplicity, and doing as little work per character as we can. Repeatedly reparsing from scratch on the other hand gets expensive quick, your parse time is quadratic in the length of the string to parse.

[0] https://github.com/rictic/jsonriver?tab=readme-ov-file#invar...

Re: Progressive JSON

#233

Post by yawaramin »

jlokier wrote: post #190
yawaramin wrote: post #167

Earlier quoted context omitted.

When you have an HTTP/2 connection already open a 'round-trip' is not really a gigantic concern performance-wise. And it gives the client application complete control and ver what nested parts it wants to get and in what order. Remember that the article said it's up to the server what order to stream the parts? That might not necessarily be a good idea on the client side though. It would probably be better for the cl…

Round trips for parallel requests work fine over HTTP/2. (As long as there aren't vast numbers of tiny requests, for example every cell in a spreadsheet). However, sequentially-dependent requests are about as slow with HTTP/2 as HTTP/1.1. For example, if your client side, after loading the page, requests data to fill a form component, and then that data indicates a map location, so your client side requests a map ima…

> When round trip times are on the order of 1 second or more (as they often are for me on mobile)

For an already-open HTTP/2 connection? Or for a new connection for each request?

Re: Progressive JSON

#234

Post by owebmaster »

cluckindan wrote: post #210
owebmaster wrote: post #160

Earlier quoted context omitted.

Everywhere I worked with GraphQL it was always a pain for the backend team to keep the graphql server updated and also a pain to use in the frontend, simple REST apis or JSON-RPC are much better.

Interesting. Why did you not have these pains with other tech? Team unfamiliar with GraphQL?

> Why did you not have these pains with other tech?

You don't need a new layer between database -> backend -> frontend

So GraphQL became a backend-for-frontend layer that needs maintaince. The team knowing GraphQL or not is not what causes this, but definitely makes it worse as they use the tool, which is quite complex, wrongly.

NextJS is the "evolution" (or more like growth? as a tumor) of this backend-for-frontend approach, glued with the mentioned RSC in this post. Great recipe to fight accidental complexity all days and nights.

Re: Progressive JSON

#235

Post by igouy »

dominicrose wrote: post #214
krzat wrote: post #86

Earlier quoted context omitted.

Am I the only person that dislikes progressive loading? Especially if it involves content jumping around. And the most annoying antipattern is showing empty state UI during loading phase.

Smalltalk UIs used to work with only one CPU thread. Any action from the user would freeze the whole UI while it was working, but the positive aspect of that is that it was very predictable and bug free. That's helpful since Smalltalk is OOP. Since React is functional programming it works well with parallelization so there is room for experiments. > Especially if it involves content jumping around. I remember this fr…

> Smalltalk UIs used to work with only one CPU thread. Any action from the user would freeze the whole UI while it was working …

If that happened maybe a programmer messed-up the green threads!

"The Smalltalk-80 system provides support for multiple independent processes with three classes named Process, ProcessorScheduler, and Semaphore. "

p251 "Smalltalk-80 The Language and it's Implementation"

https://rmod-files.lille.inria.fr/FreeBooks/BlueBook/Blueboo...

Re: Progressive JSON

#236

Post by jumploops »

rictic wrote: post #232
jumploops wrote: post #220

Earlier quoted context omitted.

What’s the benefit of `jsonriver` over one of the myriad of “best effort” parsers[0][1][2] in a try/catch loop while streaming? [0] https://github.com/beenotung/best-effort-json-parser [1] https://github.com/unjs/destr [2] https://www.npmjs.com/package/json-parse-even-better-errors

Good question. jsonriver is well optimized, exhaustively tested (tens of thousands of test cases), and provides a number of potentially useful invariants[0] about its parsing. jsonriver's performance comes primarily from simplicity, and doing as little work per character as we can. Repeatedly reparsing from scratch on the other hand gets expensive quick, your parse time is quadratic in the length of the string to par…

I've been looking for a better solution for awhile now (if you couldn't tell), and will definitely try out jsonriver for our use-case. Thanks!

Re: Progressive JSON

#237

Post by rabiescow »

I think the main issue with this goes against the fundamental vore principles of a json object has no internal order. If it was an ordered object it would be very different and maybe feasible. But you would have to change the complete standard if JSON. It makes no sense speaking of an object without order as if it has order in terms of "header" etc...

Re: Progressive JSON

#238

Post by jsnelgro »

I feel like this is a great practical example showcasing the value of static types and designing your data model up front. If you know the structure of the object, you can put in placeholders while the real thing loads in. Then you get to choose how it loads in. Ideally you can stream the patches via serialized actions. This is basically Redux/Elm in a nutshell where the action objects can come from events sent via SSE/websockets/polling/etc.

Reducing a change log is about as stateless, functional, and elegant as it gets. I love these sorts of designs but reality seems to complicate them in unexpected ways unfortunately. Still worth striving for though!

Re: Progressive JSON

#239

Post by cluckindan »

owebmaster wrote: post #234
cluckindan wrote: post #210

Earlier quoted context omitted.

Interesting. Why did you not have these pains with other tech? Team unfamiliar with GraphQL?

> Why did you not have these pains with other tech? You don't need a new layer between database -> backend -> frontend So GraphQL became a backend-for-frontend layer that needs maintaince. The team knowing GraphQL or not is not what causes this, but definitely makes it worse as they use the tool, which is quite complex, wrongly. NextJS is the "evolution" (or more like growth? as a tumor) of this backend-for-frontend…

So because they were not generating resolvers from the schema, a maintenance burden was created. You CAN create resolvers manually but that’s really not the best idea when automated approaches exist. That’s poor engineering, not bad technology.

GraphQL federation is where the thing really shines.

Re: Progressive JSON

#240

Post by OrangeMusic »

GraphQL has a defer mode that looks like this. You receive the fast pieces first, and the slower pieces of the json come later, with the path to where they should be attached to.
Post reply on HN