Live data from Hacker News

Progressive JSON

overreacted.io

121–130 of 244 posts

Re: Progressive JSON

#121
I am not sure the wheel can be rediscovered many more times but definitely check out Kris's work from around 2010-2012 around q-connection and streaming/rpc of chunks of data. Promises themselves have roots in this and there are better formats for this.

Check our mark miller's E stuff and thesis - this stuff goes all the way back to the 80s.

Re: Progressive JSON

#122
post #120

I am not sure the wheel can be rediscovered many more times but definitely check out Kris's work from around 2010-2012 around q-connection and streaming/rpc of chunks of data. Promises themselves have roots in this and there are better formats for this. Check our mark miller's E stuff and thesis - this stuff goes all the way back to the 80s.

@dang - I hit "reply" once (I am sure of that) and I see my (identical) comment twice in the UI. Not sure what sort of logging/tracing/instrumentation you have in place - I am not delete'ing this so you have a chance to investigate but if that's not useful by all means feel free to do so.

Re: Progressive JSON

#123

Earlier quoted context omitted.

Json is just a packing format that does have that limitation. If you control the source and the destination, could you possibly use a format that supports streaming better like Protobuf?

What stops you from parsing tokens from a stream like a SAX parser for JSON? [ ["aaa", "bbb"], { "name", "foo" } ] Start array Start array String aaa String bbb End array Start object Key name String foo End object End array

Nothing, really, but I don’t have the bandwidth to write JSAX. I wonder why it hasn’t already been done by someone more qualified than I am. I suspect that I’d find out, if I started doing it.

You can do that, in a specialized manner, with PHP, and Streaming JSON Parser[0]. I use that, in one of my server projects[1]. It claims to be JSON SAX, but I haven’t really done an objective comparison, and it specializes for file types. It works for my purposes.

[0] https://github.com/salsify/jsonstreamingparser

[1] https://github.com/LittleGreenViper/LGV_TZ_Lookup/blob/main/...

Re: Progressive JSON

#124

Seems like some people here are taking this post literally, as in the author (Dan Abramov) is proposing a format called Progressive JSON — it is not. This is more of a post on explaining the idea of React Server Components where they represent component trees as javascript objects, and then stream them on the wire with a format similar to the blog post (with similar features, though AFAIK it’s bundler/framework speci…

[deleted]

Re: Progressive JSON

#125
post #121

I am not sure the wheel can be rediscovered many more times but definitely check out Kris's work from around 2010-2012 around q-connection and streaming/rpc of chunks of data. Promises themselves have roots in this and there are better formats for this. Check our mark miller's E stuff and thesis - this stuff goes all the way back to the 80s.

Not to disrespect Dan here, each discovery is impressive on its own but I wish we had a better way to preserve this sort of knowledge.

Re: Progressive JSON

#126
post #125
post #121

I am not sure the wheel can be rediscovered many more times but definitely check out Kris's work from around 2010-2012 around q-connection and streaming/rpc of chunks of data. Promises themselves have roots in this and there are better formats for this. Check our mark miller's E stuff and thesis - this stuff goes all the way back to the 80s.

Not to disrespect Dan here, each discovery is impressive on its own but I wish we had a better way to preserve this sort of knowledge.

> I wish we had a better way to preserve this sort of knowledge.

It's called "being part of the curriculum" and apparently the general insights involved aren't, so far.

Re: Progressive JSON

#127
I don't mean to be dismissive, but haven't we solved this by using different endpoints? There's so many virtues: you avoid head of line blocking; you can implement better filtering (eg "sort comments by most popular"); you can do live updates; you can iterate on the performance of individual objects (caching, etc).

---

I broadly see this as the fallout of using a document system as an application platform. Everything wants to treat a page like a doc, but applications don't usually work that way, so lots of code and infra gets built to massage the one into the other.

Re: Progressive JSON

#128
post #71

The thing I have seem in performance is people trying to shave ms loading a page, while they fetch several mbs and do complex operations in the FE, when in the reality writing a BFF, improving the architecture and leaner APIs would be a more productive solution. We tried to do that with GraphQL, http2,... And arguably failed. Until we can properly evolve web standards we won't be able to fix the main issue. Novel fra…

Too many acronyms, what's FE, BFF?

Front end and a backend for a frontend. In which you generally design apis specific for a page by aggregating multiple other apis, caching, transforming etc.

Re: Progressive JSON

#129
post #96
post #86

Earlier quoted context omitted.

Am I the only person that dislikes progressive loading? Especially if it involves content jumping around. And the most annoying antipattern is showing empty state UI during loading phase.

alternative is to stare at blank page without any indication that something is happening

It’s better than moving the link or button as I’m clicking it.

Re: Progressive JSON

#130
post #71

The thing I have seem in performance is people trying to shave ms loading a page, while they fetch several mbs and do complex operations in the FE, when in the reality writing a BFF, improving the architecture and leaner APIs would be a more productive solution. We tried to do that with GraphQL, http2,... And arguably failed. Until we can properly evolve web standards we won't be able to fix the main issue. Novel fra…

Too many acronyms, what's FE, BFF?

I was asking the same questions.

- FE is short for the Front End (UI)

- BFF is short for Backend For Frontend

Post reply on HN