Progressive JSON

[dead]

We encountered this problem when converting audio only LLM applications to visual + audio. The visuals would increase latency by a lot since they need to be parsed completely before displaying, whereas you can just play audio token by token and wait for the LLM to generate the next one while audio is playing.

jsonl is json objects separated by endline characters. Used in Bedrock batch processing.

Very cool point, and it applies to any tree data in general.

I like to represent tree data with parent, type, and data vectors along with a string table, so everything else is just small integers.

Sending the string table and type info as upfront headers, we can follow with a stream of parent and data vector chunks, batched N nodes at a time. Tye depth- or breadth-first streaming becomes a choice of ordering on the vectors.

I'm gonna have to play around with this! Might be a general way to get snappier load time UX on network bound applications.

I feel like in an ideal world, this would start in the DB: your query referencing objects and in what order to return them (so not just a bunch of wide rows, nor multiple separate queries) and as the data arrives, the back end could then pass it on to the client.