JSON River – Parse JSON incrementally as it streams in • Hacker News Board

Re: JSON River – Parse JSON incrementally as it streams in

#41

Post by benatkin » Mon, Oct 13, 2025, 6:20 PM UTC

alganet wrote: post #20

benatkin wrote: post #5

Earlier quoted context omitted.

I think this is a lot like etree in python's streaming approach for XML, but with a simpler API, and incremental text parsing. With etree in python, you can access the incomplete tree data and not have to worry about events. So it's missing the SAX API part of a SAX approach, but is built like some real world libraries that use the SAX approach, which end up having a hybrid of events and trees.

It seems to be convenient for some cases. A large object with many keys, for example. I don't see it as particularly convenient if I want to stream a large array of small independent objects and read each one of them once, then discard it. The incremental parsed array would get bigger and bigger, eventually containing all the objects I wanted to discard. I would also need to move my array pointer to the last element…

This is more versatile than it seems at first glance. Under invariants, it shows that you have arrays/objects only being mutated, so you have stable references. You could use a WeakSet to observe new children of an item coming in. You also may not even need manage this directly - you could debounce and just re-render a UI component by returning a modified virtual DOM. Or if you had a visualization in d3, it would automatically notice which ones are new.

Re: JSON River – Parse JSON incrementally as it streams in

#42

Post by rictic » Mon, Oct 13, 2025, 6:26 PM UTC

rokkamokka wrote: post #39

syx wrote: post #17

For those wondering about the use case, this is very useful when enabling streaming for structured output in LLM responses, such as JSON responses. For my local Raspberry Pi agent I needed something performant, I've been using streaming-json-js [1], but development appears to have been a bit dormant over the past year. I'll definitely take a look at your jsonriver and see how it compares! [1] https://github.com/karmi…

For LLMs I recommend just doing NDJSON, that is, newline delimited json. It's much simpler to implement

Do any LLMs support constrained generation of newline delimited json? Or have you found that they're generally reliable enough that you don't need to do constrained sampling?

Re: JSON River – Parse JSON incrementally as it streams in

#43

Post by trevor-e » Mon, Oct 13, 2025, 6:30 PM UTC

simonw wrote: post #13

holdenc137 wrote: post #11

I don't get it (and I'd call this cumulative not incremental) Why not at least wait until the key is complete - what's the use in a partial key?

If you're building a UI that renders output from a streaming LLM you might get back something which looks like this: {"role": "assistant", "text": "Here's that Python code you aske Incomplete parsing with incomplete strings is still useful in order to render that to your end user while it's still streaming in.

In this example the value is incomplete, not the key.

Re: JSON River – Parse JSON incrementally as it streams in

#44

Post by keleftheriou » Mon, Oct 13, 2025, 6:35 PM UTC

Thanks for sharing!

Roughly how does it compare with https://github.com/promplate/partial-json-parser-js ?

Re: JSON River – Parse JSON incrementally as it streams in

#45

Post by stronglikedan » Mon, Oct 13, 2025, 6:35 PM UTC

cozzyd wrote: post #16

simonw wrote: post #13

Earlier quoted context omitted.

If you're building a UI that renders output from a streaming LLM you might get back something which looks like this: {"role": "assistant", "text": "Here's that Python code you aske Incomplete parsing with incomplete strings is still useful in order to render that to your end user while it's still streaming in.

incomplete strings could be fun in certain cases {"cleanup_cmd":"rm -rf /home/foo/.tmp" }

If any part of that value actually made it, unchecked, to execution, then you have bigger problems than partial JSON keys/values.

Re: JSON River – Parse JSON incrementally as it streams in

#46

Post by quotemstr » Mon, Oct 13, 2025, 6:58 PM UTC

Awesome. You know what would be EVEN COOLER?

Given a schema and a JSON message prefix, parse the complete message but substitute missing field values with Promise objects. Likewise, represent lists as lazy sequences. Add a pubsub system.

Re: JSON River – Parse JSON incrementally as it streams in

#47

Post by chrchr » Mon, Oct 13, 2025, 7:01 PM UTC

I did something like this for Python [1]. The application I worked on at the time had a feature allowing users to import and export their data as a JSON document, and users often had enough data to make this cumbersome, especially with serialization and deserialization overhead. My implementation can also generate JSON documents as they stream out, from Python generators. The incremental JSON parsing was a little difficult to use, but incremental generation was an immediate win. We generated JSON documents from database results row-by-row and streamed the output to the web server, never producing the entire document in memory.

[1] https://github.com/chrchr/flojay

Re: JSON River – Parse JSON incrementally as it streams in

#48

Post by alganet » Mon, Oct 13, 2025, 7:08 PM UTC

benatkin wrote: post #41

alganet wrote: post #20

Earlier quoted context omitted.

It seems to be convenient for some cases. A large object with many keys, for example. I don't see it as particularly convenient if I want to stream a large array of small independent objects and read each one of them once, then discard it. The incremental parsed array would get bigger and bigger, eventually containing all the objects I wanted to discard. I would also need to move my array pointer to the last element…

This is more versatile than it seems at first glance. Under invariants, it shows that you have arrays/objects only being mutated, so you have stable references. You could use a WeakSet to observe new children of an item coming in. You also may not even need manage this directly - you could debounce and just re-render a UI component by returning a modified virtual DOM. Or if you had a visualization in d3, it would aut…

It does sound very practical indeed.

Re: JSON River – Parse JSON incrementally as it streams in

#49

Post by xg15 » Mon, Oct 13, 2025, 7:17 PM UTC

holdenc137 wrote: post #11

I don't get it (and I'd call this cumulative not incremental) Why not at least wait until the key is complete - what's the use in a partial key?

Doesn't it do exactly that?

> As a consequence of 1 and 5, we only add a property to an object once we have the entire key and enough of the value to know that value's type.

Re: JSON River – Parse JSON incrementally as it streams in

#50

Post by xg15 » Mon, Oct 13, 2025, 7:19 PM UTC

seanalltogether wrote: post #10

Maybe I'm wrong but it seems like you would only want to parse partial values for objects and arrays, but not strings or numbers. Objects and arrays can be unbounded so it makes sense to process what you can, when you can, whereas a string or number usually is not.

There is json that has very long string literals. Usually, it's either long-ish text or HTML content, or base64-encoded binary data.

So I'd definitely count strings as "unbounded" as well.