If you get partial string values that are replaced with longer string values as they are streamed in, how do you know when the value is finished being read and is safe to use?
JSON River – Parse JSON incrementally as it streams in
81–90 of 101 posts
Re: JSON River – Parse JSON incrementally as it streams in
#82Nice. You should pair it with Immer + React for a nice UI demo. This will hype hard if the value is properly demonstrated.
Re: JSON River – Parse JSON incrementally as it streams in
#83Earlier quoted context omitted.
If you're building a UI that renders output from a streaming LLM you might get back something which looks like this: {"role": "assistant", "text": "Here's that Python code you aske Incomplete parsing with incomplete strings is still useful in order to render that to your end user while it's still streaming in.
incomplete strings could be fun in certain cases {"cleanup_cmd":"rm -rf /home/foo/.tmp" }
Only because you have access to the incomplete value doesn't mean you should treat it like the complete one...
Re: JSON River – Parse JSON incrementally as it streams in
#84Earlier quoted context omitted.
Suggestion: make it clearer in the readme what happens with malformed input. I can imagine it being useful to have a made where you never emit strings until they are final, also. I don't entirely understand why strings are emitted incrementally but numbers aren't.
Seems useful to me in the context of something like a progressively rendered UI. A large block of text appearing a few characters at a time would be fine, but a number that represents something like a display metric (say, a position, or font-size) going from 0 to 0.5 or from 1 to 1000, would result in goofy gyrations on-screen that don't make any sense. Or imagine if it was just fields in the app's data. Name: John S…
If you're updating the UI every time you receive a single character from this library, you've got bigger problems than font size.
Re: JSON River – Parse JSON incrementally as it streams in
#85Earlier quoted context omitted.
Seems useful to me in the context of something like a progressively rendered UI. A large block of text appearing a few characters at a time would be fine, but a number that represents something like a display metric (say, a position, or font-size) going from 0 to 0.5 or from 1 to 1000, would result in goofy gyrations on-screen that don't make any sense. Or imagine if it was just fields in the app's data. Name: John S…
If you're updating the UI every time you receive a single character from this library, you've got bigger problems than font size.
hmm this makes sense for LLM usage
(but for other uses - nope)
Re: JSON River – Parse JSON incrementally as it streams in
#86Earlier quoted context omitted.
Don't you need to wait for some kind of delimiter (like ",", "]", "}", newline, EOF) before parsing something else than a string?
Only for numbers! Strings, objects, arrays, true, false, and null all have an unambiguous ending.
but you don't do this for strings either, as shown in the examples - partial strings are pushed even though they're not yet ended:
{"name": "Ale"}Re: JSON River – Parse JSON incrementally as it streams in
#87Interesting approach. I would expect an object JSON stream to be more like a SAX parser though. It's familiar, fast and simple. Any thougts on not chosing the SAX approach?
A nice thing about the SAX approach is it lets you layer other APIs on top too. I did something like that in BFJ:
Re: JSON River – Parse JSON incrementally as it streams in
#88Earlier quoted context omitted.
incomplete strings could be fun in certain cases {"cleanup_cmd":"rm -rf /home/foo/.tmp" }
Incremental JSON parsing is key for LLM apps, but safe progressive UIs also need to track incompleteness and per-chunk diffs. LangDiff [1] would help with that. [1]: https://github.com/globalaiplatform/langdiff/tree/main/ts
Why not just chunk the json packets instead?
Re: JSON River – Parse JSON incrementally as it streams in
#89If anyone needs to do this in Python I've had success with both ijson and jiter - notes here: https://til.simonwillison.net/json/ijson-stream and https://simonwillison.net/2024/Sep/22/jiter/
+1 for ijson. I wrote some pretty fast and lightweight parsers a while back, using ijson's basic stream. Never heard of jiter, thanks for the posts!
Re: JSON River – Parse JSON incrementally as it streams in
#90Finally, a way to make JSON as fluid as our UI needs to be.