Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JSON River – Parse JSON incrementally as it streams in

github.com

Post reply on HN
Print view View original

61–70 of 101 posts

Re: JSON River – Parse JSON incrementally as it streams in

#61

Post by stevage »

rokkamokka wrote: post #39
syx wrote: post #17

For those wondering about the use case, this is very useful when enabling streaming for structured output in LLM responses, such as JSON responses. For my local Raspberry Pi agent I needed something performant, I've been using streaming-json-js [1], but development appears to have been a bit dormant over the past year. I'll definitely take a look at your jsonriver and see how it compares! [1] https://github.com/karmi…

For LLMs I recommend just doing NDJSON, that is, newline delimited json. It's much simpler to implement

I love NDJSON in general. I use it a lot for spatial data processing (GDAL calls it GeoJsonSeq).

Re: JSON River – Parse JSON incrementally as it streams in

#63

Post by o11c »

rictic wrote: post #58
o11c wrote: post #56

Earlier quoted context omitted.

Mostly unrelated, but does anyone know how are you supposed to make path-less module specifiers work for Node if you are not using npm but rather system-installed JS packages (Debian etc. install node-* packages into /usr/share/nodejs/)? With `require` it just works, but with `import` it errors and suggests passing the absolute path (even though it clearly knows what path ...). For some reason everybody in the JS wor…

Try import maps, something like: { "imports": { "express": "/usr/share/nodejs/express/index.js", "another-module": "/usr/share/nodejs/another-module/index.js" } } Then run node like: `node --import-map=./import-map.json app.js` The Debian approach of having global versions of libraries seems like it's solving a different problem than the ones I have. I want each application to track and version its own dependencies,…

I have a simpler solution to the latter problem: if upgrading a dependency package breaks anything (barring multi-year deprecation, limited-time experimental previews, etc.), I blacklist it and never install that package ever again. After all, they are clearly lacking on either their testing infrastructure or their development guidelines.

It's amazing how much the quality of installed software improves when you do this. Something our industry desperately needs.

Re: JSON River – Parse JSON incrementally as it streams in

#64

Post by xp84 »

stevage wrote: post #60
rictic wrote: post #35

Hi HN! Didn't expect this to be on the front page today! I should really release all the optimizations that've been landing lately, the version on github is about twice as fast as what's released on npm. I wrote it when I was doing prototyping on doing streaming rendering of UIs defined by JSON generated by LLMs. Using constrained generation you can essentially hand the model a JSON serializable type, and it will alw…

Suggestion: make it clearer in the readme what happens with malformed input. I can imagine it being useful to have a made where you never emit strings until they are final, also. I don't entirely understand why strings are emitted incrementally but numbers aren't.

Seems useful to me in the context of something like a progressively rendered UI. A large block of text appearing a few characters at a time would be fine, but a number that represents something like a display metric (say, a position, or font-size) going from 0 to 0.5 or from 1 to 1000, would result in goofy gyrations on-screen that don't make any sense. Or imagine if it was just fields in the app's data.

Name: John Smith. Birth Year: A.D. 1 [Customer is a Senior: 2,024 years old]

Name: John Smith. Birth year: A.D. 19 [Customer is a Senior: 2,006 years old]

Name: John Smith. Birth year: A.D. 199 [Customer is a Senior: 1,826 years old]

Name: John Smith. Birth year: 1997

Re: JSON River – Parse JSON incrementally as it streams in

#65

Post by rictic »

stevage wrote: post #60
rictic wrote: post #35

Hi HN! Didn't expect this to be on the front page today! I should really release all the optimizations that've been landing lately, the version on github is about twice as fast as what's released on npm. I wrote it when I was doing prototyping on doing streaming rendering of UIs defined by JSON generated by LLMs. Using constrained generation you can essentially hand the model a JSON serializable type, and it will alw…

Suggestion: make it clearer in the readme what happens with malformed input. I can imagine it being useful to have a made where you never emit strings until they are final, also. I don't entirely understand why strings are emitted incrementally but numbers aren't.

Good feedback! Just updated the README with the following:

> The parse function also matches JSON.parse's behavior for invalid input. If the input stream cannot be parsed as the start of a valid JSON document, then parsing halts and an error is thrown. More precisely, the promise returned by the next method on the AsyncIterable rejects with an Error. Likewise if the input stream closes prematurely.

As for why strings are emitted incrementally, it's just that I was often dealing with long strings produced slowly by LLMs. JSON encoded numbers can be big in theory, but there's no practical reason to do so as almost everyone decodes them as 64bit floats.

Re: JSON River – Parse JSON incrementally as it streams in

#66

Post by rictic »

rictic wrote: post #35

Hi HN! Didn't expect this to be on the front page today! I should really release all the optimizations that've been landing lately, the version on github is about twice as fast as what's released on npm. I wrote it when I was doing prototyping on doing streaming rendering of UIs defined by JSON generated by LLMs. Using constrained generation you can essentially hand the model a JSON serializable type, and it will alw…

I've just published v1.0.1. It's about 2x faster, and should have no other observable changes. The speedup is mainly from avoiding allocation and string slicing as much as possible, plus an internal refactor to bind the parser and tokenizer more tightly together.

Previously the parser would get an array of tokens each time it pushed data into the tokenizer. This was easy to write, but it meant we needed to allocate token objects. Now the tokenizer has a reference to the parser and calls token-specific methods directly on it. Since most of the tokens carry no data, this keeps us from jumping all over the heap so much. If we were parsing a more complicated language this might become a huge pain in the butt, but JSON is simple enough, and the test suite is exhaustive enough, that we can afford a little nightmare spaghetti if it improves on speed.

Re: JSON River – Parse JSON incrementally as it streams in

#67

Post by kondro »

jlundberg wrote: post #59

I really like just encoding each object as JSON and then concatinating them with a new line between. Allows parsing and streaming without any special libraries and allow for an unlimited amount of data (with objects being reasonably sized). Usually gives these files the .jsonlines suffix when stored on disk. Allows for batch process without requiring huge amounts of memory.

Me too, and it's quite a common technique.

https://en.wikipedia.org/wiki/JSON_streaming

Re: JSON River – Parse JSON incrementally as it streams in

#68

Post by ww520 »

This is nice to parse incomplete JSON as they come in.

I did something similar for streaming but built it with a streaming protocol at the frame level wrapping the JSON messages [1]. The streaming protocol has support for both the LF based scheme and the HTTP Content-Length header based scheme. It's for supporting MCP and LSP.

[1] https://github.com/williamw520/zigjr/?tab=readme-ov-file#str...

Re: JSON River – Parse JSON incrementally as it streams in

#69

Post by 0x6c6f6c »

jlundberg wrote: post #59

I really like just encoding each object as JSON and then concatinating them with a new line between. Allows parsing and streaming without any special libraries and allow for an unlimited amount of data (with objects being reasonably sized). Usually gives these files the .jsonlines suffix when stored on disk. Allows for batch process without requiring huge amounts of memory.

Based on this thread that's called NDJSON

Newline Delimited JSON

TIL

Re: JSON River – Parse JSON incrementally as it streams in

#70

Post by 0x6c6f6c »

xg15 wrote: post #49
holdenc137 wrote: post #11

I don't get it (and I'd call this cumulative not incremental) Why not at least wait until the key is complete - what's the use in a partial key?

Doesn't it do exactly that? > As a consequence of 1 and 5, we only add a property to an object once we have the entire key and enough of the value to know that value's type.

Their example in the README is extremely misleading then. It indicates your stream output is

name: A name: Al name: Ale name: Alex

Which would suggest you are getting unfinished strings out in the stream.

Post reply on HN