Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a high performance JSON parser

dave.cheney.net

Post reply on HN
Print view View original

41–50 of 193 posts

Re: Building a high performance JSON parser

#41

Post by crazygringo »

Galanwe wrote: post #5

[flagged]

JSON is often the format of an externally provided data source, and you don't have a choice.

And whatever language you're writing in, you usually want to do what you can to maximize performance. If your JSON input is 500 bytes it probably doesn't matter, but if you're intaking a 5 MB JSON file then you can definitely be sure the performance does.

What more do you need to know about "the stack" in this case? It's whenever you need to ingest large amounts of JSON in Go. Not sure what could be clearer.

Re: Building a high performance JSON parser

#44

Post by TheCleric »

willsmith72 wrote: post #23
jcelerier wrote: post #11

Earlier quoted context omitted.

How does that contradict what the parent poster says? I think it's very weird to call something "high performance" when it looks like it's maybe 15-20% of the performance of a simdjson in c++. This is not "going from normal performance to high performance", this going from "very subpar" to "subpar"

Ok but how many teams are building web APIs in C++?

I worked with a guy who did this. It was fast, but boy howdy was it not simple.

Re: Building a high performance JSON parser

#45

Post by vlovich123 »

duped wrote: post #30
SmoothBrain12 wrote: post #8

Earlier quoted context omitted.

Exactly, it's not too hard to implement in C. The one I made never copied data, instead saved the pointer/length to the data. The user only had to Memory Map the file (or equivalent), pass that data into the parse. Only memory allocation was for the Jason nodes. This way they only paid the parsing tax (decoding doubles, etc..) if the user used that data. You hit the nail on the head

> The user only had to Memory Map the file (or equivalent) Having done this myself, it's a massive cheat code because your bottleneck is almost always i/o and memory mapped i/o is orders of magnitude faster than sequential calls to read(). But that said it's not always appropriate. You can have gigabytes of JSON to parse, and the JSON might be available over the network, and your service might be running on a small n…

> memory mapped i/o is orders of magnitude faster than sequential calls to read()

That’s not something I’ve generally seen. Any source for this claim?

> You can have gigabytes of JSON to parse, and the JSON might be available over the network, and your service might be running on a small node with limited memory. Memory mapping here adds quite a lot of latency and cost to the system

Why does mmap add latency? I would think that mmap adds more latency for small documents because the cost of doing the mmap is high (cross CPU TLB shoot down to modify the page table) and there’s no chance to amortize. Relatedly, there’s minimal to no relation between SAX vs DOM style parsing and mmap - you can use either with mmap. If you’re not aware, you do have some knobs with mmap to hint to the OS how it’s going to be used although it’s very unwieldy to configure it to work well.

Re: Building a high performance JSON parser

#46

Post by 3cats-in-a-coat »

Galanwe wrote: post #5

[flagged]

JSON is probably the fastest serialization format to produce and parse, which is also safe for public use, compared to binary formats which often have fragile, highly specific and vulnerable encoding as they're directly plopped into memory and used as-is (i.e. they're not parsed at all, it's just two computers exchanging memory dumps).

Compare it with XML for example, which is a nightmare of complexity if you actually follow the spec and not just make something XML-like.

We have some formats which try to walk the boundary between safe/universal and fast like ASN.1 but those are obscure at best.

Re: Building a high performance JSON parser

#47

Post by 3cats-in-a-coat »

riku_iki wrote: post #35
Galanwe wrote: post #5

[flagged]

> "Json" and "Go" seem antithetical in the same sentence as "high performance" to me. As long as we are talking about _absolute performance_. Even just "Json" is problematic here as wire protocol for absolute performance no matter what will be programming language.

I think people say that as they give disproportional weight to the fact it's text-based, while ignoring how astoundingly simple and linear it is to write and read.

The only way to nudge the needle is to start exchanging direct memory dumps, which is what ProtoBuff and the like do. But this is clearly only for very specific use.

Re: Building a high performance JSON parser

#48

Post by explaininjs »

visarga wrote: post #24

nowadays I am more interested in a "forgiving" JSON/YAML parser, that would recover from LLM errors, is there such a thing?

Perhaps not quite what you're asking for, but along the same lines there's this "Incomplete JSON" parser, which takes a string of JSON as it's coming out of an LLM and parses it into as much data as it can get. Useful for building streaming UI's, for instance it is used on https://rexipie.com quite extensively.

https://gist.github.com/JacksonKearl/6778c02bf85495d1e39291c...

Some example test cases:

    { input: '[{"a": 0, "b":', output: [{ a: 0 }] },
    { input: '[{"a": 0, "b": 1', output: [{ a: 0, b: 1 }] },

    { input: "[{},", output: [{}] },
    { input: "[{},1", output: [{}, 1] },
    { input: '[{},"', output: [{}, ""] },
    { input: '[{},"abc', output: [{}, "abc"] },
Work could be done to optimize it, for instance add streaming support. But the cycles consumed either way is minimal for LLM-output-length=constrained JSON.

Fun fact: as best I can tell, GPT-4 is entirely unable to synthesize code to accomplish this task. Perhaps that will change as this implementation is made public, I do not know.

Re: Building a high performance JSON parser

#49

Post by jzwinck »

lionkor wrote: post #37
eatonphil wrote: post #3

The walkthrough is very nice, how to do this if you're going to do it. If you're going for pure performance in a production environment you might take a look at Daniel Lemire's work: https://github.com/simdjson/simdjson . Or the MinIO port of it to Go: https://github.com/minio/simdjson-go .

simdjson has not been the fastest for a long long time

What is faster? According to https://github.com/kostya/benchmarks#json nothing is.

Re: Building a high performance JSON parser

#50

Post by gurrasson »

visarga wrote: post #24

nowadays I am more interested in a "forgiving" JSON/YAML parser, that would recover from LLM errors, is there such a thing?

The jsonrepair tool https://github.com/josdejong/jsonrepair might interest you. It's tailored to fix JSON strings.

I've been looking into something similar for handling partial JSONs, where you only have the first n chars of a JSON. This is common with LLM with streamed outputs aimed at reducing latency. If one knows the JSON schema ahead, then one can start processing these first fields before the remaining data has fully loaded. If you have to wait for the whole thing to load there is little point in streaming.

Was looking for a library that could do this parsing.

Post reply on HN