Building a high performance JSON parser
dave.cheney.net
Building a high performance JSON parser
1–10 of 193 posts
Re: Building a high performance JSON parser
#2Also interesting:
https://youtu.be/a7VBbbcmxyQ
Re: Building a high performance JSON parser
#3The walkthrough is very nice, how to do this if you're going to do it.
If you're going for pure performance in a production environment you might take a look at Daniel Lemire's work: https://github.com/simdjson/simdjson. Or the MinIO port of it to Go: https://github.com/minio/simdjson-go.
Re: Building a high performance JSON parser
#4Great to see a shout out to Phil Pearl!
Also worth looking at https://github.com/bytedance/sonic
Re: Building a high performance JSON parser
#5[flagged]
Re: Building a high performance JSON parser
#6[flagged]
Did you even open the article? Following is literally in first paragraph
> This package offers the same high level json.Decoder API but higher throughput and reduced allocations
Re: Building a high performance JSON parser
#7I'm surprised there's no way to say 'I really mean it, inline this function' for the stuff that didn't inline because it was too big.
The baseline whitespace count/search operation seems like it would be MUCH faster if you vectorized it with SIMD, but I can understand that being out of scope for the author.
Re: Building a high performance JSON parser
#8[flagged]
Exactly, it's not too hard to implement in C. The one I made never copied data, instead saved the pointer/length to the data.
The user only had to Memory Map the file (or equivalent), pass that data into the parse.
Only memory allocation was for the Jason nodes.
This way they only paid the parsing tax (decoding doubles, etc..) if the user used that data.
You hit the nail on the head
Re: Building a high performance JSON parser
#9The walkthrough is very nice, how to do this if you're going to do it. If you're going for pure performance in a production environment you might take a look at Daniel Lemire's work: https://github.com/simdjson/simdjson . Or the MinIO port of it to Go: https://github.com/minio/simdjson-go .
If your JSON always looks the same you can also do better than general JSON parsers.
Re: Building a high performance JSON parser
#10You might want to take a look at https://github.com/ohler55/ojg. It takes a different approach with a single pass parser. There are some performance benchmarks included on the README.md landing page.