On-demand JSON: A better way to parse documents?
onlinelibrary.wiley.com
On-demand JSON: A better way to parse documents?
1–10 of 55 posts
Re: On-demand JSON: A better way to parse documents?
#2So they're creating a DOM-like api in front of a sax style parser and getting faster results (barring FPGA and GPU research). It's released as part of SIMDJson.
I wonder if that kind of front end was done in the age of SAX parsers?
Such a well-written paper.
Re: On-demand JSON: A better way to parse documents?
#3So they're creating a DOM-like api in front of a sax style parser and getting faster results (barring FPGA and GPU research). It's released as part of SIMDJson. I wonder if that kind of front end was done in the age of SAX parsers? Such a well-written paper.
> I wonder if that kind of front end was done in the age of SAX parsers?
I though that XPath over SAX was a thing, and xslt was doing sax-like parsing, but turns out I'm wrong. Which is logical considering XPath can refer to previous nodes. That being said, it looks like there is streamable xslt in xslt 3.0, but that looks more niche
Re: On-demand JSON: A better way to parse documents?
#4I don't really understand what's new here compared to what SIMDJSON supported already.
Anyways, it's the best JSON parser I found (in any language), I implemented fastgron (https://github.com/adamritter/fastgron) on top of it because of the on demand library performance.
One problem with the library was that it needed extra padding at the end of the JSON, so it didn't support streaming / memory mapping.
Re: On-demand JSON: A better way to parse documents?
#5On face it, this sounds kind of like the XML::Twig perl module.
Re: On-demand JSON: A better way to parse documents?
#6I don't really understand what's new here compared to what SIMDJSON supported already. Anyways, it's the best JSON parser I found (in any language), I implemented fastgron ( https://github.com/adamritter/fastgron ) on top of it because of the on demand library performance. One problem with the library was that it needed extra padding at the end of the JSON, so it didn't support streaming / memory mapping.
Nice work! I will have to check out your implementation and see if I can borrow any of your optimization ideas. I built jindex (https://github.com/ckampfe/jindex) because I also wanted a faster gron!
Re: On-demand JSON: A better way to parse documents?
#7So they're creating a DOM-like api in front of a sax style parser and getting faster results (barring FPGA and GPU research). It's released as part of SIMDJson. I wonder if that kind of front end was done in the age of SAX parsers? Such a well-written paper.
> I wonder if that kind of front end was done in the age of SAX parsers? I though that XPath over SAX was a thing, and xslt was doing sax-like parsing, but turns out I'm wrong. Which is logical considering XPath can refer to previous nodes. That being said, it looks like there is streamable xslt in xslt 3.0, but that looks more niche
Often a combination of sax and dom is usefull. You get many GBs of SAX stream, but it usually contains the same kind of documents. Creating a DOM at the end a specific token means fast processing, but still the easy of use of DOM.
Re: On-demand JSON: A better way to parse documents?
#8I don't really understand what's new here compared to what SIMDJSON supported already. Anyways, it's the best JSON parser I found (in any language), I implemented fastgron ( https://github.com/adamritter/fastgron ) on top of it because of the on demand library performance. One problem with the library was that it needed extra padding at the end of the JSON, so it didn't support streaming / memory mapping.
This on-demand model has been implemented in simdjson for awhile. This is just the release of the paper.
Previously, simdjson only had a DOM model, where the entire document was parsed in one shot.
Re: On-demand JSON: A better way to parse documents?
#9> The JSON syntax is nearly a strict subset of the popular programming language JavaScript.
What JSON isn’t valid JS?
Re: On-demand JSON: A better way to parse documents?
#10> The JSON syntax is nearly a strict subset of the popular programming language JavaScript. What JSON isn’t valid JS?
"Any JSON text is a valid JavaScript expression, but only after the JSON superset revision. Before the revision, U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are allowed in string literals and property keys in JSON; but the same use in JavaScript string literals is a SyntaxError."
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
"In fact, since JavaScript does not support bare objects, the simple statement {"k":"v"} will emit an error in JavaScript"
https://medium.com/@ExplosionPills/json-is-not-javascript-5d...