Live data from Hacker News

Q: A faster re-implementaiton of jq written in Reason Native/OCaml

github.com

31–40 of 196 posts

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#31
post #8

Are we sure it should get a single-letter 'q' binary name though? Docs seem to point that it's short for 'query-json'? Why not call it 'query-json' and let the user decide that as a shell alias or whatever. Even the ubiquitous 'ls' and 'cd' are two characters.

Yeah. Can you imagine trying to do a web search for ‘q’?

I can and it is lightly disturbing. https://imgur.com/a/yAOF31g ^_^

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#32
post #13

I'd long for such a tool with a better comprehensible query language.

If so, and for anybody else having this wish, check out jql[0], I've created it exactly for this reason, to have the most common jq operations available in a more uniform and easier to use interface.

[0]: https://github.com/cube2222/jql

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#34

Earlier quoted context omitted.

jq is pretty fast in my experience. But there have been cases where I've wanted it to be faster (dealing with a 90GB JSON file). The main weakness seems to be streaming use cases (not having the whole file in memory at once). These are supported, but the syntax is quite awkward.

Out of interest, what created a 90GB JSON file?

A Firebase Realtime Database backup file

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#35

Curious, any description as to why it's faster? Something intrinsic to Reason Native/OCaml? Architectural changes? Reduced feature set?

Jq appears to have its own hand written json parser and requires flex/bison. I suspect something about the hand written parser is slow for large data sets.

I was somewhat surprised it didn't use an existing json parser library.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#36

Curious, any description as to why it's faster? Something intrinsic to Reason Native/OCaml? Architectural changes? Reduced feature set?

jq’s in C do probably not anything intrinsic. Not to mention I don’t think the ocaml compiler is an optimisation beast.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#37

Do we need to make jq faster ? Anyone has issues with current speed ? Is there any specific reason other than "because we can" ?

According to the "Purpose" section of the readme, it doesn't look like beating jq's speed was ever a goal. It was meant to be a learning exercise.

But if I had done something like that, and then serendipitously discovered that I was exceeding the original's performance, I certainly wouldn't be shy about it.

Also, this comes across as armchair criticism purely for the sake of armchair criticism. My own experience has been that, when I'm doing ETL that involves wrangling JSON, the "wrangling JSON" bit of it is almost always the bottleneck. So any improvement is more than welcome and deserves to be cheered. Even if it's an improvement on something that's already the current fastest way to do it.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#38
The speed is not concern for me. I am wondering if there something better than `jq` in terms of syntax. Whenever I want to get something more that just prettify json output in the console or simply get value by specific field name I have a problem, for me it is just difficult to remember jq syntax without looking into history. As well have in my notes links to examples like this one

https://mosermichael.github.io/jq-illustrated/dir/content.ht...

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#39
post #13

I'd long for such a tool with a better comprehensible query language.

If so, and for anybody else having this wish, check out jql[0], I've created it exactly for this reason, to have the most common jq operations available in a more uniform and easier to use interface. [0]: https://github.com/cube2222/jql

Nice!

I will try to bring it to the brenchmark, thanks for sharing

Post reply on HN