Live data from Hacker News

Q: A faster re-implementaiton of jq written in Reason Native/OCaml

github.com

161–170 of 196 posts

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#161

JMESPath is the only viable alternative, which probably has a wider footprint than even jq as it's part of AWS CLI.

It's definitely popular but “only viable alternative” is a bit strong: that's only if you need compatibility with particular tools which support only one of the two formats. There's no reason why anyone who doesn't like those tools couldn't create a different syntax to scratch whatever particular itch they have.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#162
post #161

JMESPath is the only viable alternative, which probably has a wider footprint than even jq as it's part of AWS CLI.

It's definitely popular but “only viable alternative” is a bit strong: that's only if you need compatibility with particular tools which support only one of the two formats. There's no reason why anyone who doesn't like those tools couldn't create a different syntax to scratch whatever particular itch they have.

It's embeddable and available as a library for all languages [0]. Everything else is nothing but an CLI tool pretty much, which further limits its adoption.

[0]: https://github.com/jmespath

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#163
This is cool, but I’m not sure it’s fair to claim it’s “faster” yet when it doesn’t do 95% of what jq does—-particularly the command line options. If it’s still faster when you can match 80% of the functionality, then it might be a claim worth making.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#164
post #114

Earlier quoted context omitted.

Tools that accept filenames often expect you give them a real file, as they’ll do things on it that may not be supported by the various “it’s a file descriptor pretending to be something on disk” solutions.

Hm, do you have any examples handy? It's not that I don't believe you, it's just that in all the years I've been using this, it has always worked. Granted, I'm only using it for reading data, not for saving stuff to /dev/stdin, which would obviously fail.

Untested, but I expect unzip will fail because it keeps metadata after the files, so needs to seek back. (Unless they detect the pipe and buffer everything instead)

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#165
In case anyone is interested in yet another alternative, I have this old, unpolished project: https://github.com/bauerca/jv

It is a JSON parser in C without heap allocations. The query language is piddly, but the tool can be useful for grabbing a single value from a very large JSON file. I don't have time for it, but someone could fork and make it a real deal.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#166
post #114

Earlier quoted context omitted.

Tools that accept filenames often expect you give them a real file, as they’ll do things on it that may not be supported by the various “it’s a file descriptor pretending to be something on disk” solutions.

Hm, do you have any examples handy? It's not that I don't believe you, it's just that in all the years I've been using this, it has always worked. Granted, I'm only using it for reading data, not for saving stuff to /dev/stdin, which would obviously fail.

atop parsing its logfiles.

  # Cuts off partway through:
  zcat atop.log.gz | atop -r /dev/stdin | less
  # Works fine:
  zcat atop.log.gz > atop && atop -r atop | less
That said, I agree with your experience that /dev/stdin usually works for programs that read a file straight in.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#167
post #54

Earlier quoted context omitted.

Yes, I don't understand how people end up with assertions that the filename is a require argument. At least we've got /dev/stdin or /proc/self/fd/0 as workarounds.

Much Unix software today is written by people who really don't appreciate or understand Unix. It leads to things like Homebrew (early on, at least) completely taking over and breaking /usr/local; to command-line utilities using a single dash (-) precending both short and long option names (granted, --long-opts is a GNUism, but it's a well-established standard); commands that output color by default even when the outp…

"There are an enormous number of tools out there that only exist because people don't know how to chain together basic 1970s Unix text-processing tools in a pipeline."

Arguably that is why the original implementation of Perl was written. If I remember the story correctly, we can never know for sure whether, e.g., AWK would have sufficed, because the particular the job the author wrote Perl for as a contractor was confidential.

Are people using jq most concerned about speed, or are they more concerned about syntax.

JSON suffers a problem from which line-oriented untilities generally have immunity: a large enough and deeply nested JSON structure will choke or crash a program that tries to read all the data into memory at once, or even in large chunks. The process is resource-constrained as the size of the data increases. There are no limits placed on the size or depth of JSON files.

I use sed and tr for most simple JSON files. It is possible to overlfow the sed buffer but it rarely ever happens. sed is found everywhere and it's resource-friendly. Others might choose a program for speed or syntax but the issue of reliability is even more important to me. jq alone is not a reliable solution for any and all JSON. It can be overkill for simple json and resource-constrained for large, complex JSON.

https://stackoverflow.com/questions/59806699/json-to-csv-usi...

netstrings (https://cr.yp.to/proto/netstrings.txt) do not suffer from the same problem as JSON.

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#168
post #161

Earlier quoted context omitted.

It's definitely popular but “only viable alternative” is a bit strong: that's only if you need compatibility with particular tools which support only one of the two formats. There's no reason why anyone who doesn't like those tools couldn't create a different syntax to scratch whatever particular itch they have.

It's embeddable and available as a library for all languages [0]. Everything else is nothing but an CLI tool pretty much, which further limits its adoption. [0]: https://github.com/jmespath

Well, there is XPath 3.1 if you want standards[1] but my point was simply that it depends on whether your question is “I need compatibility with existing jq scripts”, “I need an embeddable library I can integrate in other programs”, or “I want to process JSON for my own usage”.

For example, someone who works with a lot of Python might prefer something like https://github.com/kellyjonbrazil/jello to write comprehensions using the full capabilities of Python, especially since that would provide a direct path to using the final expressions in a Python program or even embedded in one of the environments where Python is used as a scripting language. Is that a viable alternative? The answer depends entirely on who's asking.

1. https://www.w3.org/TR/xpath-31/#id-introduction

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#169
post #82

For everyone pining for a Jq with a different syntax: I have a bunch of links to alternatives collected, you might want to try some of them (some may be for different things than JSON): https://github.com/fiatjaf/awesome-jq https://github.com/TomConlin/json2xpath https://github.com/antonmedv/fx https://github.com/fiatjaf/jiq https://github.com/simeji/jid https://github.com/jmespath/jp https://github.com/cube2222/jql…

My humble attempt https://github.com/jsqry/jsqry-cli2

Re: Q: A faster re-implementaiton of jq written in Reason Native/OCaml

#170
post #92
post #47

1) refuses to operate on stdin; requires a filename argument, which is so irritating. 2) doesn't accept values that jq accepts % time jq -r '[expression]'

1 is easy to work around (handy tip incoming for any tools that _seem_ to not support stdin but actually do, as stdin is also available as a file in unix): echo '{"foo": "bar"}' | query-json ".foo" /dev/stdin

alternatively in bash:

   query-json ".foo" 
Post reply on HN