Jq cries out for a good O'Reilly cookbook.
Extracting Objects Recursively with Jq
51–60 of 76 posts
Re: Extracting Objects Recursively with Jq
#52Jq has a playground that’s great for learning how to use it https://jqplay.org/
I often build my Jq steps in the playground before committing.
Re: Extracting Objects Recursively with Jq
#53Earlier quoted context omitted.
I'm the opposite, I find that jq doesn't have a reason to exist, other than pretty printing JSON files on a terminal and doing basic filtering on a JSON object and only as interactive shell usage, NOT in a script. It's the classical tool, like sed, like awk, like a ton of unix utility that at first they seem to you easy to use, then you have to do something complex and you start abusing them, by piping things multipl…
Speaking as a long-time python developer... If you actually "get into" jq you find out that it's a significantly neater language than it appears on the surface. Firstly it does allow you write multi-line scripts, and things start to look a lot neater once you do. Secondly it's actually a real, working, functional programming language, which allows very succinct expression of ideas which, in python, would likely requi…
> Secondly it's actually a real, working, functional programming language, which allows very succinct expression of ideas which, in python, would likely require the reader to track state across explicit loops and the like.
This is the problem. Is yet another language that someone has to learn and know, like the ton of other UNIX commands that have their own DSL.
> Once you dig into the manual, you also tend to discover that a lot of the things that cause you to string multiple jq invocations together aren't actually necessary because there are quite sensible ways of handling them in-language.
Nice. I can dig into the manual and spend a day to learn it, but I don't have that time. I have a script to fix, I know how to program in python, and throw away the jq code and substitute it with 10 lines of python in a minute, and problems solved.
> It's quite laughable though to tout python over jq because of it adding a dependency
If jq enters your codebase then everywhere you have to install it. It's not that difficult (but not trivial on Windows), but it is annoying, you run a script and you then find out that you don't have jq installed and you need to install it.
Also what is faster: eliminating the need for the jq command in a script, or installing jq on tens of different systems with different operating systems?
No, where I work I established a rule that every script should be written in python and should use only the standard library, with a few exception (e.g. we work with AWS so boto3 is an exception). If not every developer did use whatever tool they thought it was cool (like jq), write a ton of bash spaghetti code with all these tools used together, and forced every other developer to install them on their system (and cause a lot of work to the IT, i.e. me to do so if they weren't able and fix all the problem).
> Perhaps if you're already embedded in python-land and all your environments already have python
Python is everywhere. Every Linux distribution have a python interpreter in them, same for macOS, and in Windows nowadays you install directly from the store with one click. The problem is that if jq enters the codebase then of course every developer machine has to have it installed. This is annoying.
Re: Extracting Objects Recursively with Jq
#54To better appreciate the structure of the document the author is dealing with (and to cast a bit of light on which words are variables in the document and which are `jq` syntax. I offer a shameless plug to a one liner well, I would but the result is "too long for a HN comment" so here a bunch is sniped out of the middle (unedited the result would currently be 140 lines) curl -s https://hn.algolia.com/api/v1/items/279…
Looks a bit like `jq -rc '[path(..)|map(strings//"[]")]|unique[]|"."+join("|.")'`, but might I suggest `jq -rc '[paths|map(("."+strings)//"[]")|join("")]|unique[]'`?
Re: Extracting Objects Recursively with Jq
#55Could the title be changed to specify `Extracting JSON objects recursively with jq` please?
Re: Extracting Objects Recursively with Jq
#56JQ is often frustrating when you want to do something non-trivial but you can't figure out how and the documentation is of little help. I think JQ could really benefit from having a classic programming language style "book", like "The AWK Programming Language". JQ is fundamentally a functional programming language with semantics that are not obvious reading its current docs.
This exists. At least, a detailed manual exists. It's the jq wiki on github [1]. Lots of detailed information/recipes.
Re: Extracting Objects Recursively with Jq
#57Forget about jq. JSONata is much more powerful - http://docs.jsonata.org/overview.html
It doesn't appear to have a CLI, which is the whole point of jq
Re: Extracting Objects Recursively with Jq
#58In a job I did years ago, we had to deal with errors caused by small changes made in huge object trees in C# in a domain that requires handling thousands of parameters that all influence each other.
Fortunately there was a Visual Studio extension that could export objects into JSON/XML/C# representation; then all I had to do was to diff the object dump. Was often called a "Jedi Wizard" for using this trick lol
Re: Extracting Objects Recursively with Jq
#59I wish jq could be used in Python. It would be so nice. Imagine writing a simple jq string and running that, rather than writing complicated loops for examining json data.
It does exist as a library for Python, but unfortunately it links to jq’s C library. That means if I use it, then whoever uses my program has to compile C code whenever they pip install my program. I’m unwilling to do that, because history shows that as t approaches infinity, the chance of C code failing to compile during a pip install approaches 100%.
If anyone’s looking for a challenge, please implement all of jq’s logic in pure Python. It would be so wonderful…
Re: Extracting Objects Recursively with Jq
#60Earlier quoted context omitted.
Speaking as a long-time python developer... If you actually "get into" jq you find out that it's a significantly neater language than it appears on the surface. Firstly it does allow you write multi-line scripts, and things start to look a lot neater once you do. Secondly it's actually a real, working, functional programming language, which allows very succinct expression of ideas which, in python, would likely requi…
> Secondly it's actually a real, working, functional programming language, which allows very succinct expression of ideas which, in python, would likely require the reader to track state across explicit loops and the like. This is the problem. Is yet another language that someone has to learn and know, like the ton of other UNIX commands that have their own DSL. > Once you dig into the manual, you also tend to discov…
I sympathize with both points of view. I think of autotools as an extreme example of how less general tools tend to grow in complexity to the point that the effort to learn them outweighs the benefit you get from knowing them well. But there is something missing from the "just use python everywhere" argument as well. For many usecases, choosing a less expressive language can guard against excessive complexity creeping in at that layer. IMO, json+jq fills a useful niche between semistructured/unstructured-text+awk/sed/bash and APIs+some-gp-scripting-language