The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I'm interested in, and do specialized queries and transformations on just the data I care about. I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other…
Zq: An easier and faster alternative to jq
191–200 of 237 posts
Re: Zq: An easier and faster alternative to jq
#192The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I'm interested in, and do specialized queries and transformations on just the data I care about. I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other…
Re: Zq: An easier and faster alternative to jq
#193I see a lot of JQ experts on this thread, so I'll bite the bullet here as a novice. The purpose of life is not to know JQ. I just want to process the JSON so I can move on and do whatever is actually important. Ideally, I'd just be able to tell GPT-codex to do what I want to do to the JSON in English. We're not there yet, but in the meantime if there's another tool that allows me to know less in exchange for doing mo…
Re: Zq: An easier and faster alternative to jq
#194The thing that I find myself wanting, which is lacking in both jq and zq afaik, is interactive exploration. I want to move around in a large JSON file, narrow my context to the portion I'm interested in, and do specialized queries and transformations on just the data I care about. I wrote a tool to do this -- https://github.com/hotsphink/sfink-tools/blob/master/bin/jso... -- but I do not recommend it to anyone other…
Re: Zq: An easier and faster alternative to jq
#195Earlier quoted context omitted.
> However, presumably PowerShell requires at least the .NET Runtime if not the .NET SDK, doesn't it? Nope, these days .NET programs (like PowerShell) bundle the runtime. But even if they did a lighter distribution that depended on the runtime already being installed, there would be no .NET telemetry sent. > Does running the recommended "dotnet --list-runtimes" command send telemetry, like most of the commands? This i…
So the "dotnet" command is only in the SDK, not in the separately downloadable Runtime? Does the Runtime have some other command to launch an executable? Edit: Actually, the ".NET Runtime 6.0.4" [1] (not the SDK) definitely has a "dotnet" command included. Presumably with the telemetry? [1] https://dotnet.microsoft.com/en-us/download/dotnet/6.0
Yes, if you download a .NET Runtime distribution, it will include the `dotnet` command from the SDK so that basic commands like `dotnet --list-runtimes` and `dotnet --list-sdks` are available. These commands may send telemetry. But as you probably saw on https://docs.microsoft.com/en-us/dotnet/core/tools/telemetry , using `dotnet path/to/program.dll` to run an unbundled .NET program will never send telemetry.
Re: Zq: An easier and faster alternative to jq
#196The name of its corporate progenitor may leave a bad taste in some mouths, but I highly recommend PowerShell for this sort of thing. It's cross platform, MIT licensed, and comes with excellent JSON parsing and querying capabilities. Reading, parsing, and querying JSON to return all red cars: Get-Content cars.json | ConvertFrom-Json | ? { $_.color -eq 'red' } The beauty of this is that the query syntax applies not jus…
Powershell's object pipes are more inspectable than any of Bourne's text-based decendants. But the tool itself occupies a niche between "write shell, dealing with esoteria of grep/sed/awk/jq/etc" and "write python getting constructs than handle complexity better than pipes". Looking at the popularity of VSCode, I don't think Microsoft hatred blocks its adoption.
In-apt comparison. The people using VS Code are more likely to be migrating from proprietary tools like PyCharm, Sublime Text, etc or bloated offerings like NetBeans or roughly equivalent offerings like Atom.
The people that would use PowerShell would be migrating from the likes of Zsh, Bash, Fish, and other “hard core free” software.
Re: Zq: An easier and faster alternative to jq
#197Re: Zq: An easier and faster alternative to jq
#198Earlier quoted context omitted.
> It's not as arcane as it seems. The issue with jq is that I use it maybe once a month, or even less. The syntax is "arcane enough" that I keep forgetting how to use it because I use it so sporadically. In comparison awk – which I also don't use that often – has a much easier syntax that I can mostly remember. Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forg…
I wonder if someone tried to use plain JS as a filtering language? It would be more verbose but it would be easy to remember. For example: [1,2,3] | js "out = 0; for (const n of this) out += n" That would print "6". `out` would be a special variable you write to to print the result, and `this` would be the input.
Re: Zq: An easier and faster alternative to jq
#199Earlier quoted context omitted.
I don't know; I wouldn't presume to tell you what you do or don't find arcane, but once I understood the somewhat unusual flow of awk ("for every line, check if the line matches this condition, and if it does run this block of code") I found it's quite easy to work with. It's "arcane" in the sense that it has an implicit loop and that it's a specialized language for a very limited class of problems, but I found that…
> an implicit loop As an occasional awk user, I'd love if you expand on this. Maybe it will help clear things up for me. You're not referring to the fact that awk operates on every line independently, are you?
... | awk '{print $2}'
I thought there was all this confusing syntax, but something like awk '/pattern/ {print}'
was more clear to me. In the first case, the empty pattern matches every line of the input, and the action is simply to print the second field of each line. Patterns can vary in complexity from the empty pattern to long chains of logical operators and regular expressions, such as /pattern/ in the second example. The outer quotes are just to prevent the shell from eating your dollar signs or other special characters. In a standalone AWK script you can write it like /pattern/ {
print
}
which also makes it look more like another language.If you can get your hands on a copy of The AWK Programming Language, it's a pretty quick and pleasant read that helped everything make more sense to me. I do most of my data analysis for my research using AWK and really enjoy working with it.
Re: Zq: An easier and faster alternative to jq
#200https://docs.microsoft.com/en-us/archive/msdn-magazine/2003/...
Anyway, I've installed ZQ and will look to use it, even my simple usage of JQ had already led to thoughts of writing my own, better version :)
Quick bug report: On the Aggregate Functions page the link to _countdistinct_ goes to the page for _count_, and there actually isn't a page at https://zed.brimdata.io/docs/language/aggregates/countdistin....