Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Faster Alternative to Jq

micahkepe.com

Post reply on HN
Print view View original

181–190 of 281 posts

Re: A Faster Alternative to Jq

#181

Post by sigseg1v »

amelius wrote: post #157
latexr wrote: post #156

Earlier quoted context omitted.

Because the output you get can have hallucinations, which don’t happen with a deterministic tool. Furthermore, by getting the `jq` command you get something which is reusable, fast, offline, local, doesn’t send your data to a third-party, doesn’t waste a bunch of tokens, … Using an LLM to filter the data is worse in every metric.

You can use a local LLM and you can ask it to use tools so it is faster.

"so it is faster" than what? A cloud hosted LLM? That's a pretty low bar. It's certainly not faster than jq.

Re: A Faster Alternative to Jq

#182

Post by Shorel »

amelius wrote: post #166
Shorel wrote: post #161

Earlier quoted context omitted.

You really need to go and learn about the concept of determinism and why for some tasks we need and want deterministic solutions. It's an important idea in computer science. Go and learn.

You need to learn to adapt to the real world where most things are not deterministic. Go and learn.

I already know that. That's why we have deterministic algorithms, to simplify that complexity. You have much to learn, witty answers mean nothing here, particularly empty witty answers, which are no better than jokes. Maybe stand-up comedy is your call in life.

Re: A Faster Alternative to Jq

#183

Post by mattbis »

1a527dd5 wrote: post #43

I appreciate performance as much as the next person; but I see this endless battle to measure things in ns/us/ms as performative. Sure there are 0.000001% edge cases where that MIGHT be the next big bottleneck. I see the same thing repeated in various front end tooling too. They all claim to be _much_ faster than their counterpart. 9/10 whatever tooling you are using now will be perfectly fine. Example; I use grep a…

Was about to post exactly this... It is impressive engineering wise, but for data and syntax, ease of use or all the great features, I care about that more. Speed isn't that important to me for a lot of these tools.

If I/you was working with JSON of that size where this was important, id say you probably need to stop using JSON! and some other binary or structured format... so long as it has some kinda tooling support.

And further if you are doing important stuff in the CLI needing a big chain of commands, you probably should be programming something to do it anyways...

that's even before we get to the whole JSON isn't really a good data format whatsoever... and there are many better ways. The old ways or the new ways. One day I will get to use my XSLT skills again :D

Re: A Faster Alternative to Jq

#185

Post by mlmonkey »

Minor suggestion: often I just want to extract one field, whose name I know exactly. I see that `jg` has an option `-F` like this:

$ cat sample.json | jg -F name

I would humbly suggest that a better syntax would be:

$ cat sample.json | jg .name

for a leaf node named "name"; or

$ cat sample.json | jg -F .name.

for any node named "name".

Re: A Faster Alternative to Jq

#186

Post by randusername »

d35007 wrote: post #160
regus wrote: post #137

Jq's syntax is so arcane I can never remember it and always need to look up how to get a value from simple JSON.

That’s interesting! Can you say a little more? I find jq’s syntax and semantics to be simple and intuitive. It’s mostly dots, pipes, and brackets. It’s a lot like writing shell pipelines imo. And I tend to use it in the same way. Lots of one-time use invocations, so I spend more time writing jq filters than I spend reading them. I suspect my use cases are less complex than yours. Or maybe jq just fits the way I think…

I'm not GP, I use jq all the time, but I each time I use it I feel like I'm still a beginner because I don't get where I want to go on the first several attempts. Great tool, but IMO it is more intuitive to JSON people that want a CLI tool than CLI people that want a JSON tool. In other words, I have my own preconceptions about how piping should work on the whole thing, not iterating, and it always trips me up.

Here's an example of my white whale, converting JSON arrays to TSV.

cat input.json | jq -S '(first|keys | map({key: ., value: .}) | from_entries), (.[])' | jq -r '[.[]] | @tsv' > out.tsv

Re: A Faster Alternative to Jq

#188

Post by password4321 »

Jenk wrote: post #106

I switched to Jaq[0] a while back for the 'correctness' sake rather than performance. But Jaq also claims to be more performant than jq. [0]: https://github.com/01mf02/jaq

Thank you for the recommendation.

It looks like jaq has already progressed much further in the right direction than jsongrep has just started in the not-quite-as-right direction.

Re: A Faster Alternative to Jq

#189

Post by rlonstein »

ifh-hn wrote: post #19

I learned a number of data processing cli tools: jq, mlr, htmlq, xsv, yq, etc; to name a few. Not to the level of completing advent of code or anything, but good enough for my day to day usage. It was never ending with the amount of formats I needed to extract data from, and the different syntax's. All that changed when I found nushell though, its replaced all of these tools for me. One syntax for everything, breath…

+1. I switched to using Nushell as my daily driver around mid-2023 (0.84.0?) and use it in preference to other interactive tools. I do keep at hand jq, yq, and mlr because I need to exchange stuff with colleagues who don't use Nu.

Re: A Faster Alternative to Jq

#190

Post by mikkupikku »

sgarland wrote: post #140
mikkupikku wrote: post #134

Earlier quoted context omitted.

If somebody needs performance, they probably shouldn't be calling out to a separate process for json of all things, no? (Honestly, who even still writes shell scripts? Have a coding agent write the thing in a real scripting language at least, they aren't phased by the boilerplate of constructing pipelines with python or whatever. I haven't written a shell script in over a year now.)

If you’re writing the script to be used by multiple people, or on multiple systems, or for CI runners, or in containers, etc. then there’s no guarantee of having Python (mostly for the container situation, but still), much less of its version. It’s far too easy to accidentally use a feature or syntax that you took for granted, because who would still be using 3.7 today, anyway? I say this from painful recent experien…

If you're distributing the script to other people then the benifit of using python and getting stuff like high quality argument parsing for free is even greater.
Post reply on HN