Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Faster Alternative to Jq

micahkepe.com

Post reply on HN
Print view View original

71–80 of 281 posts

Re: A Faster Alternative to Jq

#71

Post by rennokki »

nchmy wrote: post #14
n_e wrote: post #11

Earlier quoted context omitted.

I process TB-size ndjson files. I want to use jq to do some simple transformations between stages of the processing pipeline (e.g. rename a field), but it so slow that I write a single-use node or rust script instead.

This isn't for you then > The query language is deliberately less expressive than jq's. jsongrep is a search tool, not a transformation tool-- it finds values but doesn't compute new ones. There are no filters, no arithmetic, no string interpolation. Mind me asking what sorts of TB json files you work with? Seems excessively immense.

> Uses jq for TB json files

> Hadoop: bro

> Spark: bro

> hive: bro

> data team: bro

Re: A Faster Alternative to Jq

#73

Post by s_dev »

enricozb wrote: post #67

I am excited for some alternative syntax to jq's. I haven't given much thought to how I'd write a new JSON query syntax if I were writing things from scratch, but I personally never found the jq syntax intuitive. Perhaps I haven't given it enough effort to learn properly.

You don't learn it properly. It's not supposed to be intuitive, it's supposed to be concise at the cost of it being intuitive. Would be like somebody saying typing words in to Google is more intuitive than writing regex.

jq is supposed to fit in to other bash scripts as a one liner. That's it's super power. I know very few people who write regex on the fly either (unless you were using it everyday) they check the documentation and flesh it out when they need it.

Just use Claude to generate the jq expression you need and test it.

Re: A Faster Alternative to Jq

#74

Post by xlii »

Kovah wrote: post #5

I wonder so often about many new CLI tools whose primary selling point is their speed over other tools. Yet I personally have not encountered any case where a tool like jq feels incredibly slow, and I would feel the urge to find something else. What do people do all day that existing tools are no longer enough? Or is it that kind of "my new terminal opens 107ms faster now, and I don't notice it, but I simply feel bet…

It's a simple loop:

- Someone likes tool X

- Figures, that they can vibe code alternative

- They take Rust for performance or FAVORITE_LANG for credentials

- Claude implements small subset of features

- Benchmark subset

- Claim win, profit on showcase

Note: this particular project doesn't have many visible tells, but there's pattern of overdocumentation (17% comment-to-code ratio, >1000 words in README, Claude-like comment patterns), so it might be a guided process.

I still think that the project follows the "subset is faster than set" trend.

Re: A Faster Alternative to Jq

#76

Post by j1elo »

1a527dd5 wrote: post #43

I appreciate performance as much as the next person; but I see this endless battle to measure things in ns/us/ms as performative. Sure there are 0.000001% edge cases where that MIGHT be the next big bottleneck. I see the same thing repeated in various front end tooling too. They all claim to be _much_ faster than their counterpart. 9/10 whatever tooling you are using now will be perfectly fine. Example; I use grep a…

Whenever you have this kind of impressions on some development, here are my 2 cents: just think "I'm not the target audience". And that's fine.

The difference between 2ms and 0.2ms might sound unneeded, or even silly to you. But somebody, somewhere, is doing stream processing of TB-sized JSON objects, and they will care. These news are for them.

Re: A Faster Alternative to Jq

#77

Post by jmalicki »

httpsterio wrote: post #60
dalvrosa wrote: post #47

Earlier quoted context omitted.

Fair, but agentic tooling can benefit quite a lot from this Opencode, ClaudeCode, etc, feel slow. Whatever make them faster is a win :)

The 2ms it takes to run jq versus the 0.2ms to run an alternative is not why your coding agent feels slow.

Still, jq is run a whole lot more than it used to be due to coding agents, so every bit helps.

The vast majority of Linux kernel performance improvement patches probably have way less of a real world impact than this.

Re: A Faster Alternative to Jq

#78

Post by jmalicki »

CodeCompost wrote: post #34
Bigpet wrote: post #3

When initially opening the page it had broken colors in light mode. For anyone else encountering it: switch to dark mode and then back to light mode to fix it.

I suspect the website is vibe-coded, like the tool itself.

I can forgive vibe code... It needs to execute if it works it's fine.

Unedited vibe documentation is unforgivable.

Re: A Faster Alternative to Jq

#79

Post by hrmtst93837 »

Kovah wrote: post #5

I wonder so often about many new CLI tools whose primary selling point is their speed over other tools. Yet I personally have not encountered any case where a tool like jq feels incredibly slow, and I would feel the urge to find something else. What do people do all day that existing tools are no longer enough? Or is it that kind of "my new terminal opens 107ms faster now, and I don't notice it, but I simply feel bet…

[flagged]

Re: A Faster Alternative to Jq

#80

Post by oefrha »

InfinityByTen wrote: post #17
Kovah wrote: post #5

I wonder so often about many new CLI tools whose primary selling point is their speed over other tools. Yet I personally have not encountered any case where a tool like jq feels incredibly slow, and I would feel the urge to find something else. What do people do all day that existing tools are no longer enough? Or is it that kind of "my new terminal opens 107ms faster now, and I don't notice it, but I simply feel bet…

You don't know something is slow until you encounter a use case where the speed becomes noticeable. Then you see the slowness across the board. If you can notice that a command hasn't completed and you are able to fully process a thought about it, it's slow(er than your mind, ergo slow!). Usually, a perceptive user/technical mind is able to tweak their usage of the tools around their limitations, but if you can find…

Well grep is just better sometimes. Like you want to copy some lines and grep at the end of a pipeline is just easier than rg -N to suppress line numbers. Whatever works, no need to facepalm.
Post reply on HN