31–40 of 281 posts
I was a bit skeptical at first, but after reading more into jsongrep, it's actually very good. Only did a very quick test just now, and after stumbling over slightly different syntax to jq, am actually quite impressed. Give it a try
However, as someone who always loved faster software and being an optimisation nerd, hat's off!
Earlier quoted context omitted.
I process TB-size ndjson files. I want to use jq to do some simple transformations between stages of the processing pipeline (e.g. rename a field), but it so slow that I write a single-use node or rust script instead.
This isn't for you then > The query language is deliberately less expressive than jq's. jsongrep is a search tool, not a transformation tool-- it finds values but doesn't compute new ones. There are no filters, no arithmetic, no string interpolation. Mind me asking what sorts of TB json files you work with? Seems excessively immense.
When initially opening the page it had broken colors in light mode. For anyone else encountering it: switch to dark mode and then back to light mode to fix it.
Having used `jq` and `yq` (which followed from the former, in spirit), I have never had to complain about performance of the _latter_ which an order of magnitude (or several) _slower_ than the former. So if there's something faster than `jq`, it's laudable that the author of the faster tool accomplished such a goal, but in the broader context I'd say the performance benefit would be required by a niche slice of the u…
Earlier quoted context omitted.
The use case could be e.g. exactly processing an old trove of logs into something more easily indexed and queryable, and you might want to use jq as part of that processing pipeline
Fair, but for a once-off thing performance isn't usually a major factor. The comment I was replying to implied this was something more regular. EDIT: why is this being downvoted? I didn't think I was rude. The person I responded to made a good point, I was just clarifying that it wasn't quite the situation I was asking about.
Earlier quoted context omitted.
So what's the use case for keeping them in that format rather than something more easily indexed and queryable? I'd probably just shove it all into Postgres, but even a multi terabyte SQLite database seems more reasonable.
Replying here because the other comment is too deeply nested to reply. Even if it's once off, some people handle a lot of once-offs, that's exactly where you need good CLI tooling to support it. Sure jq isn't exactly super slow, but I also have avoided it in pipelines where I just need faster throughput. rg was insanely useful in a project I once got where they had about 5GB of source files, a lot of them auto-genera…
Earlier quoted context omitted.
The use case could be e.g. exactly processing an old trove of logs into something more easily indexed and queryable, and you might want to use jq as part of that processing pipeline
Fair, but for a once-off thing performance isn't usually a major factor. The comment I was replying to implied this was something more regular. EDIT: why is this being downvoted? I didn't think I was rude. The person I responded to made a good point, I was just clarifying that it wasn't quite the situation I was asking about.
Having used `jq` and `yq` (which followed from the former, in spirit), I have never had to complain about performance of the _latter_ which an order of magnitude (or several) _slower_ than the former. So if there's something faster than `jq`, it's laudable that the author of the faster tool accomplished such a goal, but in the broader context I'd say the performance benefit would be required by a niche slice of the u…