[flagged]
A Faster Alternative to Jq
91–100 of 281 posts
Re: A Faster Alternative to Jq
#92Earlier quoted context omitted.
This isn't for you then > The query language is deliberately less expressive than jq's. jsongrep is a search tool, not a transformation tool-- it finds values but doesn't compute new ones. There are no filters, no arithmetic, no string interpolation. Mind me asking what sorts of TB json files you work with? Seems excessively immense.
> Uses jq for TB json files > Hadoop: bro > Spark: bro > hive: bro > data team: bro
[deleted]
Re: A Faster Alternative to Jq
#93I appreciate performance as much as the next person; but I see this endless battle to measure things in ns/us/ms as performative. Sure there are 0.000001% edge cases where that MIGHT be the next big bottleneck. I see the same thing repeated in various front end tooling too. They all claim to be _much_ faster than their counterpart. 9/10 whatever tooling you are using now will be perfectly fine. Example; I use grep a…
Whenever you have this kind of impressions on some development, here are my 2 cents: just think "I'm not the target audience". And that's fine. The difference between 2ms and 0.2ms might sound unneeded, or even silly to you. But somebody, somewhere, is doing stream processing of TB-sized JSON objects, and they will care. These news are for them.
Also as someone who looks at latency charts too much, what happens is a request does a lot in series and any little ms you can knock off adds up. You save 10ms by saving 10 x 1ms. And if you are a proxyish service then you are a 10ms in a chain that might be taking 200 or 300ms. It is like saving money, you have to like cut lots of small expenses to make an impact. (unless you move etc. but once you done that it is small numerous things thay add up)
Also performance improvements on heavy used systems unlocks:
Cost savings
Stability
Higher reliability
Higher throughput
Fewer incidents
Lower scaling out requirements.
Re: A Faster Alternative to Jq
#94Just about to read, but I had to change to dark mode to be able to see the examples, which are bold white on a white background.
Re: A Faster Alternative to Jq
#95[dead]
Re: A Faster Alternative to Jq
#96Nowadays I'd just use clickhouse-local / chdb / duckdb to query json files (and pretty much any standard format files)
Re: A Faster Alternative to Jq
#97I appreciate performance as much as the next person; but I see this endless battle to measure things in ns/us/ms as performative. Sure there are 0.000001% edge cases where that MIGHT be the next big bottleneck. I see the same thing repeated in various front end tooling too. They all claim to be _much_ faster than their counterpart. 9/10 whatever tooling you are using now will be perfectly fine. Example; I use grep a…
Whenever you have this kind of impressions on some development, here are my 2 cents: just think "I'm not the target audience". And that's fine. The difference between 2ms and 0.2ms might sound unneeded, or even silly to you. But somebody, somewhere, is doing stream processing of TB-sized JSON objects, and they will care. These news are for them.
But even in this example, the 2ms vs 0.2 is irrelevant - its whatever the timings are for TB-size objects.
So went not compare that case directly? We'd also want to see the performance of the assumed overheads i.e. how it scales.
Re: A Faster Alternative to Jq
#98Another alternative is oj, https://github.com/ohler55/ojg. I don't know how the performance compares to jq or any others but it does use JSONPath as the query language. It has a few other options for making nicely formatted JSON and colorizing JSON.
Re: A Faster Alternative to Jq
#99Earlier quoted context omitted.
Whenever you have this kind of impressions on some development, here are my 2 cents: just think "I'm not the target audience". And that's fine. The difference between 2ms and 0.2ms might sound unneeded, or even silly to you. But somebody, somewhere, is doing stream processing of TB-sized JSON objects, and they will care. These news are for them.
Also as someone who looks at latency charts too much, what happens is a request does a lot in series and any little ms you can knock off adds up. You save 10ms by saving 10 x 1ms. And if you are a proxyish service then you are a 10ms in a chain that might be taking 200 or 300ms. It is like saving money, you have to like cut lots of small expenses to make an impact. (unless you move etc. but once you done that it is s…
Wait what? I don't get why performance improvement implies reliability and incident improvement.
For example, doing dangerous thing might be faster (no bound checks, weaker consistency guarantee, etc), but it clearly tend to be a reliability regression.
Re: A Faster Alternative to Jq
#100I wonder so often about many new CLI tools whose primary selling point is their speed over other tools. Yet I personally have not encountered any case where a tool like jq feels incredibly slow, and I would feel the urge to find something else. What do people do all day that existing tools are no longer enough? Or is it that kind of "my new terminal opens 107ms faster now, and I don't notice it, but I simply feel bet…
[flagged]