Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gojq: Pure Go Implementation of Jq

github.com

Post reply on HN
Print view View original

21–30 of 78 posts

Re: Gojq: Pure Go Implementation of Jq

#21

Post by aaronbee »

alecthomas wrote: post #16
tialaramex wrote: post #13

Earlier quoted context omitted.

Right, the startling thing about Python's previous dict was that it was so terrible that the ordered dict was actually significantly faster. It's like if you did such a bad job making a drag racer that the street legal model of the same car was substantially faster over a quarter mile despite also having much better handling and reliability. In some communities the reaction would have been to write a good unordered d…

> Right, the startling thing about Python's previous dict was that it was so terrible that the ordered dict was actually significantly faster. I've never heard that before and it would be really surprising, given that Python's builtin dict is used for everything from local symbol to object field lookup. Do you have more information?

Here’s a description of the new map implementation and why it’s more efficient: https://www.pypy.org/posts/2015/01/faster-more-memory-effici...

Re: Gojq: Pure Go Implementation of Jq

#22

Post by brundolf »

spullara wrote: post #20

i neither know nor care what language the original jq was implemented in.

I can think of two reasons it matters here:

- Can be used as a library in Go projects

- Memory-safe (could be relevant when processing foreign data, esp as a part of some automated process)

Re: Gojq: Pure Go Implementation of Jq

#23

Post by donio »

brundolf wrote: post #22
spullara wrote: post #20

i neither know nor care what language the original jq was implemented in.

I can think of two reasons it matters here: - Can be used as a library in Go projects - Memory-safe (could be relevant when processing foreign data, esp as a part of some automated process)

Yep, Benthos is an example of a cool project that uses gojq for its jq syntax support.

Re: Gojq: Pure Go Implementation of Jq

#24

Post by vips7L »

haasted wrote: post #5
simonw wrote: post #2

"gojq does not keep the order of object keys" is a bit disappointing. I care about key order purely for cosmetic reasons: when I'm designing JSON APIs I like to put things like the "id" key first in an object layout, and when I'm manipulating JSON using jq or similar I like to maintain those aesthetic choices. I know it's bad to write code that depends on key order, but it's important to me as a way of keeping JSON a…

I bet it's an artifact of Go having a randomized iteration order over maps [0]. Getting a deterministic ordering requires extra work. [0] https://stackoverflow.com/questions/9619479/go-what-determin...

Does Go not have more than one Map implementation in the standard library?

Re: Gojq: Pure Go Implementation of Jq

#25

Post by zxcvbn4038 »

simonw wrote: post #2

"gojq does not keep the order of object keys" is a bit disappointing. I care about key order purely for cosmetic reasons: when I'm designing JSON APIs I like to put things like the "id" key first in an object layout, and when I'm manipulating JSON using jq or similar I like to maintain those aesthetic choices. I know it's bad to write code that depends on key order, but it's important to me as a way of keeping JSON a…

Yeah, this is a deal breaker. While technically the key order doesn’t matter, in the real world it really does matter. People have to read this stuff. People have to be able to differentiate between actual changes and stuff moving around just because. Luckily it’s a solved problem and you can write marshalers that preserve order, but it’s extra work and generally specific to an encoding format. It would be nice to have ordered maps in the base library as an option.

Re: Gojq: Pure Go Implementation of Jq

#26

Post by [deleted] »

vips7L wrote: post #24
haasted wrote: post #5

Earlier quoted context omitted.

I bet it's an artifact of Go having a randomized iteration order over maps [0]. Getting a deterministic ordering requires extra work. [0] https://stackoverflow.com/questions/9619479/go-what-determin...

Does Go not have more than one Map implementation in the standard library?

[deleted]

Re: Gojq: Pure Go Implementation of Jq

#27

Post by esprehn »

vips7L wrote: post #24
haasted wrote: post #5

Earlier quoted context omitted.

I bet it's an artifact of Go having a randomized iteration order over maps [0]. Getting a deterministic ordering requires extra work. [0] https://stackoverflow.com/questions/9619479/go-what-determin...

Does Go not have more than one Map implementation in the standard library?

It does not. Maps are not even a real interface you can implement, it's compiler magic encoded in the language spec: https://dave.cheney.net/2018/05/29/how-the-go-runtime-implem...

This is all fallout of not having generics.

Re: Gojq: Pure Go Implementation of Jq

#28

Post by laqq3 »

cube2222 wrote: post #11

This looks quite cool! I'm not sure though why I would use this over the original jq. However, I can definitely see the value in embedding this into my own applications, to provide jq scripting inside of them. Shameless plug: As I'm not a fan of the jq syntax, I've created jql[0] as an alternative to it. It's also written in Go and presents a lispy continuation-based query language (it sounds much scarier than it rea…

One reason to prefer gojq is that gojq’s author is one of the most knowledgeable person for the original jq (as seen by GitHub PRs and issues), and his gojq fixes many long standing issues in jq.

Plus, for my use cases, gojq runtime performance beats jq by a fair margin.

Re: Gojq: Pure Go Implementation of Jq

#29

Post by cerved »

simonw wrote: post #2

"gojq does not keep the order of object keys" is a bit disappointing. I care about key order purely for cosmetic reasons: when I'm designing JSON APIs I like to put things like the "id" key first in an object layout, and when I'm manipulating JSON using jq or similar I like to maintain those aesthetic choices. I know it's bad to write code that depends on key order, but it's important to me as a way of keeping JSON a…

into it's not about code, it's about predicable and consistent layout so that you can easily diff

Re: Gojq: Pure Go Implementation of Jq

#30

Post by cerved »

brundolf wrote: post #18
lapser wrote: post #10

I have actually fully replaced my jq installation with gojq (including an `ln -s gojq jq`) for a few years, and no script has broken so far. I'm super impressed by the jq compatibility. If you are going down this route, do be careful with performance. I don't know which is more performant as I've never really had to work with large data sets, but I can't help but feel jq will be faster than gojq in such case. I have…

It's very possible it could be faster; jq seems to actually be fairly unoptimized. This implementation in OCaml was featured on HN a while back and it trashes the original jq in performance: https://github.com/davesnx/query-json After seeing that one I did my own (less-complete) version in Rust and managed to squeeze out even more performance in the operations it supports: https://github.com/brundonsmith/jqr

jq is particularly bad at large stream progressing
Post reply on HN