Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gojq: Pure Go Implementation of Jq

github.com

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Gojq: Pure Go Implementation of Jq

#61

Post by Someone »

haasted wrote: post #5
simonw wrote: post #2

"gojq does not keep the order of object keys" is a bit disappointing. I care about key order purely for cosmetic reasons: when I'm designing JSON APIs I like to put things like the "id" key first in an object layout, and when I'm manipulating JSON using jq or similar I like to maintain those aesthetic choices. I know it's bad to write code that depends on key order, but it's important to me as a way of keeping JSON a…

I bet it's an artifact of Go having a randomized iteration order over maps [0]. Getting a deterministic ordering requires extra work. [0] https://stackoverflow.com/questions/9619479/go-what-determin...

No, it isn’t. “gojq does not keep the order of object keys” isn’t about ordering keys consistently across runs, it’s about keeping them in the order of the input file.

Re: Gojq: Pure Go Implementation of Jq

#62

Post by Someone »

silon42 wrote: post #56
esprehn wrote: post #27

Earlier quoted context omitted.

It does not. Maps are not even a real interface you can implement, it's compiler magic encoded in the language spec: https://dave.cheney.net/2018/05/29/how-the-go-runtime-implem... This is all fallout of not having generics.

So, golang is no longer suitable to nicely process json documents.

Nobody forces golang programmers to use the built-in map.

Also, some people would argue any map is unsuitable for many use cases of jq. If you want to keep the keys in the order of the input file, it certainly isn’t.

And yes, formally, json doesn’t change when reordering keys, but json often is treated as text and then, it is. You can use jq, for example, to do some transforms on a json file and get a nice git diff.

This tool may (or may not) produce a diff that’s much larger than necessary.

Re: Gojq: Pure Go Implementation of Jq

#64

Post by akpa1 »

Someone wrote: post #61
haasted wrote: post #5

Earlier quoted context omitted.

I bet it's an artifact of Go having a randomized iteration order over maps [0]. Getting a deterministic ordering requires extra work. [0] https://stackoverflow.com/questions/9619479/go-what-determin...

No, it isn’t. “gojq does not keep the order of object keys” isn’t about ordering keys consistently across runs, it’s about keeping them in the order of the input file.

Which it can't do because, as mentioned, Go randomly iterates over maps. That's the data structure that most would use to load arbitrary input files into the program.

Re: Gojq: Pure Go Implementation of Jq

#65

Post by kalefranz »

tialaramex wrote: post #60
nemothekid wrote: post #51

Earlier quoted context omitted.

> The main thrust of your claim obviously can't be true It's surprising that iterating a dense array is faster than iterating a hashmap? I don't think you are parsing the parent post correctly. If dictionaries are commonly iterated in python, then iterating an array of 100 items that fits in one cache-line will be faster than iterating a hashmap which might have 100 items in 100 cache lines.

The claim was that: "Actually an ordered dictionary has improved performance over an unordered dictionary" Having a dense array is not, as it seems both you and chippiewill imagine, somehow a unique property of ordered dictionaries. An unordered dictionary is free to use exactly the same implementation detail. The choice to preserve order is in addition to using dense arrays. The OrderedDict must use tombstones in or…

Enjoy and appreciate the discussion. Is this in the same neighborhood of why the reworked dict implementation in python 3.6 had insertion order as a detail, but explicitly claimed it was not a feature that should be relied upon? At least until python 3.7 cemented the behavior as a feature.

Re: Gojq: Pure Go Implementation of Jq

#66

Post by xorcist »

simonw wrote: post #2

"gojq does not keep the order of object keys" is a bit disappointing. I care about key order purely for cosmetic reasons: when I'm designing JSON APIs I like to put things like the "id" key first in an object layout, and when I'm manipulating JSON using jq or similar I like to maintain those aesthetic choices. I know it's bad to write code that depends on key order, but it's important to me as a way of keeping JSON a…

Ordered keys in json is not only for cosmetic reasons. If this ever touches disk you want the ability to diff them or stash them in git without the whole file changing with every update.

Re: Gojq: Pure Go Implementation of Jq

#67

Post by Someone »

akpa1 wrote: post #64
Someone wrote: post #61

Earlier quoted context omitted.

No, it isn’t. “gojq does not keep the order of object keys” isn’t about ordering keys consistently across runs, it’s about keeping them in the order of the input file.

Which it can't do because, as mentioned, Go randomly iterates over maps. That's the data structure that most would use to load arbitrary input files into the program.

If you have a hammer in your toolbox, it doesn’t mean you have to use it in every job. It golangs maps don’t do what you want, pick a different data structure.

This is like claiming that, because updating native integers isn’t guaranteed to be atomic in a language, you can’t do multi-threaded programming.

Re: Gojq: Pure Go Implementation of Jq

#68

Post by krab »

fwip wrote: post #3

Not implementing key-sorting is a curious decision: > gojq does not keep the order of object keys. I understand this might cause problems for some scripts but basically, we should not rely on the order of object keys. Due to this limitation, gojq does not have keys_unsorted function and --sort-keys (-S) option. I would implement when ordered map is implemented in the standard library of Go but I'm less motivated. I f…

You can always add this feature but it's problematic to remove it.

Re: Gojq: Pure Go Implementation of Jq

#69

Post by rfiat »

Someone wrote: post #61
haasted wrote: post #5

Earlier quoted context omitted.

I bet it's an artifact of Go having a randomized iteration order over maps [0]. Getting a deterministic ordering requires extra work. [0] https://stackoverflow.com/questions/9619479/go-what-determin...

No, it isn’t. “gojq does not keep the order of object keys” isn’t about ordering keys consistently across runs, it’s about keeping them in the order of the input file.

GP is correct, per the README:

> gojq does not keep the order of object keys. I understand this might cause problems for some scripts but basically, we should not rely on the order of object keys. Due to this limitation, gojq does not have keys_unsorted function and --sort-keys (-S) option. I would implement when ordered map is implemented in the standard library of Go but I'm less motivated.

And later in the same file:

  gojq does not support some functions intentionally;
  
  --sort-keys, -S (sorts by default because map[string]interface{} does not keep the order),

Re: Gojq: Pure Go Implementation of Jq

#70

Post by tialaramex »

hxtk wrote: post #46
xmonkee wrote: post #40

Earlier quoted context omitted.

I fucking hate this so much. Honestly Go isn't a bad language, but I dunno why these kind of things just piss me off.

It seems as though a lot of people view it as hypocritical, e.g., generics for me but not for thee (dated example since there are now generics for everyone). The fact that they needed to make a map a part of the language in order to allow it to be generic and statically-typed proves that generics are useful and should therefore have been a language feature much earlier than they became one. There are a variety of thi…

It's fair to just say "Well it's not for that" if you're not a general purpose language.

Like it sure is hard to write a grammar checker in WUFFS. Well, it's not for that, it's a special purpose language, it doesn't even have strings, stop trying to write a grammar checker.

For a general purpose language this is a poor excuse. I think the best argument might be a desire to avoid the Turing Tar-pit where everything is possible but nothing is easy. C++ allows you to write a type that's generic over the floating point values, so e.g. Foo. What does that mean? Nothing useful. But they could so they did.

In avoiding the Turing Tar-pit you must make some choices which weren't necessary but were, in your opinion (as Benevolent dictator, Steering Committee, Riotous Assembly Of Interested People, or whatever) more aesthetic, more practical for some particular purpose you had in mind, easier to implement or whatever.

My impression with Go, which I spent a few years programming but never really loved, was that it's main value was in being surprisingly efficient for that type of language. Particularly startup of Go is good, which would matter for gojq for example.

Post reply on HN