Live data from Hacker News

FX: An interactive alternative to jq to process JSON

github.com

41–50 of 65 posts

Re: FX: An interactive alternative to jq to process JSON

#41

Earlier quoted context omitted.

> or you get only one set of outputs per key (iyswim) because they get overlapped in memory That sounds like a major bug. So it will silently skip data that you wanted?

> That sounds like a major bug. It's definitely an oddness when you have multiple objects at the same level that aren't in an array but I guess the explanation there is "they should all be on their own individual lines as streaming json" which `gron` does handle correctly. (echo '{"a":"23"}'; echo '{"a":"25"}') | gron -s json = []; json[0] = {}; json[0].a = "23"; json[1] = {}; json[1].a = "25"; > So it will silently…

Had a look at the source and I think I've figured out why and maybe how to fix it. Will have a bash at making a PR this week.

Re: FX: An interactive alternative to jq to process JSON

#43

Earlier quoted context omitted.

> I'll think about whether to add an entries function or a map function that would allow doing this in a simple way. That would be super, ta. `to_entries[]` is pretty much the major reason I've not managed to move off `jq` to anything else yet because it's just incredibly powerful in this situation.

I've actually just gone ahead and added a way to do this - the zip function - in the v0.2.0 release. The relevant jql snippet to solve this in the general case now is: (pipe (zip (keys) ((keys))) ((keys) (object "key" (0) "value" (1)))) It's not as terse as the jq equivalent - I'll probably add a way to create user-defined functions, so you can alias stuff like this to shorter forms - but that one will require more t…

Wasn't expecting such a quick (if any!) response! Excellent, ta. That gives me the same output from my file as jq does with `to_entries[]`.

Unfortunately my next issue is how do I iterate over an array of objects (like jq `.[]`)? I'm guessing it's maybe something to do with `range` but I don't know how many I have in order to fill in those indices and I can't do `(elem 0) ... (elem 1)` for the same reason.

Re: FX: An interactive alternative to jq to process JSON

#45
post #37

That's pretty cool. Would be nice if there was an option to put the currently used filter/query into the shell history or the clipboard. So that you could experiment to find the right one, then back out and use it in a pipeline.

Cat xxx.json | fx 'whatever' | fx

Is how I usually do so, if you don't want the interactive mode drop the last pipe

Re: FX: An interactive alternative to jq to process JSON

#46

Earlier quoted context omitted.

I've actually just gone ahead and added a way to do this - the zip function - in the v0.2.0 release. The relevant jql snippet to solve this in the general case now is: (pipe (zip (keys) ((keys))) ((keys) (object "key" (0) "value" (1)))) It's not as terse as the jq equivalent - I'll probably add a way to create user-defined functions, so you can alias stuff like this to shorter forms - but that one will require more t…

Wasn't expecting such a quick (if any!) response! Excellent, ta. That gives me the same output from my file as jq does with `to_entries[]`. Unfortunately my next issue is how do I iterate over an array of objects (like jq `.[]`)? I'm guessing it's maybe something to do with `range` but I don't know how many I have in order to fill in those indices and I can't do `(elem 0) ... (elem 1)` for the same reason.

Not sure if you've gone through the README - especially the first few paragraphs should help you get an intuition on how to structure nested jql queries.

Basically, you can think about the query as a composition of many functions which result in one big function taking in your JSON and outputting a new JSON.

When you do ("mykey") or (0) you dive in one level deeper. You can also transform what is that one level deeper by writing ("mykey" (mytransform)). There is a keys function which returns the list of keys or the list of indices, for the current object or list, respectively. And you can use those lists of indices for indexing purposes.

Thus, if you have an input list and want to transform it element by element, you can write ((keys) (my-single-element-transformer)). It gets the indices, uses them as an index, and transforms each object contained in the list.

So let's say you have a list of objects {"name": "abc", "surname": "xyz"} and would like to transform them into a list of {"abc": "xyz"}. You can write ((keys) (object ("name") ("surname"))). This goes over all elements and for each returns a single object with a key that is the name (it's actually a transformer/continuation which gets the name from the current object that we pass there) and value that is the surname.

You can also see that in the original "entries" query. It first zips the keys with the values, so for a list of {"mykey": "myvalue}, it will give you a list of lists ["mykey", "myvalue"]. Then it pipes that into another transform, which for each such pair creates an object {"key": "", "value": ""}.

The overall system isn't that straightforward at first, but playing around with it for a while should make it click and then it's easy to write even more complex queries.

Re: FX: An interactive alternative to jq to process JSON

#48
post #45
post #37

That's pretty cool. Would be nice if there was an option to put the currently used filter/query into the shell history or the clipboard. So that you could experiment to find the right one, then back out and use it in a pipeline.

Cat xxx.json | fx 'whatever' | fx Is how I usually do so, if you don't want the interactive mode drop the last pipe

Not sure I understand. I mean using the interactive mode to figure out the right query ("[].mumble.whatever...") and then being able to save the text of the query that you figured out by pointing and clicking. Like a graphical SQL query builder allows you to do.

Re: FX: An interactive alternative to jq to process JSON

#49
post #5
post #3

Apparently, the author wrote this tool because jid was struggling with a 7MB JSON file. See https://github.com/simeji/jid/issues/66#issuecomment-4436718...

Just tried out both of these for a large endpoint. - FX "expand/collapse" functionality seems way better for exploring APIs whose shape you don't know - jid is maybe marginally better for APIs where you have instant recall of the exact shape and need to rapidly query it Overall, I like FX better because it provides feedback on your query faster. I am grateful to the author(s) for creating it and I'll be using it inst…

FYI, fx has a jid-like mode that can be started by pressing the dot key. See this post[1] for more information.

[1] https://medium.com/@antonmedv/discover-how-to-use-fx-effecti...

Re: FX: An interactive alternative to jq to process JSON

#50
post #48
post #45

Earlier quoted context omitted.

Cat xxx.json | fx 'whatever' | fx Is how I usually do so, if you don't want the interactive mode drop the last pipe

Not sure I understand. I mean using the interactive mode to figure out the right query ("[].mumble.whatever...") and then being able to save the text of the query that you figured out by pointing and clicking. Like a graphical SQL query builder allows you to do.

You can do this with `jid -q`: https://github.com/simeji/jid

(Its interactivity is keyboard- rather than mouse-based.)

Post reply on HN