Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jo – a shell command to create JSON (2016)

jpmens.net

Post reply on HN
Print view View original

81–90 of 100 posts

Re: Jo – a shell command to create JSON (2016)

#81

Post by masklinn »

matja wrote: post #41
pydry wrote: post #39

Earlier quoted context omitted.

The problem is avoiding your surprise ruined Sunday night three months from now is predicated on you not forgetting to put the -s in. Implicit typing sucks : https://www.destroyallsoftware.com/talks/wat

Agreed, luckily there is a solution ;)

It’s not a solution, it’s a work-around. A solution would be fixing this issue. This allows working around the issue if you remember that it exists.

It’s like saying that it’s fine if an appliance might randomly burn your house, because there’s a button you can press to not burn your house.

Re: Jo – a shell command to create JSON (2016)

#82

Post by timwis »

jkbr wrote: post #15
seletskiy wrote: post #8

Some time ago I wrote zsh helper [1] that uses jo to quickly construct complex JSON requests, mainly for testing and quering services from console. Paired with httpie [2] aliases [3] it produces concise APL-like syntax: POST https://httpbin.org/post test:=j`a=b c=`e=3` l=`*1 2 3`` Which translates to: http POST https://httpbin.org/post test:="$(jo a=b c="$(jo e=3)" l="$(jo -a 1 2 3)")" Or, in other words, sending POS…

HTTPie creator here. We’ve recently[0] added support for nested JSON[1] to the HTTPie request language, so you can now craft complex JSON request directly: $ http pie.dev/post test[a]=b test[c][e]:=3 test[l][]:=1 [0] https://httpie.io/blog/httpie-3.0.0 [1] https://httpie.io/docs/cli/nested-json

jo immediately reminded me of httpie’s CLI syntax. Is it just a coincidence, or does httpie use jo under the hood?

Btw loving your new desktop app so far!

Re: Jo – a shell command to create JSON (2016)

#83

Post by fooyc » 

    > jo normally treats value as a literal string value, unless it begins with one of the following characters:  
    > value  action
    > @file  substitute the contents of file as-is
    > %file  substitute the contents of file in base64-encoded form
    > :file  interpret the contents of file as JSON, and substitute the result
This is convenient but also very dangerous. This feature will cause the content of an arbitrary file to be embed in the JSON if any value starts with a `@`, `%`, or `:`.

This will be a source of bugs or security issues for any script generating json with dynamic values.

Re: Jo – a shell command to create JSON (2016)

#84

Post by ZeroGravitas »

EdSchouten wrote: post #29

I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…

Often when writing scripts, I'm chaining tools together, e.g. using git to find a thing in a specific commit, using curl to grab something from the web, decoding some json, maybe unzipping a file.

I've never really found any language that feels good for that kind of thing, there's definately a middle ground where it's getting too much for bash, but jumping to a language loses too much at the initial conversion to make it feel worth it until you are well past the point where your future self will think you should have made the switch.

Some languages have things like backticks in php to inter-operate but it's still not great experience to mix between them. For my own little things I'm currently looking at fish, but bash is omnipresent.

This tool seems to delay that point even further, as currently dealing with generating json is definately a pain point (whereas manipulating it in jq is often really good).

But if anyone can point to good examples of this transition in python then I'd be very interested.

edit: jq is more powerful than I thought for creating json, see https://spin.atomicobject.com/2021/06/08/jq-creating-updatin...

Re: Jo – a shell command to create JSON (2016)

#85

Post by jkbr »

timwis wrote: post #82
jkbr wrote: post #15

Earlier quoted context omitted.

HTTPie creator here. We’ve recently[0] added support for nested JSON[1] to the HTTPie request language, so you can now craft complex JSON request directly: $ http pie.dev/post test[a]=b test[c][e]:=3 test[l][]:=1 [0] https://httpie.io/blog/httpie-3.0.0 [1] https://httpie.io/docs/cli/nested-json

jo immediately reminded me of httpie’s CLI syntax. Is it just a coincidence, or does httpie use jo under the hood? Btw loving your new desktop app so far!

We did look at `jo`, and also `jarg`[0], the W3C HTML JSON form syntax[1], and pretty much every other approach we could find. We had quite a few requirements that the new nested syntax had to meet: be simple/flexible, easy to read/write, backward/forward compatible, and play well with the rest of the HTTPie request language.

The final syntax is heavily inspired by the HTML JSON forms one. But we made it stricter, added type safety, and some other features. It also supports all the existing functionality like embedding raw JSON strings and JSON/text files by paths.

The final implementation[2] is completely custom. We have plans to ship it as a standalone tool as well, publish the test suite in a language-independent format and write a formal spec so that other tools can easily adopt it. This spec will eventually be a subset of one for the overall HTTPie request language, which is currently tied to our CLI implementation but we want to decouple it.

Happy to hear you like the desktop app!

[0] https://github.com/jdp/jarg

[1] https://www.w3.org/TR/html-json-forms/

[2] https://github.com/httpie/httpie/blob/master/httpie/cli/nest... — this is the path parser

Re: Jo – a shell command to create JSON (2016)

#86

Post by bb88 »

ZeroGravitas wrote: post #84
EdSchouten wrote: post #29

I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…

Often when writing scripts, I'm chaining tools together, e.g. using git to find a thing in a specific commit, using curl to grab something from the web, decoding some json, maybe unzipping a file. I've never really found any language that feels good for that kind of thing, there's definately a middle ground where it's getting too much for bash, but jumping to a language loses too much at the initial conversion to mak…

Typically if you have bash you have a bunch of other utilities installed too.

The problem with python here is that while python-sh might be nice, you have to install any extra libraries you need with it, and that's not a trivial problem for installing scripts into prod.

Xonsh is better since you kind of get both the benefits of python and a shell like language, but frankly it's broken in a number of ways still. I use it daily, but hopefully you don't need to ctrl-c out of something since signal handling is iffy at best. It is kind of nice to be able to import python directly on the command line and use it as python though...

Re: Jo – a shell command to create JSON (2016)

#87

Post by bb88 »

zimpenfish wrote: post #35
EdSchouten wrote: post #29

I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…

> Even though Python isn't the fastest language out there, it's likely still faster than the shell command above. Taking these two command lines: jo -p name=JP object=$(jo fruit=Orange point=$(jo x=10 y=20) number=17) sunday=false >/dev/null python -c 'import json;print(json.dumps({"name": "JP", "object": {"fruit": "Orange", "point": {"x": 10, "y": 20}, "number": 17}, "sunday": False}))' >/dev/null For jo (x86_64, Ro…

I'm not disagreeing that python is slow, but why would you choose to do either in a shell script?

    $ time cat

Re: Jo – a shell command to create JSON (2016)

#88

Post by bb88 »

deathanatos wrote: post #52
detaro wrote: post #50

Earlier quoted context omitted.

That's not really an issue for small scripts, no.

Since I have no idea why this is downvoted: it really isn't for small scripts: if you just ignore pip/packages, Python is going to give you way more functionality out of the box than shell would, with a lot fewer sharp corners that will translate into a less buggy/more correct script at the end of day. If you do take the time to deal with pip (which, yes, is a problem) you get access to even more batteries that would…

It's behavior is really well defined and will stop on a syntax error, type error, or some other error that wasn't handled.

Bash on error will just move on to the next line as if nothing ever happened unless you:

    set -eu
    set -o pipefail

Re: Jo – a shell command to create JSON (2016)

#89

Post by matja »

masklinn wrote: post #81
matja wrote: post #41

Earlier quoted context omitted.

Agreed, luckily there is a solution ;)

It’s not a solution, it’s a work-around. A solution would be fixing this issue. This allows working around the issue if you remember that it exists. It’s like saying that it’s fine if an appliance might randomly burn your house, because there’s a button you can press to not burn your house.

I'm sure it'd be a patch of a few lines to make the type specification mandatory on the command line (I would certainly prefer that also), but it comes down to the opinion of the maintainer if that is wanted or not.

Re: Jo – a shell command to create JSON (2016)

#90

Post by zimpenfish »

bb88 wrote: post #87
zimpenfish wrote: post #35

Earlier quoted context omitted.

> Even though Python isn't the fastest language out there, it's likely still faster than the shell command above. Taking these two command lines: jo -p name=JP object=$(jo fruit=Orange point=$(jo x=10 y=20) number=17) sunday=false >/dev/null python -c 'import json;print(json.dumps({"name": "JP", "object": {"fruit": "Orange", "point": {"x": 10, "y": 20}, "number": 17}, "sunday": False}))' >/dev/null For jo (x86_64, Ro…

I'm not disagreeing that python is slow, but why would you choose to do either in a shell script? $ time cat

> why would you choose to do either in a shell script?

In the normal case, you'd have variables interpolated in there, not static JSON. And then you run into the quoting problems that jo was created to work around...

Post reply on HN