Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jo – a shell command to create JSON (2016)

jpmens.net

Post reply on HN
Print view View original

71–80 of 100 posts

Re: Jo – a shell command to create JSON (2016)

#71

Post by 1vuio0pswjnm7 »

EdSchouten wrote: post #29

I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…

"Even though Python isn't the fastest language out there, it's likely still faster than the shell command above."

That is going a bit far. By all means use Python. Go ahead and attack people who use the shell. But let's be honest. The shell is faster, assuming one knows how to use it. A similar claim is often made by Python advocates, something along the lines of Python is not slow if one knows how to use it.

The startup time of a Python interpreter is enormous for someone who is used to a Bourne shell. This is what always stops me from using Python as a shell replacement for relatively simple jobs; I have never written a large shell script and doubt I ever will. I write small scripts.

If anyone knows how to mitigate the Python startup delay, feel free to share. I might become more interested in Python.

Anyway, this "jo" thing seems a bit silly. Someone at Google spent their 20% time writing a language called jsonnet to emit JSON. It has been discussed on HN before. People have suggested dhall is a perhaps better alternative.

https://jsonnet.org

https://dhall-lang.org

Re: Jo – a shell command to create JSON (2016)

#72

Post by nfw2 »

viraptor wrote: post #59
nfw2 wrote: post #54

Earlier quoted context omitted.

I feel this whenever anyone advocates using jq. It boggles my mind that anyone would want to learn a whole new DSL for something that js makes trivial, especially considering js a much more expressive scripting language anyway

JS is not that easy to embed in scripting though for trivial situations. After seeing a couple of jq examples I can run `jq '.foo[] | {bar, baz}' without really "learning" its DSL. But doing the same with node? That would be much larger.

And now everyone who needs to read your script needs to also find those examples to figure out what your code does. Even your not-real-world example isn't obvious to anyone unfamiliar with jq what the intent is

Re: Jo – a shell command to create JSON (2016)

#74

Post by Too »

visarga wrote: post #55

You can implement this and many other single purpose CLI tools with inline Python or Perl or other language. Easier to remember because it's your favorite language. python -c "import json; print(json.dumps(dict(a=10, b=False)))"

While a handy trick, it doesn't entirely solve the original problem when the values have dynamic content. In your example, replace 10 with $FOO and then you are back to square one, having to escape python-strings within a shell-string. Better avoid the problem entirely by not using shell to begin with. To instead continue on the dirty track, replace 10 with int(sys.argv[1]) and call it as python -c "..." $FOO.

Re: Jo – a shell command to create JSON (2016)

#75

Post by Too »

jsnelgro wrote: post #67
EdSchouten wrote: post #29

I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…

I just assume at this point that a new python script from a coworker won’t run without an hour of tinkering and yelling obscenities at my screen. Or resorting to running in docker, which seems asinine. Python’s everywhere and does everything though, so I don’t have a good alternative. Shell scripts definitely aren’t it, but they generally hold up better when sharing in my experience.

Shell scripts can't declare dependencies though, in the same way a pip package can. A shell script using this tool requires one to manually apt install it first, or run in a common docker image - asinine. If you don't, your script will fail halfway through during runtime (actually, likely it will not fail, just produce corrupted output, since shell by default ignores errors), a python IDE or mypy will tell you about missing packages during analysis before you try to build and run it.

Besides that, looking at json only, it's part of the standard library so is more likely to already exist on any given machine rather than this.

Re: Jo – a shell command to create JSON (2016)

#76

Post by viraptor »

nfw2 wrote: post #72
viraptor wrote: post #59

Earlier quoted context omitted.

JS is not that easy to embed in scripting though for trivial situations. After seeing a couple of jq examples I can run `jq '.foo[] | {bar, baz}' without really "learning" its DSL. But doing the same with node? That would be much larger.

And now everyone who needs to read your script needs to also find those examples to figure out what your code does. Even your not-real-world example isn't obvious to anyone unfamiliar with jq what the intent is

That's about the only example you need explained to understand ~99% of real world jq usage. $dayjob has quite a bit of it around in various repos, and this actually is as real-world as it gets in my experience. Comparing that to having to learn enough JS to do the same thing in a verbose way, I'm still on the side of jq having an advantage in that case.

Re: Jo – a shell command to create JSON (2016)

#77

Post by johncs »

In the common case where you trust your input entirely you can just interpret your string as JavaScript. Then you don't even need to use quotes for the key names.

    $ alias fooson="node --eval \"console.log(JSON.stringify(eval('(' + process.argv[1] + ')')))\""
    $ fooson "{time: $(date +%s), dir: '$HOME'}"
    {"time":1457195712,"dir":"/Users/jpm"}
It may be a bit nicer to place that JavaScript in your path as a node script instead of using an alias.

    #!/usr/bin/env node
    console.log(JSON.stringify(eval('(' + process.argv[2] + ')')))
Since fooson's argument is being interpreted as JavaScript, you can access your environment through process.env. But you could make a slightly easier syntax in various ways. Like with this script:

    #!/usr/bin/env node
    for(const [k, v] of Object.entries(process.env)) {
        if (!global.hasOwnProperty(k)) {
            global[k] = v;
        }
    }
    console.log(JSON.stringify(eval('(' + process.argv[2] + ')')))
Now environmental variables can be access as if they were JS variables. This can let you handle strings with annoying quoting.

    $ export BAR="\"'''\"\""
    $ fooson '{bar: BAR}'
    {"bar": "\"'''\"\""}
If you wanted to do this without trusting your input so much, a JSON dialect where you can use single-quoted strings would get you pretty far.

    $ fooson "{'time': $(date +%s), 'dir': '$HOME'}"
    {"time":1457195712,"dir":"/Users/jpm"}
If you taught the utility to expand env variables itself you'd be able to handle strings with mixed quoting as well.

    $ export BAR="\"'''\"\""
    $ fooson '{"bar": "$BAR"}'
    {"bar": "\"'''\"\""}
You'd only need small modifications to a JSON parser to make this work.

Re: Jo – a shell command to create JSON (2016)

#78

Post by c0npr »

jsnelgro wrote: post #67
EdSchouten wrote: post #29

I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…

I just assume at this point that a new python script from a coworker won’t run without an hour of tinkering and yelling obscenities at my screen. Or resorting to running in docker, which seems asinine. Python’s everywhere and does everything though, so I don’t have a good alternative. Shell scripts definitely aren’t it, but they generally hold up better when sharing in my experience.

That depends on how well your bash script is constructed. If you carefully handle the falling case such as missing commands, non-root permissions, etc. It can be easy to use and kind of portable. Of course python scripts have better error trace so if the script doesnt work others can debug with relatively easily.

Re: Jo – a shell command to create JSON (2016)

#79

Post by jasfi »

EdSchouten wrote: post #29

I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…

Although good advice, this is also an area where Nim could shine.

Re: Jo – a shell command to create JSON (2016)

#80

Post by masklinn »

alexander-litty wrote: post #24
pydry wrote: post #19

>Bam! Jo tries to be clever about types and knows null, booleans, strings and numbers. I'm very skeptical of this. If I put x=001979 in as a value I dont think I want you trying to guess if that's supposed to be an integer or a string. This sounds like the Norway Problem waiting to happen.

It looks like you can specify the value type per property, for those cases where it matters.

Opt-in safety has such a great track record after all.
Post reply on HN