41–50 of 100 posts
Earlier quoted context omitted.
jo -- -s a=123 -s b=00123 -s c="$(date)" {"a":"123","b":"00123","c":"Sat 5 Feb 21:29:08 GMT 2022"} What's the issue?
The problem is avoiding your surprise ruined Sunday night three months from now is predicated on you not forgetting to put the -s in. Implicit typing sucks : https://www.destroyallsoftware.com/talks/wat
I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…
> Even though Python isn't the fastest language out there, it's likely still faster than the shell command above. Taking these two command lines: jo -p name=JP object=$(jo fruit=Orange point=$(jo x=10 y=20) number=17) sunday=false >/dev/null python -c 'import json;print(json.dumps({"name": "JP", "object": {"fruit": "Orange", "point": {"x": 10, "y": 20}, "number": 17}, "sunday": False}))' >/dev/null For jo (x86_64, Ro…
$ time python3 -c ''
real 0m0.029s
$ time python2 -c ''
real 0m0.010s
$ time bash -c ''
real 0m0.001s
Earlier quoted context omitted.
> Even though Python isn't the fastest language out there, it's likely still faster than the shell command above. Taking these two command lines: jo -p name=JP object=$(jo fruit=Orange point=$(jo x=10 y=20) number=17) sunday=false >/dev/null python -c 'import json;print(json.dumps({"name": "JP", "object": {"fruit": "Orange", "point": {"x": 10, "y": 20}, "number": 17}, "sunday": False}))' >/dev/null For jo (x86_64, Ro…
Now put a thousand of those JSON objects in a list, invoking jo for every element.
Example for jo:
docker run --rm -it debian bash
apt update && apt install -y jo nano
nano bash-loop.sh && chmod +x bash-loop.sh
#!/bin/bash
for ((i=0;i/dev/null
docker run --rm -it debian bash
apt update && apt install -y python3 nano
nano python-loop.py
import json
for i in range(1000):
print(json.dumps({"name": "JP", "object": {"fruit": "Orange", "point": {"x": 10, "y": 20}, "number": 17}, "sunday": False}))
time python3 python-loop.py >/dev/null
Debian GNU/Linux 11 (bullseye)
jo 1.3
Python 3.9.2
real 0m2.230s
user 0m1.106s
sys 0m1.076s
real 0m0.027s
user 0m0.021s
sys 0m0.005s
Note: jo seems to pretty print because of the "-p" parameter, which is not the case with Python, might not be a 1:1 comparison in this case. Would be better to remove it. Though when i did that, the performance improvement was maybe 1%, not significant.
Admittedly, it would be nice to test with actually random data to make sure that nothing gets optimized away, such as just replacing one of the numbers in JSON with a random value, say, the UNIX timestamp. But then you'd have to prepare all of the data beforehand (to avoid differences due to using Python to get those timestamps, or one of the GNU tools), or time the execution separately however you wish.
Edit to explain my rationale: Why bother doing this? Because i disagree with the sibling comment:
> The claim was that the Python snippet would be quicker than the jo snippet.
In my eyes that's almost meaningless, since in practice when you'll actually care about the runtimes will be when working with larger amounts of data, or alternatively really large files. Therefore this should be tested, not just the startup times, which become irrelevant in most real world programs, except for cases when you'd make a separate invocation per request, which you sometimes shouldn't do.
Edit #2: here's a lazy edit that uses the UNIX time and makes the data more dynamic, ignoring the overhead to retrieve this value, to get a ballpark figure.
Use time value for jo:
jo -p name=JP object=$(jo fruit=Orange point=$(jo x=10 y=20) number=$(date +%s)) sunday=false
import time
...
print(json.dumps({"name": "JP", "object": {"fruit": "Orange", "point": {"x": 10, "y": 20}, "number": int(time.time())}, "sunday": False}))
real 0m2.794s
user 0m1.422s
sys 0m1.313s
real 0m0.027s
user 0m0.020s
sys 0m0.006s
Edit #3: probably should have started with a test to verify whether the initially observed performance differences (Python being slower due to startup time) were also present.
Single iteration results for jo:
real 0m0.003s
user 0m0.000s
sys 0m0.002s
real 0m0.022s
user 0m0.017s
sys 0m0.004s
I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…
And this lets you write json without (or with less) braces, commas and quotes. That alone is already a big win.
Does it suffer from the Norway problem?
I think you're thinking of YAML; JSON doesn't interpret "no" as a boolean. Scroll down here to see the JSON grammar: https://www.crockford.com/mckeeman.html I actually think the "Norway problem" is a PEBKAC from users not learning the data format. But this tool may confuse some people or applications who don't know what a boolean, integer, float or string are, and try to mix types when the program reading them wasn't…
Should “a” be number or a string in the resulting JSON?
And if it’s number, how can I tell it to output a string?
This is cool. In the spirit of "do one thing well", I'd so rather use this to construct JSON payloads to curl requests than the curl project's own "json part" proposal[1] under consideration. [1]: https://github.com/curl/curl/wiki/JSON#--jp-part
Agree, I was surprised that the cURL feature was considered as it seems to go against the "Do One Thing" and composability points of the UNIX philosophy.
curl looks like it has hundreds of flags.
I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…
echo '{"name": "JP", "object": {"fruit": "Orange", "point": {"x": 10, "y": 20}, "number": 17}, "sunday": false}'
I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…
Earlier quoted context omitted.
Now put a thousand of those JSON objects in a list, invoking jo for every element.
I actually did that for a more realistic comparison. Example for jo: docker run --rm -it debian bash apt update && apt install -y jo nano nano bash-loop.sh && chmod +x bash-loop.sh #!/bin/bash for ((i=0;i /dev/null Example for Python 3: docker run --rm -it debian bash apt update && apt install -y python3 nano nano python-loop.py import json for i in range(1000): print(json.dumps({"name": "JP", "object": {"fruit": "Or…
I remember that when I worked at Google about a decade ago, there was this common saying: "If the first version of your shell script is more than five lines long, you should have written it in Python." I think there's a lot of truth in that. None of the examples presented in the article look better than had they been written in some existing scripting/programming language. In fact, had they been written in Python or…
Sure, but using Python gives you 100 more problems. Would you like to set pip and your package manager on a fight to the death? Or is today the day you learn all about venv? Might as well use Docker. Oh nice, my 600MB script is ready!