LZ77 compression/decompression in pure Bash using hashmaps:
Today I learned that bash has hashmaps (2024)
71–80 of 137 posts
Re: Today I learned that bash has hashmaps (2024)
#72Re: Today I learned that bash has hashmaps (2024)
#73Oh dear. I've been trying to get people to not use this feature for a while. One thing that has bitten me in the past is that, if you declare your associative arrays within a function, that associative array is ALWAYS global. Even if you declare it with `local -A` it will still be global. Despite that, you cannot pass an associative array to a function by value. I say "by value" because while you can't call `foo ${as…
I'm not seeing this local scope leak with bash 5.2.15. The below script works as I'd expect: #!/bin/bash declare -A map1=([x]=2) echo "1. Global scope map1[x]: ${map1[x]}" func1() { echo " * Enter func1" local -A map1 map1[x]=3 echo " Local scope map1[x]: ${map1[x]}" } func1 echo "2. Global scope map1[x]: ${map1[x]}" outputting 1. Global scope map1[x]: 2 * Enter func1 Local scope map1[x]: 3 2. Global scope map1[x]: 2
The local scope leak seems to only happen when you drop down the call stack. See below how I can call func2 from the top level and it's fine, but if I call it from within func1, it will leak.
#!/bin/bash
declare -A map1=([x]=2)
echo "1. Global scope map1[x]: ${map1[x]}"
func1() {
echo " * Enter func1"
local -A map1
map1[x]=3
echo " Local scope map1[x]: ${map1[x]}"
func2
}
func2() {
echo " * Enter func2"
echo " Local scope map1[x]: ${map1[x]}"
}
func1
func2
echo "2. Global scope map1[x]: ${map1[x]}"
UPDATE: I did a bit of exploration and it turns out ANY variable declared `local` is in the scope of a function lower down in the call stack. But if you declare a variable as `local` in a called function that shadows the name of a variable in a callee function, it will shadow the callee's name and reset the variable back to the vallee's value when the function returns. I have been writing bash for years and did not realise this is the case. It is even described in the man page: When local is used within a function, it causes the variable name to have a visible scope restricted to that function and its children.
Thank you. You have taught me two things today. One is a bash feature I did not know existed. The second is a new reason to avoid writing complex bash.
Re: Today I learned that bash has hashmaps (2024)
#74Earlier quoted context omitted.
>they have some fantastic foot guns! Otherwise wouldn't be getting the full shell experience.
Why hasn't shell scripting evolved? It's god-awful.
- bash is available pretty much everywhere, so if you learn it you can always use it, whereas if you learn a weird new shell you'll be occasionally forced to fall back on bash anyway, so people learn just bash for efficiency's sake (because learning one shell is painful enough as it is). And any proposed replacement will be non-portable.
- some of the things that make shell scripting terrible can't be fixed in the shell itself, and need changes to the entire ecosystem of console applications. e.g. it would be awesome if every Unix utility output structured data like JSON which could be parsed/filtered, instead of the soup of plaintext that has to be memorized and manipulated with awk, but that almost certainly won't happen. There's a bunch of backward-compatibility requirements like VT-100 and terminal escape sequences limiting the scope of potential improvements as well
- there's a great deal of overlap between "people who do a lot of shell scripting" and "people who are suspicious of New Stuff and reluctant to try it"
Re: Today I learned that bash has hashmaps (2024)
#75They are also slow AF because a lookup takes linear time.
Which is because they're not hashmaps, they're associative arrays. Article treats them as the same thing, but they're not - that's why the "declare -A" is A and not H - it stands for "associative".
Re: Today I learned that bash has hashmaps (2024)
#76Earlier quoted context omitted.
>they have some fantastic foot guns! Otherwise wouldn't be getting the full shell experience.
Why hasn't shell scripting evolved? It's god-awful.
https://www.nushell.sh/
I use this as my main shell on Windows, and as a supplementary on Mac and Linux.
Re: Today I learned that bash has hashmaps (2024)
#77They are also slow AF because a lookup takes linear time.
Which is because they're not hashmaps, they're associative arrays. Article treats them as the same thing, but they're not - that's why the "declare -A" is A and not H - it stands for "associative".
Author of the article here. As far as I care, if it quacks like a hashmap, it's better to describe it as a hashmap. The fact that it's an associative array under the hood is irrelevant.
Re: Today I learned that bash has hashmaps (2024)
#78Earlier quoted context omitted.
>they have some fantastic foot guns! Otherwise wouldn't be getting the full shell experience.
Why hasn't shell scripting evolved? It's god-awful.
What would you like to see?
I’d guess there’s a solution to almost any set of priorities you have for shell scripting.
The domain space is extremely challenging: by default, executing any program on behalf of the caller using arbitrary text inputs, and interpreted.
All modern shells allow calling of literally any binary using the #!/usr/bin/env randombinary incantation.
Upshot: bash has its place, and while some of that place is unearned inertia, much of it is earned and chosen often by experienced technologists.
In my case, if I’m doing a lot of gluing together of text outputs from binaries, bash is my go-to tool. It’s extremely fast and expressive, and roughly 1/4 the length of say python and 1/8 the length of say go.
If I’m doing a lot of logic on text: python. Deploying lots of places/different architectures: go
Re: Today I learned that bash has hashmaps (2024)
#79Don't loop on values with `*`; the difference key/value is the lack of `!` at the start of the expression; `*` and `@` rules are the sames a $@ and $* and you almost never want *.
Re: Today I learned that bash has hashmaps (2024)
#80I advocate the following rules for when to write and when not to write a shell script. # Write a shell script: * Heavy lifting done by powerful tool (sort, grep, curl, git, sed, find, …) * Script will glue diverse tools * Workflow resembles a pipeline * Steps can be interactively developed as shell commands * Portability * Avoid dependency hell * One-off job # Avoid shell scripting: * Difficult to see the preceding p…
A couple things:
- Between using Bash's functions and breaking large scripts into multiple, separate scripts, one can keep things reasonably tidy. Also, functions and scripts can be combined in all the ways (e.g., piping and process substition) that other programs can.
- If I run into a case where Bash is a poor fit for a job, I ask myself, "Self, what program would make this easy to do in a shell script?" If I can respond with a good answer, I write that, then continue with the shell scripting. If not, I write in something else (what the kids would call a "proper" language).