Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My deployment platform is a shell script

j3s.sh

Post reply on HN
Print view View original

71–80 of 141 posts

Re: My deployment platform is a shell script

#71

Post by mst »

adamtaylor_13 wrote: post #55
chasil wrote: post #33

The use of ls in this way is not good form: cd /root for project in $(ls go-cicd); I think a better expression would be: for project in ./* do [ -d "$project" ] || continue ...

Why is that not good form?

https://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.... describes the failure modes (and note that it's the very first pitfall because it's a common one - and one I've perpetrated plenty of times myself).

Given the situation and the project names, I wouldn't expect the use of `ls` described in TFA to ever be a problem, but doing it with a simple glob would still be nicer and is a good habit to get into overall since then you don't have to ask yourself "is this use of ls going to be safe?"

Re: My deployment platform is a shell script

#72

Post by cl3misch »

pfitzsimmons wrote: post #68

As a pythonista, I am a huge fan of the plumbum library as a replacement for bash. It makes it very straightforward to run a sequence of *nix commands, but you get all the simplicity and power of the python language in terms of loops and functions and so forth. These days, I do all my server management and deployment scripts with python/plumbum. And while simple is great, the necessary features not included in OP's s…

How do you deal with plumbum not being a builtin module? Do you install it system-wide? This currently holds me back from using sh (the Python lib) for maintaining my servers, especially if I need it with root.

Re: My deployment platform is a shell script

#73

Post by mst »

strzibny wrote: post #44

I agree that sometimes Bash is enough, which is what I show in Deployment from Scratch. However I am moving pretty much everything to Kamal now...

Guessing you're talking about https://kamal-deploy.org/ which looks interesting, though I tend to like reconciliation logic based systems ... but often only fired off imperatively with a plan/apply separation. So I shall be having a poke around anyway :)

Re: My deployment platform is a shell script

#74

Post by tacone »

I use this deploy script for my hobby project: https://gist.github.com/tacone/230d5c305a9c5eff7f58ea2744f20...

It will connect over ssh, pull the code, build the containers and restart them (scripts/live is just a wrapper around docker-compose).

If the build fails, the services will keep running.

The only problem I have is that hitting CTRL+C in the very moment the containers are being restarted will leave me with the services down.

Re: My deployment platform is a shell script

#75

Post by kragen »

here's the deployment script i use most often

http://canonical.org/~kragen/sw/dev3.git/hooks/post-update

    #!/bin/sh
    set -e

    echo -n 'updating... '
    git update-server-info
    echo 'done. going to dev3'
    cd /home/kragen/public_html/sw/dev3
    echo -n 'pulling... '
    env -u GIT_DIR git pull
    echo -n 'updating... '
    env -u GIT_DIR git update-server-info
    echo 'done.'
dev3.git is the origin for dev3, so the `git pull` in there pulls from the bare repo that just got pushed to

it doesn't have the 60-second lag and it doesn't load the server all the time. it also doesn't run `go build` or restart a server with openrc, but those would be easy things to add if i wanted them

Re: My deployment platform is a shell script

#76

Post by pfitzsimmons »

cl3misch wrote: post #72
pfitzsimmons wrote: post #68

As a pythonista, I am a huge fan of the plumbum library as a replacement for bash. It makes it very straightforward to run a sequence of *nix commands, but you get all the simplicity and power of the python language in terms of loops and functions and so forth. These days, I do all my server management and deployment scripts with python/plumbum. And while simple is great, the necessary features not included in OP's s…

How do you deal with plumbum not being a builtin module? Do you install it system-wide? This currently holds me back from using sh (the Python lib) for maintaining my servers, especially if I need it with root.

That's not a big deal for me since I only am running a handful of servers. I install it system-wide during initial setup of a new server. Plumbum has the ability to run remote shell commands as well, so I have a script that can login to a new remote machine and do that initial setup.

Re: My deployment platform is a shell script

#77

Post by kragen »

mattbuilds wrote: post #54
cqqxo4zV46cp wrote: post #41

Earlier quoted context omitted.

[flagged]

It’s funny because in my many years of development I don’t think I’ve ever encounter a “mess of shell scripts” that was difficult to maintain. They were clear, did their job and if they needed to be replaced it was usually simple and straightforward. Can’t say the same for whenever the new abstraction of the day comes along. In my experience what the OP is saying is exactly my experience. The abstractions get picked…

i have encountered messes of shell scripts that were difficult to maintain; in my first sysadmin job in 01996 i inherited a version control system written as a bunch of csh scripts, built on top of rcs

but they were messy not because they lacked 'abstractions' but because they had far too many

i think shell scripts are significantly more bug-prone per line than programs in most other programming languages, but if the choice is hundreds of thousands of lines in an external dependency, or a ten-line or hundred-line shell script, it's easy for the shell script to be safer

Re: My deployment platform is a shell script

#78

Post by redman25 »

mattbuilds wrote: post #54
cqqxo4zV46cp wrote: post #41

Earlier quoted context omitted.

[flagged]

It’s funny because in my many years of development I don’t think I’ve ever encounter a “mess of shell scripts” that was difficult to maintain. They were clear, did their job and if they needed to be replaced it was usually simple and straightforward. Can’t say the same for whenever the new abstraction of the day comes along. In my experience what the OP is saying is exactly my experience. The abstractions get picked…

Shell seems great until your tens of lines in googling every other line of obscure error-prone syntax.

Re: My deployment platform is a shell script

#79

Post by kragen »

forinti wrote: post #66
cqqxo4zV46cp wrote: post #41

Earlier quoted context omitted.

[flagged]

Just yesterday I was working on a monitoring agent that was a python script deployed through gitlab-ci, and executed inside a docker container through logstash. I bet you could do the same with about 5 lines of shell (wget|grep|curl -XPOST). It would be simpler, easier to modify, and it wouldn't ever break. Shell scripts aren't necessarily messy or complex.

python is a particularly bad choice for things that are not supposed to break; the python maintainers have adopted a new policy of intentionally breaking new things every release. recent casualties include asyncore, distutils, and imp

https://docs.python.org/3/whatsnew/3.12.html#removed

there are people who foolishly depend on python for things that need to work. i hope that at some point they band together behind a fork that doesn't pull this kind of nonsense, but for the time being, that hasn't happened. if you wrote stuff in python in 01995 or 02000 or 02005 it probably still worked in 02015, but today, basically no python from before 02015 works

Re: My deployment platform is a shell script

#80

Post by candiddevmike »

runlaszlorun wrote: post #56
cqqxo4zV46cp wrote: post #41

Earlier quoted context omitted.

[flagged]

I don’t think ‘axe to grind’ was what I took away from the comment. And if it’s a ‘mess of shell scripts’, I can’t imagine any of the mentioned solutions to this being better. I think we lose focus of the fact that we invented a whole new career track- devops. And it basically is about getting code from developers’ machine or environments to production. And we’re still terrible at it. Perhaps that should be a signal…

There is a constellation of companies (living and dead) that have attempted to solved this problem in different ways because everyone thinks we're doing it wrong. The reality is every company has different needs. There isn't (will never be?) a one-size-fits-all approach to deployments because they're the fun intersection of people/process/technology.

The best deployment pipeline is the one that no one notices. Everything else is yak shaving/bike shedding.

Post reply on HN