Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My deployment platform is a shell script

j3s.sh

Post reply on HN
Print view View original

121–130 of 141 posts

Re: My deployment platform is a shell script

#121

Post by Danjoe4 »

from-nibly wrote: post #57
mattbuilds wrote: post #54

Earlier quoted context omitted.

It’s funny because in my many years of development I don’t think I’ve ever encounter a “mess of shell scripts” that was difficult to maintain. They were clear, did their job and if they needed to be replaced it was usually simple and straightforward. Can’t say the same for whenever the new abstraction of the day comes along. In my experience what the OP is saying is exactly my experience. The abstractions get picked…

Hello. I have found the mess of shell scripts. Please don't do this. I was able to deal with the weird skaffold mess by getting rid of it, and replacing it with argocd. I was able to get rid of jenkins by migrating to github actions. I have yet to replace the magic servers with magic bash scripts. They take just enough effort that i can't spend the time. Use a tool i can google. If your bash script is really this str…

Bash is good but can grow out of control. The problem is solo engineers and managers who push/approve 500+ line bash scripts that do way too much. A good engineer will say it's getting too complicated and reimplement it in Python.

Re: My deployment platform is a shell script

#122

Post by icedchai »

kragen wrote: post #118
icedchai wrote: post #98

Earlier quoted context omitted.

Perhaps don't upgrade your Python willy-nilly? That seems like a sane solution to this problem.

debian stable no longer has python 2. debian's pypy package has its own fork of cpython 2 in order to be able to build itself. numerous other projects are doing similar things debian's versioning system for packages doesn't really contemplate needing to install old versions of a package because the maintainers are deliberately breaking new versions of it, so in 'bookworm' debian 12.5, the only cpython available to in…

My "willy-nilly" comment meant "don't upgrade it without planning in advance" not "don't upgrade it, period." Part of that planning can be choosing to stay on an older LTS distro until you have time to properly migrate your code to a Python released in this decade.

Re: My deployment platform is a shell script

#123

Post by kragen »

icedchai wrote: post #122
kragen wrote: post #118

Earlier quoted context omitted.

debian stable no longer has python 2. debian's pypy package has its own fork of cpython 2 in order to be able to build itself. numerous other projects are doing similar things debian's versioning system for packages doesn't really contemplate needing to install old versions of a package because the maintainers are deliberately breaking new versions of it, so in 'bookworm' debian 12.5, the only cpython available to in…

My "willy-nilly" comment meant "don't upgrade it without planning in advance" not "don't upgrade it, period." Part of that planning can be choosing to stay on an older LTS distro until you have time to properly migrate your code to a Python released in this decade.

or you could just write the code in the first place in a language that doesn't have maintainers that intentionally break old code. that's what you would do if you aspire to write code that constitutes part of the intellectual heritage of humanity, as opposed to being an expedient way to resolve a problem you have this week, to be thrown away in a month or two. you can still compile c code that uses gets(), you know. a couple of years ago i fixed a perl cgi script i'd written in 01998 to work with current perl; it took me half an hour. and euclid's proofs are still valid 2500 years after he wrote them down

Re: My deployment platform is a shell script

#124

Post by chasil »

kragen wrote: post #116
chasil wrote: post #89

Earlier quoted context omitted.

If it was in RCS, then you could directly move the archives under a CVSROOT and use them natively. CVS had been out since Brian Berliner's version of 1989. I actually moved a PVCS archive into RCS->CVS this way, and I'm still using it.

that version control system provided a number of facilities cvs didn't (locking, and also a certain degree of integration with our build system permitting the various developers to only recompile the part of the system they were working on, which was important because recompiling the whole thing usually took me about a week, once a month), but it had never actually occurred to me that turning an rcs repository into a…

PVCS did not use the RCS format, but the RPM distribution included a perl script to convert the archives.

  $ rpm -ql cvs | grep pvcs
  /usr/share/cvs/contrib/pvcs2rcs

Re: My deployment platform is a shell script

#125

Post by kragen »

chasil wrote: post #124
kragen wrote: post #116

Earlier quoted context omitted.

that version control system provided a number of facilities cvs didn't (locking, and also a certain degree of integration with our build system permitting the various developers to only recompile the part of the system they were working on, which was important because recompiling the whole thing usually took me about a week, once a month), but it had never actually occurred to me that turning an rcs repository into a…

PVCS did not use the RCS format, but the RPM distribution included a perl script to convert the archives. $ rpm -ql cvs | grep pvcs /usr/share/cvs/contrib/pvcs2rcs

ooh. that would have been very useful two jobs later when i got stuck with pvcs

Re: My deployment platform is a shell script

#126

Post by elevation »

OddMerlin wrote: post #34

Why not use Ansible for something like this? Don’t get me wrong, I love bash scripts like any other old hat, but Ansible scratches this exact itch. You’ve got playbooks that can execute shell, provide logging, better management, history of execution, fleet management, and it’s light weight. And there’s a robust community of shared modules, etc.

Bash is better than ansible for configuring the core infrastructure underneath ansible.

In a devops workflow you "treat servers like cattle instead of pets" but your org still needs a few pets. Some host you control must either host DNS or manage your DNS provider's API key. Same for CA, IdP, git, backup and monitoring services, and the ansible machine itself. You'll have to manually configure these things before your "cattle" tools can run.

Once you're up and running, it's possible to make ansible manage it own dependencies, but this introduces circular dependencies complicates bootstrapping (consider a disaster recovery situation) and amplifies both the impact of faults and the difficulty of troubleshooting them. Do you want to be debugging python dependencies in the middle of the night so you can finally get ansible to execute the couple bash commands that will bring your ACME CA back up? I'd rather run bash directly.

At a small scale with a stable set of requirements, your core infrastructure is better served by a good operations manual and a simple deployment toolset with minimal dependencies. Plain bash fits the bill!

Re: My deployment platform is a shell script

#127

Post by bravetraveler »

cess11 wrote: post #117
bravetraveler wrote: post #91

Earlier quoted context omitted.

Let's take a step back from names for a second - I feel we're getting a little distracted by that. My gripe isn't with the tech, or even the solution. It's perfectly fine. I know I'm being overly critical, but I think they opened themselves to some judgement by making a post! I would do something very similar. Sure, the tool wouldn't have the exact same name, but the mechanism would be ~the same~ very similar. If the…

Pretty sure bash and Perl and C (and make and autotools and probably something I'm forgetting) are the beaten path in this context, i.e. software building and deployment on Unix. Ansible has surprised me way more often than bash has. The latter is upfront about being weird and a little bit insane, Ansible tries to have a better image. In my experience that is true also for Puppet and some other similar tools, which I…

Bash/Perl/C being precursors don't disqualify Python/Ansible, though. Otherwise my retort would be this: assembly.

While I empathize, your experience isn't [widely] representative. A reproducible Python environment is just as attainable as any other - anyone struggling with it has accepted it, in my opinion.

Most of my peers/community uses the distribution packages. They don't even need to care about venv/pip at all.

Re: My deployment platform is a shell script

#128

Post by ffsm8 »

from-nibly wrote: post #57
mattbuilds wrote: post #54

Earlier quoted context omitted.

It’s funny because in my many years of development I don’t think I’ve ever encounter a “mess of shell scripts” that was difficult to maintain. They were clear, did their job and if they needed to be replaced it was usually simple and straightforward. Can’t say the same for whenever the new abstraction of the day comes along. In my experience what the OP is saying is exactly my experience. The abstractions get picked…

Hello. I have found the mess of shell scripts. Please don't do this. I was able to deal with the weird skaffold mess by getting rid of it, and replacing it with argocd. I was able to get rid of jenkins by migrating to github actions. I have yet to replace the magic servers with magic bash scripts. They take just enough effort that i can't spend the time. Use a tool i can google. If your bash script is really this str…

Wasn't there a rule about that?

Something like "in software development the only solution that sticks is the bad one, because the good ones will keep getting replaced until it's so bad, nobody can replace it anymore"

Re: My deployment platform is a shell script

#129

Post by dang »

cqqxo4zV46cp wrote: post #41
anonzzzies wrote: post #27

I use similar things for bigger (multi-server) deploys too. It's light and it just works and works for decades without changes/updates. People say it's brittle; I have a proof of n>0 that this is not the case compared to many other solutions, this post making that point too. Sh/bash/perl(8) have been around forever, they don't break after update etc. I sadly don't recommend it for my day job, simply because of liabil…

[flagged]

You've been breaking the site guidelines frequently lately by posting much too aggressively. We have to ban accounts that won't stop doing this, so if you'd please stop doing this, that would be good.

HN is for thoughtful, respectful, curious conversation—not attacking other users or elbowing.

https://news.ycombinator.com/newsguidelines.html

Re: My deployment platform is a shell script

#130

Post by prmoustache »

bravetraveler wrote: post #96
prmoustache wrote: post #48

Earlier quoted context omitted.

I think even ansible is overkill for such a simple thing. Ansible use case works better when you need to do stuff on multiple hosts. For years I've started using and abandoned ansible and puppet recipes for setting up my own computers and everytime the conclusion was that I would spend more time installing git, ansible and puppet in the first place and debugging my recipes than using them. Now all my setup lives in s…

Ansible is great even for simple single-host 'shell scripts'. Lean into the module ecosystem. Want to ensure a config file is a certain way? Jinja/template it, or use lineinfile instead of echo/shell redirects. That's a lot of mumbo-jumbo. The point is, there's a lot of stuff scripts want to do. Ansible provides these as modules. Using the modules spares you from writing code to do something in a robust/repeatable wa…

> Using the modules spares you from writing code to do something in a robust/repeatable way.

That is a huge lie: Declarative code is still code. Using modules is similar to reusing functions. The things is, while reusability and declarative code is nice when you want to deploy and manage multiple machines and have an automated network install that bootstrap your automated configuration tool. It is worth the effort because there are many machines but all that automation need to be tested/fixed on a regular basis (distro releases, etc). If you are reinstalling your machine from an usb pendrive, or image once every so many full moons, you need first to bootstrap ansible and the playbook. How do you that in an idempotent manner? The time you have done that you are probably already ready.

The only thing I need on my dev machines is :

- my software configs: comes with a git repo of my dotfiles.

- a dev directory: mkdir is idempotent, it will not destroy dir if it exists so no need for a declarative language

- some packages: a single package install is needed. While using a configuration tool allows you to declare package name depending on distro version and regardless of package tool, usually someone who is managing 2-3 machines stick to one OS so in my case a single `dnf install -y ` is enough.

- a few tools I curl from github or other places. I have one bash function for each of them to get latest release (a one liner) and one to compare installed version with latest release and install if needed. Ansible doesn't do a better job at it. I checked ansible-galaxy for some of the tools I download, for some no module exists, for those that have modules, they are just made of ... shell scripts called by an ansible task that is larger than my own script. See example[1]

- a few desktop files, they come with my dotfiles git repo, no need to "declare them"

- a handful of stuff that comes with an install shell script (the infamous curl | bash). Ansible don't help me much or I'd have to rewrite the install script as an ansible playbook and maintain it myself forever.

No handler necessary, none of this requires a reboot or a service restart.

Also Ansible is probably the worst example as it is an half assed declarative language that doesn't even encourage idempotence. Basically it is made by and for old unix guys who want to continue writing sequencial scripts the old and crappy way while pretending they do things in a modern way. And it is the reason it has become so popular against cfengine, puppet, chef and salt. I don't see the point of using ansible if it is to have the same low standard of quality as plain old scripts.

My experience working with teams using ansible has only reinforced my view that this language is for people who like to do things the dirtiest way anyway but wrapping it in a declarative language so they can put devops engineer in their resume.

[1] https://github.com/andrewrothstein/ansible-eksctl/tree/main

Post reply on HN