Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'll think twice before using GitHub Actions again

ninkovic.dev

Post reply on HN
Print view View original

181–190 of 284 posts

Re: I'll think twice before using GitHub Actions again

#181

Post by eru »

arccy wrote: post #70
jicea wrote: post #27

Earlier quoted context omitted.

I don't understand why this is not the evident approach for everyone writing GitHub Actions/GitLab CI/CD yaml etc.... I've struggled in some teams to explained why it's better to extract your command in scripts (ShellCheck on it, scripts are simple to run locally etc...) instead of writing a Frankenstein of YAML and shell commands. I hope someday to find an authoritative guidelines on writing pipeline that promote th…

it can be quite hard to write proper scripts that work consistently... different shells have different behaviours, availability of local tools, paths, etc and it feels like fighting against the flow when you're trying to make it reusable across many repos

Pick a single shell and treat it like a programming language.

Or write your stuff in eg Python in the first place. GitHub's CI yaml supports scripts in arbitrary languages, not just shell.

Re: I'll think twice before using GitHub Actions again

#182

Post by eru »

maccard wrote: post #115
carlmr wrote: post #90

Earlier quoted context omitted.

If you're not containerizing your CI/CD, you're really lost.

How do I containerize building desktop apps for windows with MSVC?

Wine?

Less snarky: https://learn.microsoft.com/en-us/virtualization/windowscont... seems to be a thing?

And in any case, you can use VMs instead of containers.

Re: I'll think twice before using GitHub Actions again

#183

Post by harrall »

eru wrote: post #180
harrall wrote: post #173

Earlier quoted context omitted.

A shell script has many extremely sharp edges like dealing with stdin, stderr, stdout, subprocesses, exit codes, environmental variables, etc. Most programmers have never written a shell script and writing CI files is already frustrating because sometimes you have to deploy, run, fix, deploy, run, fix, which means nobody is going to stop in the middle of that and try to learn shell scripting. Instead, they copy comma…

GitHub's CI yaml also accepts eg Python. (Or anything else, actually.) That's generally a bit less convenient, ie it takes a few more lines, but it has significantly fewer sharp edges than your typical shell script. And more people have written Python scripts, I guess?

I find that Python scripts that deal with calling other programs have even more sharp edges because now you have to deal with stdin, stderr and stdout much more explicitly. Now you need to both know shell scripting AND Python.

Python’s subprocess has communicate(), check_output() and other helpers which takes care of a lot but (a) you need to know what method you should actually call and (b) if you need to do something outside of that, you have to use Popen directly and it’s much more work than just writing a shell script. All doable if you understand pipes and all that but if you don’t, you’ll be just throwing stuff at the wall until something sticks.

Re: I'll think twice before using GitHub Actions again

#185

Post by dpkirchner »

jicea wrote: post #27
arghwhat wrote: post #5

> no way of running actions locally My policy is to never let pipeline DSLs contain any actual logic outside orchestration for the task, relying solely on one-liner build or test commands. If the task is more complicated than a one-liner, make a script for it in the repo to make it a one-liner. Doesn't matter if it's GitHub Actions, Jenkins, Azure DevOps (which has super cursed yaml), etc. This in turn means that you…

I don't understand why this is not the evident approach for everyone writing GitHub Actions/GitLab CI/CD yaml etc.... I've struggled in some teams to explained why it's better to extract your command in scripts (ShellCheck on it, scripts are simple to run locally etc...) instead of writing a Frankenstein of YAML and shell commands. I hope someday to find an authoritative guidelines on writing pipeline that promote th…

This all reminds me of the systemd ini-like syntax vs shell scripts debate. Shell scripts are superior, of course, but they do require deeper knowledge of unix-like systems.

Re: I'll think twice before using GitHub Actions again

#186

Post by atkbrah »

thatxliner wrote: post #141
arghwhat wrote: post #5

> no way of running actions locally My policy is to never let pipeline DSLs contain any actual logic outside orchestration for the task, relying solely on one-liner build or test commands. If the task is more complicated than a one-liner, make a script for it in the repo to make it a one-liner. Doesn't matter if it's GitHub Actions, Jenkins, Azure DevOps (which has super cursed yaml), etc. This in turn means that you…

How would you set up tool installations? Inside the CI or inside the script?

In our environment (our product is a windows desktop application) we use packer to build a custom windows server 2022 image with all the required tools installed. Build agents run on a azure vm scale set that uses the said image for the instance os.

Re: I'll think twice before using GitHub Actions again

#187

Post by slt2021 »

dpkirchner wrote: post #185
jicea wrote: post #27

Earlier quoted context omitted.

I don't understand why this is not the evident approach for everyone writing GitHub Actions/GitLab CI/CD yaml etc.... I've struggled in some teams to explained why it's better to extract your command in scripts (ShellCheck on it, scripts are simple to run locally etc...) instead of writing a Frankenstein of YAML and shell commands. I hope someday to find an authoritative guidelines on writing pipeline that promote th…

This all reminds me of the systemd ini-like syntax vs shell scripts debate. Shell scripts are superior, of course, but they do require deeper knowledge of unix-like systems.

yeah if you author CI jobs, you should know linux, otherwise a person should not even touch the CI system with 10ft pole

Re: I'll think twice before using GitHub Actions again

#188

Post by eru »

harrall wrote: post #183
eru wrote: post #180

Earlier quoted context omitted.

GitHub's CI yaml also accepts eg Python. (Or anything else, actually.) That's generally a bit less convenient, ie it takes a few more lines, but it has significantly fewer sharp edges than your typical shell script. And more people have written Python scripts, I guess?

I find that Python scripts that deal with calling other programs have even more sharp edges because now you have to deal with stdin, stderr and stdout much more explicitly. Now you need to both know shell scripting AND Python. Python’s subprocess has communicate(), check_output() and other helpers which takes care of a lot but (a) you need to know what method you should actually call and (b) if you need to do somethi…

Yes, but at least that's all fairly obvious---you might not know how to solve the problem, but at least you know you have a problem that needs solving. Compare that to the hidden pitfalls of eg dealing with whitespace in filenames in shell scripts. Or misspelled variable names that accidentally refer to non-existent variables but get treated as if they are set to be "".

Re: I'll think twice before using GitHub Actions again

#189

Post by chubot »

hardwaresofton wrote: post #136
0xbadcafebee wrote: post #96

Earlier quoted context omitted.

100%. The ci/cd job should be nothing more than a wrapper around the actual logic which is code in your repo. I write a script called `deploy.sh` which is my wrapper for my ci/cd jobs. It takes options and uses those options to find the piece of code to run. The ci/cd job can be parameterized or matrixed. The eventually-run individual jobs have arguments, and those are passed to deploy.sh. Secrets/environment variabl…

Hey if you’ve never heard of it consider using just[0], it’s a better makefile and supports shell scripting explicitly (so at least equivalent in power, though so is Make) [0]: https://github.com/casey/just

The shell also supports shell scripting! You don't need Just or Make

Especially for Github Actions, which is stateless. If you want to reuse computation within their VMs (i.e. not do a fresh build / test / whatever), you can't rely on Just or Make

A problem with Make is that it literally shells out, and the syntax collides. For example, the PID in Make is $$$$, because it's $$ in shell, and then you have to escape $ as $$ with Make.

I believe Just has similar syntax collisions. It's fine for simple things, but when it gets complex, now you have {{ just vars }} as well as $shell_vars.

It's simpler to "just" use shell vars, and to "just" use shell.

Shell already has a lot of footguns, and both Just and Make only add to that, because they add their own syntax on top, while also depending on shell.

Re: I'll think twice before using GitHub Actions again

#190

Post by smaudet »

harrall wrote: post #183
eru wrote: post #180

Earlier quoted context omitted.

GitHub's CI yaml also accepts eg Python. (Or anything else, actually.) That's generally a bit less convenient, ie it takes a few more lines, but it has significantly fewer sharp edges than your typical shell script. And more people have written Python scripts, I guess?

I find that Python scripts that deal with calling other programs have even more sharp edges because now you have to deal with stdin, stderr and stdout much more explicitly. Now you need to both know shell scripting AND Python. Python’s subprocess has communicate(), check_output() and other helpers which takes care of a lot but (a) you need to know what method you should actually call and (b) if you need to do somethi…

1) If possible, don't run shell scripts with Python. Evaluate why you are trying to do that and don't. 2) Python has a bunch of infrastructure compared to shell, you can use it. Shell scripts don't. 3) Apply the same you used for the script to what it calls. CI calls control script for job, script calls tools/libraries for heavy lifting.

Often the shell script just calls a python/Ruby/rust exec anyways...

Shell scripts are for scripting...the shell. Nothing else.

Post reply on HN