Honestly part of the reason why I left my last job was because I had to extensively work with GitHub workflows and actions. Debugging was absolutely hell, especially with long running tasks (~1hr) that would fail with next to no debug or traceability. I offered many times to overhaul the system and make it easier to maintain, but we had no time budget. I now work for 30% less but am substantially happier in life without github. It sounds crazy, but I’m not exaggerating.
I hate GitHub Actions with passion
261–270 of 347 posts
Re: I hate GitHub Actions with passion
#262So many engineers could put the hours spent debugging GH actions to use developing expertise to run their own CI. But people either don’t believe they can, can’t convince decision makers to let them try, or just want to fix their own problem and move on. I was convinced GH actions was best practice and it was normal to waste hours on try-and-pray build debugging, until one day GH actions went down and I ran deploys f…
> So many engineers could put the hours spent debugging GH actions to use developing expertise to run their own CI. If I run my own CI, then the compliance team has to get involved to run various endpoint security and update management tools on whatever system I'm running the CI on.
Then the costs of staying with GH actions need to be made known so they can be balanced against the cost of doing things differently. Of course there’s a cost involved in just getting those numbers too.
It’s all trade offs.
Re: I hate GitHub Actions with passion
#263Earlier quoted context omitted.
I think in this case they hate the fact that they cannot easily SSH into the failing VM and debug from there. Like "I have to edit my workflow, push it, wait for it to run and fail, and repeat".
One can get the ssh access with self-hosted runners but it is problematic because uncovering secrets becomes trivial.
Uncovering secrets is usually trivial. `printenv` in a build script does that pretty reliably.
Re: I hate GitHub Actions with passion
#264Its not Github Actions' fault but the horrors people create in it, all under the pretense that automation is simply about wrapping a GitHub Action around something. Learn to create a script in Python or similar and put all logic there so you can execute it locally and can port it to the next CI system when a new CTO arrives.
> Learn to create a script in Python or similar and put all logic there so you can execute it locally and can port it to the next CI system when a new CTO arrives.
That's a perfectly good several weeks of time un-estimateable busywork I could be getting paid for!
Re: I hate GitHub Actions with passion
#265I am surprised that these sort of declarative specs are so popular in certain domains. Essentially you always seem to be putting settings into the ether and hoping they interact with each other in the way you expect.
I prefer an api with documented contracts between its abstractions
Re: I hate GitHub Actions with passion
#266Earlier quoted context omitted.
Usually, the people writing the Makefile are the same that could also be writing this stuff out in a YAML (lol) file as the CI instructions, often located in the same repository anyway. The irony in that is striking. And then we have people who can change environment variables for the CI workflows. Usually, also developers, often the same people that can commit changes to the Makefile. I don't think it changes much,…
You're correct. It's absolutely security theater. Either you trust the repository contents or you don't. There's no, none, zilch trust improvement arising from the outer orchestration being done in a YAML file checked into the repo and executed by CI instead of a Makefile also executed by CI. What's the threat model Wilder is using exactly? Look, I'm ordinarily all for nuance and saying reasonable people can disagree…
That's a great point. If we keep following the requirement for attestation to its logical conclusion we would end up replicating the entire server running the repository at the source, then the cycle repeats
Re: I hate GitHub Actions with passion
#267Earlier quoted context omitted.
I've standardized on getting github actions to create/pull a docker image and run build/test inside that. So if something goes wrong I have a decent live debug environment that's very similar to what github actions is running. For what it's worth.
I do the same with Nix as it works for macOS builds as well It has the massive benefit of solving the lock-in problem. Your workflow is generally very short so it is easy to move to an alternative CI if (for example) Github were to jack up their prices for self hosted runners... That said, when using it in this way I personally love Github actions
same here. though, i think bazel is better for DAGs. i wish i could use it for my personal project (in conjunction with, and bootstrapped with nix), but that's a pretty serious tooling investment that I just feel is just going to be a rabbit hole.
Re: I hate GitHub Actions with passion
#268Earlier quoted context omitted.
It's insane to me that being able to run CI steps locally is not the first priority of every CI system. It ought to be a basic requirement.
This goes against every incentive for the CI service provider
Not necessarily. For example, Buildkite lets you host your own runners.
Re: I hate GitHub Actions with passion
#269I like Github Actions and it is better than what I used before (Travis) and I think it solves an important problem. For OSS projects it's a super valuable free resource. For me what worked wonders was adopting Nix. Make sure you have a reproducible dev environment and wrap your commands in `nix-shell --run`, or even better `nix develop --command`, or even better your most of your CI tasks derivations that run with `n…
Second the Nix approach. One can even build a github actions-compatible container out of a flake and have actions run in it. I have done so for my personal projects https://github.com/anttiharju/compare-changes
isn't this better served with something like Bazel (behind Nix of course?)
Re: I hate GitHub Actions with passion
#270I recently, for Rust targets reasons, have decided to punt GitHub Actions AND GitHub. Opted for Radicle. Had to figure out my own CI.
Doc'd it here: https://revolveteam.com/blog/goa-radicle-ci/