Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'll think twice before using GitHub Actions again

ninkovic.dev

Post reply on HN
Print view View original

151–160 of 284 posts

Re: I'll think twice before using GitHub Actions again

#151

Post by ripped_britches »

angoragoats wrote: post #54

Why is this team sticking multiple directories that are “independent of each other” into a single repository? This sounds like a clear case of doing version control wrong. Monorepos come with their own set of challenges, and I don’t think there are many situations where they’re actually warranted. They certainly don’t help for completely independent projects.

No project within a single organization is completely independent. In general they all serve to meet a unified business objective and developers often need the global context on occasion.

I used to be a multirepo proponent, but have fallen in love with Bazel and “everything bagel” repo.

Re: I'll think twice before using GitHub Actions again

#152

Post by 0xbadcafebee »

hardwaresofton wrote: post #136
0xbadcafebee wrote: post #96

Earlier quoted context omitted.

100%. The ci/cd job should be nothing more than a wrapper around the actual logic which is code in your repo. I write a script called `deploy.sh` which is my wrapper for my ci/cd jobs. It takes options and uses those options to find the piece of code to run. The ci/cd job can be parameterized or matrixed. The eventually-run individual jobs have arguments, and those are passed to deploy.sh. Secrets/environment variabl…

Hey if you’ve never heard of it consider using just[0], it’s a better makefile and supports shell scripting explicitly (so at least equivalent in power, though so is Make) [0]: https://github.com/casey/just

Thank you, I have seen it, but I prefer Make.

Re: I'll think twice before using GitHub Actions again

#153

Post by datavirtue »

mongol wrote: post #113
datavirtue wrote: post #100

Earlier quoted context omitted.

I just joined as the enterprise architect for company that has never had one. There is an existing devops team that is making everyone pull their hair out and I haven't had a single spare minute to dig in on their mess but this sounds early familiar.

Is this really the job of an enterprise architect? To dig into the details of a devops team's mess?

It's more about giving the team and the overall strategy a thumbs up or down, so yes.

Re: I'll think twice before using GitHub Actions again

#154

Post by mhh__ »

arghwhat wrote: post #5

> no way of running actions locally My policy is to never let pipeline DSLs contain any actual logic outside orchestration for the task, relying solely on one-liner build or test commands. If the task is more complicated than a one-liner, make a script for it in the repo to make it a one-liner. Doesn't matter if it's GitHub Actions, Jenkins, Azure DevOps (which has super cursed yaml), etc. This in turn means that you…

Indeed. Anything else is just asking for trouble.

CI must only be an application of tools available to developers locally.

This, other than for practical reasons, is a tax on complexity.

Re: I'll think twice before using GitHub Actions again

#155

Post by evilduck »

bluGill wrote: post #132
Tainnor wrote: post #9

Earlier quoted context omitted.

The reason why many CI configs devolve into such a mess isn't typically that they don't extract complicated logic into scripts, it's about all the interactions with the CI system itself. This includes caching, sharing of artifacts, generating reports, configuring permissions, ordering of jobs, deciding when which jobs will run, deciding what to do when jobs fail, etc. All of this can get quite messy in a large enough…

You should generate your report with regular scripts. You need ci config to deploy them but that is the only part that should be different.

This doesn’t really work when you start sharding tests.

Re: I'll think twice before using GitHub Actions again

#156

Post by no_wizard »

ledauphin wrote: post #95
no_wizard wrote: post #68

Earlier quoted context omitted.

Folks pick the wrong tool for the job at hand. I suspect the author of the article could greatly simplify matters if they used a task running tool to orchestrate running tasks, for example. Pick whatever manner of decoupling you want really, most of the time this is the path to simplified CI actions. CI is best when its thought of as a way to stand up fresh copies of an environment to run things inside of. I have nev…

I generally agree with you, but I'd be interested to hear your take on what the purpose of CI _actually is_. It seems to me that a big part of the problem here (which I have also seen/experienced) is that there's no one specific thing that something like GitHub Actions is uniquely suited for. Instead, people want "a bunch of stuff to happen" when somebody pushes a commit, and they imagine that the best way to trigger…

Standing up fresh images for validation and post validation tasks.

CI shines for running tests against a clean environment for example.

Really any task that benefits from a clean image being stood before running a task.

The key though is to decouple the tasks from the CI. Complexity like pre-packaging artifacts is not a great fit for CI configuration, that is best pushed to a tool that doesn’t require waterfall logic to make it work.

There is a reason task runners are very popular still

Re: I'll think twice before using GitHub Actions again

#157

Post by 0xbadcafebee »

jicea wrote: post #27
arghwhat wrote: post #5

> no way of running actions locally My policy is to never let pipeline DSLs contain any actual logic outside orchestration for the task, relying solely on one-liner build or test commands. If the task is more complicated than a one-liner, make a script for it in the repo to make it a one-liner. Doesn't matter if it's GitHub Actions, Jenkins, Azure DevOps (which has super cursed yaml), etc. This in turn means that you…

I don't understand why this is not the evident approach for everyone writing GitHub Actions/GitLab CI/CD yaml etc.... I've struggled in some teams to explained why it's better to extract your command in scripts (ShellCheck on it, scripts are simple to run locally etc...) instead of writing a Frankenstein of YAML and shell commands. I hope someday to find an authoritative guidelines on writing pipeline that promote th…

I think I can summarize it in a rough, general way.

  CI/CD is a method to automate tasks in the background that you would otherwise run on your laptop. The output
  of the tasks are used as quality gates for merging commits, and for deployments.
  
  - Step 1. Your "laptop in the cloud" requires some configuration (credentials, installed software, cached artifacts)
    before a job can be run.
    - Requires logic specific to the CI/CD system
  
  - Step 2. Running many jobs in parallel, passing data from step to step, etc requires some instructions.
    - Requires logic specific to the CI/CD system
  
  - Step 3. The job itself is the execution of a program (or programs), with some inputs and outputs.
    - Works the same on any computer (assuming the same software, environment, inputs, etc)
    - Using a container in Step 1. makes this practical and easy
  
  - Step 4. After the job finishes, artifacts need to be saved, results collected, and notifications sent.
    - Some steps are specific to the CI/CD system, others can be a reusable job
  
  Step 3 does not require being hard-coded into the config format of the CI/CD system. If it is instead
  just executable code in the repo, it allows developers to use (and work on) the code locally without
  the CI/CD system being involved. It also allows moving to a different CI/CD system without ever rewriting
  all the jobs; the only thing that needs to be rewritten are the CI/CD-specific parts, which should be
  generic and apply to all jobs pretty much the same.
  
  Moving the CI/CD-specific parts to a central library of configuration allows you to write some code
  once and reuse it many times (making it DRY). CircleCI Orbs, GitHub Actions, Jenkins Shared Libraries/
  Groovy Libraries, etc are examples of these. Write your code once, fix a bug once, reuse it everywhere.

Re: I'll think twice before using GitHub Actions again

#158

Post by baby_souffle »

arghwhat wrote: post #33
jicea wrote: post #27

Earlier quoted context omitted.

I don't understand why this is not the evident approach for everyone writing GitHub Actions/GitLab CI/CD yaml etc.... I've struggled in some teams to explained why it's better to extract your command in scripts (ShellCheck on it, scripts are simple to run locally etc...) instead of writing a Frankenstein of YAML and shell commands. I hope someday to find an authoritative guidelines on writing pipeline that promote th…

My "favorite" is when I see people go all in, writing thousands of lines of Jenkins-flavor Groovy that parses JSON build specifications of arbitrary complexity to sort out how to build that particular project. "But then we can reuse the same pipeline for all our projects!"

> "But then we can reuse the same pipeline for all our projects!"

oh god just reading that gave me PTSD flash backs.

At $priorGig there was the "omni-chart". It was a helm chart that was so complex it needed to be wrapped in terraform and used composable terraform modules w/ user var overrides as needed.

Debugging anything about it meant clearing your calendar for the day and probably the following day, too.

Re: I'll think twice before using GitHub Actions again

#159

Post by akoboldfrying »

carlmr wrote: post #90
akdev1l wrote: post #86

Earlier quoted context omitted.

Containerize the build environment so everything is captured (dependencies, build tools, etc)

If you're not containerizing your CI/CD, you're really lost.

That might be the case if Docker did in fact guarantee (or at least make it easy to guarantee) deterministic builds -- but it doesn't really even try:

1. Image tags ("latest", etc.) can change over time. Does any layer in your Dockerfile -- including inside transitive deps -- build on an existing layer identified by tag? If so, you never had reproducibility.

2. Plenty of Dockerfiles include things like "apt-get some-tool" or its moral equivalent, which will pull down whatever is the latest version of that tool.

It's currently common and considered normal to use these "features". Until that changes, Docker mostly adds only the impression of reproducibility, but genuine weight and pain.

Re: I'll think twice before using GitHub Actions again

#160

Post by mplanchard »

arghwhat wrote: post #46
alkonaut wrote: post #37

Earlier quoted context omitted.

That’s usually very hard or impossible for many things. The AzDo yaml consists of a lot of steps that are specific to the CI environment (fetching secrets, running tests on multiple nodes, storing artifacts of various kinds). Even if the ”meat” of the script is a single build.ps oneliner, I quickly end up with 200 line yaml scripts which have no chance of working locally.

Azure DevOps specifically has a very broken approach to YAML pipelines, because they effectively took their old graphical pipeline builder and just made a YAML representation of it. The trick to working with this is that you don't need any of their custom Azure DevOps task types, and can use the shell type (which has a convenient shorthand) just as well as in any other CI environment. Even the installer tasks are red…

This doesn’t seem to address the parent comment’s point at all, which was about required non-shell configuration such as for secrets, build parallelism, etc.
Post reply on HN