Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Pain That Is GitHub Actions

feldera.com

Post reply on HN
Print view View original

251–260 of 584 posts

Re: The Pain That Is GitHub Actions

#251

Post by jimmcslim »

oulipo wrote: post #143
JanMa wrote: post #112

Earlier quoted context omitted.

Whenever possible I now just use GitHub actions as a thin wrapper around a Makefile and this has improved my experience with it a lot. The Makefile takes care of installing all necessary dependencies and runs the relevant build/Test commands. This also enables me to test that stuff locally again without the long feedback loop mentioned in other comments in this thread.

mise ( https://mise.jdx.dev/ ) and dagger ( https://github.com/dagger/dagger ) seem like nice candidates too! Mise can install all your deps, and run tasks

In addition to the other comments suggesting dagger is not the saviour due to being VC-funded, it seems like they have decided there's no money in CI, but AI... yes there's money there! And "something something agents".

From dagger.io...

"The open platform for agentic software.

Build powerful, controllable agents on an open ecosystem. Deploy agentic applications with complete visibility and cross-language capabilities in a modular, extensible platform.

Use Dagger to modernize your CI, customize AI workflows, build MCP servers, or create incredible agents."

Re: The Pain That Is GitHub Actions

#252

Post by joshstrange »

mook wrote: post #110
silverwind wrote: post #31

GHA is full of such obure behaviours. One I recently discovered is that one action can not trigger another: If one action pushes a tag to the repo, `on:tag` does not trigger. The workaround apparently is to make the first action push the tag using a custom SSH key, which magically has the ability to trigger `on:tag`.

That actually seemed reasonable when I hit it, because you can easily accidentally have an action triggered on commit that makes a new commit, ending up in an infinite loop. The workaround is to use a token tied to you instead of GitHub Actions, so you get charged (or run out of quota).

> The workaround is to use a token tied to you instead of GitHub Actions, so you get charged (or run out of quota).

You get charged no matter what, a personal access token doesn’t change anything.

If they are concerned about infinite loops then put a limit on how many workflows can be triggered but another workflow. Each time a workflow chains off another pass along some meta data of “runsDeep” and stop when that hits X, which can be configured.

No, requiring a PAT to kick off a workflow from a workflow is gross and makes zero sense. I don’t want every tag associated with my user, I want it to be generic, the repo itself should be attributed. The only way to solve this is to create (and pay for) another GH user that you create PAT tokens under. A bunch of overhead, cost, and complexity for no good reason.

Re: The Pain That Is GitHub Actions

#253

Post by deng »

motorest wrote: post #226
deng wrote: post #153

Earlier quoted context omitted.

YAML is fine for what it is: a markup language. I have no problem with it being used in simple configuration files, for instance. However, CI is not "configured", it is coded. It is simply the wrong tool. YAML was continuously extended to deal with that, so it developed into much more than just "markup", but it grew into this terrible chimera. Once you start using advanced features in GitLab's YAML like anchors and r…

> However, CI is not "configured", it is coded. No, it really isn't. I'll clarify why. Pretty much all pipeline services share the same architecture pattern: * A pipeline run is comprised of one or more build jobs, * Pipeline runs are triggered by external events * Build jobs have contexts and can output artifacts, * Build jobs are grouped into stages, * Stages are organized as a directed graph, * Transitions between…

We don't disagree here. There are tools which support you in doing this, and I mentioned a few of them in my post (Make, Just, doit, mage). There are many more. I also think that re-inventing these tools is a waste of time, but it is still better than shoehorning this into YAML. You seem to think YAML is some kind of DSL for pipelines. It really is not.

Re: The Pain That Is GitHub Actions

#254

Post by joshstrange »

demaga wrote: post #162

We recently discover that if the last person to change cron __schedule__ of the workflow is removed from the organization, workflow fails with cryptic errors. It turns out, the last person to change cron __schedule__ (not the workflow file in general) is an 'actor' associated with this workflow. Very, very confusing implementation. Error messages are even more confusing - workflow runs are renamed as "{Unknown event}…

Things like this are why I hate having to use PATs in workflows. What if I leave the company? I’ll leave a wake of broken actions in my wake. I do not like that at all, a huge point of CI/CD is automation, reproducibility, and NOT being dependent on specific developers/machines.

Re: The Pain That Is GitHub Actions

#255

Post by z3t4 »

ManBeardPc wrote: post #177

CI environments like Gitlab or Github are my nemesis. Another technology that everyone swears is absolute necessary but somehow makes everything more complicated. The provided environments in companies so far are hell 100% the time and managed by inexperienced personnel with zero or little programming experience. * Barely reproducible because things like the settings of the server (environment variables are just one…

It's such a waste of resources to rebuild an operating system every time you want to run some tests, and these CI machines are much less powerful then personal computers so it takes much longer in the cloud too. If you have your CI in your own scripts it will be easy to migrate between CI environments too. build: ./buuild.sh test: ./test.sh deploy: ./deploy.sh maybe pass some env variables to the scripts. There are some advantages to CI platforms, like nightly build/test and automatic security scan on already deployed software, so that you will be notified when something suddenly stops working or a vulnerability is discovered.

Re: The Pain That Is GitHub Actions

#256

Post by chuckadams »

normie3000 wrote: post #41
yeswecatan wrote: post #32

Earlier quoted context omitted.

Unfortunately people will use --no-verify to bypass hooks.

I don't understand commit hooks - they're like binding a macro to the MS Word save button to make it conditional.

> like binding a macro to the MS Word save button to make it conditional

You have no idea how much I'd love that feature. Inasmuch as "save" is still a thing anyway. I don't miss explicit saves in IDEA, I see commit as the "real" save operation now, and I don't mind being able to hook that in an IDE-independent way.

I think the UX of git hooks has been sub-par for sure, but tools like the confusingly named pre-commit are helping there.

Re: The Pain That Is GitHub Actions

#257

Post by magicalhippo »

kylegalbraith wrote: post #55

This was an interesting read and highlighted some of the author's top-of-mind pain points and rough edges. However, in my experience, this is definitely not an exhaustive list, and there are actually many, many, many more. Things like 10 GB cache limits in GitHub, concurrency limits based on runner type, the expensive price tag for larger GitHub runners, and that's before you even get to the security ones. Having bee…

Depot looks nice, but also looks fairly expensive to me. We're a small B2B company, just 10 devs, but we'd be looking at 200+500 = $700/mo just for building and CI.

I guess that would be reasonable if we really needed the speedup, but if you're also offering a better QoL GHA experience then perhaps another tier for people like us who don't necessarily need the blazing speed?

Re: The Pain That Is GitHub Actions

#258

Post by jFriedensreich »

I am completely switching my mental model of what a ci/cd system should be at the moment: i use docker compose for absolutely everything possible. unit tests? runs as part of the container build. linear build dependent steps? multi stage docker biuld. DAG of build steps? dependencies in docker compose. This way every developer has the same system that ci/cd uses locally. debugging the dev setup is the same as debugging the ci/cd. The purpose of the actual ci/cd is reduced to handling/configuring triggers, handling env vars/secrets and triggering the docker compose command with the proper selected docker context

This also reduces the lock in by orders of magnitude.

Re: The Pain That Is GitHub Actions

#259

Post by never_inline »

forrestthewoods wrote: post #119
deng wrote: post #86

Already see people saying GitLab is better: yes it is, but it also sucks in different ways. After years of dealing with this (first Jenkins, then GitLab, then GitHub), my takeaway is: * Write as much CI logic as possible in your own code. Does not really matter what you use (shell scripts, make, just, doit, mage, whatever) as long as it is proper, maintainable code. * Invest time that your pipelines can run locally o…

> pipelines can run locally on a developer machine as well (as much as possible at least) Facts. However I’ll go a step further and say “only implement your logic in a tool that has a debugger”. YAML is the worse. But shell scripts are second worst. Use a real language.

Python with click and PyYAML can go a long way - then you can build it as a CLI application and use the same from CI. In a java shop, picocli + graalvm probably. I wouldn't like Go for this purpose (against the conventional wisdom - because boilerplate and pretty bad debugging capabilities).

That said, if you absolutely need to use shell script for reasons, keep it all in single script, define logging functions including debug logs, rigorously check every constraint and variable, use shellcheck, factor the code well into functions - I should sometimes write a blog post about it.

Re: The Pain That Is GitHub Actions

#260

Post by ManBeardPc »

riperoni wrote: post #236
ManBeardPc wrote: post #177

CI environments like Gitlab or Github are my nemesis. Another technology that everyone swears is absolute necessary but somehow makes everything more complicated. The provided environments in companies so far are hell 100% the time and managed by inexperienced personnel with zero or little programming experience. * Barely reproducible because things like the settings of the server (environment variables are just one…

To be honest, some of your points can be a hindrance, but as a GitLab user, others are solveable without massive efforts. - env vars can be scripted, either in YAML or through dotenv files. Dotenv files would also be portable to dev machines - how is security a joke? Do you mean secrets management? Otherwise, i don't see a big issue when using private runners with containers - jobs can pass artifacts to each other. W…

TLDR: True, most things can be fixed if configured and setup properly. Just the way the are often used and provided examples encourage many of the problems.

Env vars can be scripted, many companies use a tree of instance/group/project scoped vars though, leading to easily breaking some projects when things higher up change. Solvable for sure, guidelines in companies make it a pain. There are other settings like allowed branch names etc. that can break things.

With security, yes I mean mostly secrets management. Essentially everyone who can push to any branch has access to every token. Or just having a typo or mixing up some variables lead to stuff being pushed to production. Running things in the public cloud is another issue.

Passing artifacts between jobs is a possibility. Still leads to data pushed between machines. Merging jobs is also possible, just defeats the purpose of having multiple jobs and stages. The examples often show a separation between things like linting, testing, building, uploading, etc. so people split it up.

With dependencies I mean everything you need to execute jobs. OS, libraries, tools like curl, npm, poetry, jfrog-cli, whatever. Prebuilt images work, but it is another thing you have to do yourself. Building more containers, storing them, downloading them. Also containers are not composable, so for each project or job has its own. The curse of being stateless and the way Docker works.

Starting containers is not slow on a good runner. But I noticed significant delays on many Kubernetes clusters, even if the nodes are 30s are common. Still, even if it would be faster it is still a delay that quickly adds up if you have many jobs in a pipeline.

I agree that dev machines and runners have different behavior and properties. What I mean is local-first development. For most tasks it is totally fine to run a different version of Postgres, Redis and Go for example. Docker containers bring it even closer to a realistic setup. What I want is quick feedback and being able to see the state of something when there a bugs. Not needing to do print debugging via git push and waiting for pipelines. Pipelines that setup a fresh environment and tear it down after are nice for reproducibility, but prevent me to inspect the system aside from logs and other artifacts. Certainly this doesn't mean you shouldn't have a CI/CD environment at all, especially for releases/production deployments.

Post reply on HN