Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Pain That Is GitHub Actions

feldera.com

Post reply on HN
Print view View original

501–510 of 584 posts

Re: The Pain That Is GitHub Actions

#501

Post by jiggawatts »

maratc wrote: post #327
motorest wrote: post #250

Earlier quoted context omitted.

> iterating a GitHub Actions workflow is a gigantic pain in the ass. Capturing all of the important logic in a script/makefile/whatever means I can iterate it locally way faster and then all I need github to do is provision an environment and call my scripts in the order I require. What's wrong with this? https://docs.github.com/en/actions/writing-workflows/choosin...

Q: How do you determine what date it was 180 days ago? A: Easy! You just spin up a Kubernetes pod with Alpine image, map a couple of files inside, run a bash script of "date" with some parameters, redirect output to a mapped file, and then read the resulting file. That's all. Here's a YAML for you. Configuration, baby! (based on actual events)

At first I assumed you were kidding, then I realised that sadly… you probably weren’t.

Re: The Pain That Is GitHub Actions

#502

Post by julik »

deng wrote: post #86

Already see people saying GitLab is better: yes it is, but it also sucks in different ways. After years of dealing with this (first Jenkins, then GitLab, then GitHub), my takeaway is: * Write as much CI logic as possible in your own code. Does not really matter what you use (shell scripts, make, just, doit, mage, whatever) as long as it is proper, maintainable code. * Invest time that your pipelines can run locally o…

Seconded. Moreover...

> as long as it is proper, maintainable code

...in an imperative language you know well and which has a sufficient amount of type/null checking you can tolerate.

Re: The Pain That Is GitHub Actions

#503

Post by speleding »

claytonjy wrote: post #368
speleding wrote: post #185

Earlier quoted context omitted.

I would like to add one point: * Consider whether it's not easier to do away with CI in the cloud and just build locally on the dev's laptop With fast laptops and Docker you can get perfectly reproducible builds and tests locally that are infinitely easier to debug. It works for us.

How do you ensure what a dev builds and tags and pushes is coherent, meaning the tag matches the code commit it’s expected to? I think builds must be possible locally, but i’d never rely on devs for the source of truth artifacts running in production, past a super early startup.

You can add all kind of verification scripts to git hooks, that trigger before and after someone pushes, like you do with GitHub actions. Whether you trust you devs less than your build pipeline is an organizational issue, but in our org only a few senior devs can merge to master.

Re: The Pain That Is GitHub Actions

#504

Post by julik »

no_wizard wrote: post #349
baq wrote: post #118

Earlier quoted context omitted.

> I know some people might consider this insanity Some people here still can’t believe YAML is used for not only configuration, but complex code like optimized CI pipelines. This is insane. You’re actually introducing much needed sanity into the process by admitting that a real programming language is the tool to use here. I can’t imagine the cognitive dissonance Lisp folks have when dealing with this madness, not be…

I'm shocked there isn't a 'language for config' that hasn't become the de facto standard and its YAML all the way down seemingly. I am with you 100%. It would really benefit from a language that intrinsically understood its being used to control a state machine. As it is, that is what nearly all folks want in practice is a way to run different things based on different states of CI. A lisp DSL would be perfect for th…

I think the issue here is mostly the background the CI setups came from. They were frequently coming from the "ops" part of the ecosystem, and some ops folks held some ideas very strongly back then (I heard this first hand):

"Application and configuration should be separate, ideally in separate repos. It is the admin's job to configure, not the developer's"

"I do not need to learn your garbage language to understand how to deploy or test your application"

"...as a matter of fact, I don't need to learn the code and flows of the application itself either - give me a binary that runs. But it should work with stale configs in my repo."

"...I know language X works for the application but we need something more ubiquitous for infra"

Then there was a crossover of three streams, as I would call it:

YAML was emerging "hard" on the shoulders of Rails

Everyone started hating on XML (and for a good reason)

Folks working on CI services (CruiseControl and other early solutions) and ops tooling (chef, ansible) saw JSON's shortcomings (now an entire ecosystem has configuration files with no capability to put in a comment)

Since everybody hated each other's languages, the lowest common denominator for "configuration code" came out to be YAML, and people begrudgingly agreed to use it

The situation then escalated severely with k8s, which adopted YAML as "the" configuration language, and a whole ecosystem of tooling sprung up on top using textual templating (!) of YAML as a layer of abstraction. For k8s having a configuration language was an acute need, because with a compiled language you need something for configuration that you don't have to compile with the same toolchain just to use - and I perfectly "get it" why they settled for YAML. I do also get why tools like Helm were built on top of YAML trickery - because, be it that Helm were written in some other language, and have its charts use that, they would alienate all the developers that either hate that language personally, or do not have it on the list of "golden permitted" at their org.

Net result is that YAML was chosen not because it is good, but because it is universally terrible in the same way for everyone, and people begrudgingly settled on it.

With CI there is an extra twist that a good CI setup functions as a DAG - some tasks can - and should - run in parallel for optimization. These tasks produce artifacts which can be cached and reused, and a well-set CI pipeline should be able to make use of that.

Consequently, I think a possible escape path - albeit an expensive one - would be for a "next gen" CI system to expose those _task primitives_ via an API that is easy to write SDKs for. Read: not a grpc API. From there, YAML could be ditched as "actual code" would manipulate the CI primitives during build.

Re: The Pain That Is GitHub Actions

#505

Post by carlmr »

deng wrote: post #86

Already see people saying GitLab is better: yes it is, but it also sucks in different ways. After years of dealing with this (first Jenkins, then GitLab, then GitHub), my takeaway is: * Write as much CI logic as possible in your own code. Does not really matter what you use (shell scripts, make, just, doit, mage, whatever) as long as it is proper, maintainable code. * Invest time that your pipelines can run locally o…

>Invest time that your pipelines can run locally on a developer machine as well (as much as possible at least), otherwise testing/debugging pipelines becomes a nightmare.

This so much. This ties into the previous point about using as much shell as possible. Additionally I'd say environment control via Docker/Nix, as well as modularizing the pipeline so you can restart it just before the point of failure instead of rerunning the whole business just to replay one little failure.

Re: The Pain That Is GitHub Actions

#506

Post by dmuso »

mike_hearn wrote: post #499
dmuso wrote: post #409

Earlier quoted context omitted.

Try doing a clean git clone in TeamCity. Nope, not even with the plugins that claim “clean clone” capability. You should be confident that CI can build/run/test an app with a clean starting point. If the CI forces a cached state on an agent that you can’t clear… TeamCity just does it wrong.

You just check the "delete files in checkout directory" box in the run screen. Are you thinking of something different? I've never had trouble doing a clean clone.

It’s been a while since I used it but I do remember that it doesn’t do a clean checkout and you can’t force it to. It leaves artifacts on the agent that can interfere with subsequent builds. I assume they do it for speed but it can affect reliability of builds

Re: The Pain That Is GitHub Actions

#507

Post by oulipo »

Aeolun wrote: post #189
oulipo wrote: post #144

Earlier quoted context omitted.

can you give more feedback about dagger? what is good/not good about it? I was going to start looking into it

I liked their setup before, though I never got around to actually using it, but the tagline on the website has changed to “AI powered workflow orchestration”, which is quite different from the original “Write pipeline once, run everywhere”

yes but I went on their Discord, and the AI thing is more an "extension" of the CI/CD thing, it's just marketing-speak for the investors, they are still building the CI/CD tool

Re: The Pain That Is GitHub Actions

#508

Post by oulipo »

shykes wrote: post #360
jimmcslim wrote: post #251

Earlier quoted context omitted.

In addition to the other comments suggesting dagger is not the saviour due to being VC-funded, it seems like they have decided there's no money in CI, but AI... yes there's money there! And "something something agents". From dagger.io... "The open platform for agentic software. Build powerful, controllable agents on an open ecosystem. Deploy agentic applications with complete visibility and cross-language capabilitie…

Hello! Dagger CEO here. Yes, we discovered that, in addition to running CI pipelines, Dagger can run AI agents. We learned this because our own users have told us. So now we are trying to capitalize on it, hence the ongoing changes to our website. We are trying to avoid the "something something agents" effect, but clearly, we still have work to do there :) It's hard to explain in marketing terms why a ephemeral execu…

I did ask you on your Discord, but perhaps you could summarize here / ELI5 why Dagger would be a great fit for a kind of "MCP-like" operating system?

Re: The Pain That Is GitHub Actions

#509

Post by oulipo »

shykes wrote: post #373
colemannerd wrote: post #369

Earlier quoted context omitted.

Please be careful. I'd love to adopt Dagger, but the UI in comparison to GHA, is just not a value add. I'd hate for y'all to go the AI route that Arc did... and lose all your users. There is A LOT to CICD, which can be profitable. I think there's still a lot more features needed before it's compelling and I would worry Agentic AI will lead you to a hyper-configurable, muddled message.

Thank you. Yes, I worry about muddling the message. We are looking for a way to communicate more clearly on the fundamentals, then layer use cases on top. It is the curse of all general-purpose platforms (we had the same problem with Docker). The risk of muddling is limited to the marketing, though. It's the exact same product powering both use cases. We would not even consider this expansion if it wasn't the case. F…

On another note, I'd love if you open-sourced your timeline / log UI component :) it's quite pretty, and could be useful in many contexts haha

Or I'll ask v0.dev to reimplement it, but I think it'd be more complete if you did it

Re: The Pain That Is GitHub Actions

#510

Post by oulipo »

shykes wrote: post #373
colemannerd wrote: post #369

Earlier quoted context omitted.

Please be careful. I'd love to adopt Dagger, but the UI in comparison to GHA, is just not a value add. I'd hate for y'all to go the AI route that Arc did... and lose all your users. There is A LOT to CICD, which can be profitable. I think there's still a lot more features needed before it's compelling and I would worry Agentic AI will lead you to a hyper-configurable, muddled message.

Thank you. Yes, I worry about muddling the message. We are looking for a way to communicate more clearly on the fundamentals, then layer use cases on top. It is the curse of all general-purpose platforms (we had the same problem with Docker). The risk of muddling is limited to the marketing, though. It's the exact same product powering both use cases. We would not even consider this expansion if it wasn't the case. F…

Hmmmm... so I think the crux of the matter is here: that you clearly articulate why your platform (for both containers and agents) is really helpful to handle cases where there are both states and side-effects

I can understand what you're trying to say, but because I don't have clear "examples" at hand which show me why in practice handling such cases are problematic and why your platform makes that smooth, I don't "immediately" see the value-added

For me right now, the biggest "value-added" that I perceive from your platform is just the "CI/CD as code", a bit the same as say Pulumi vs Terraform

But I don't see clearly the other differences that you mention (eg observability is nice, but it's more "sugar" on top, not a big thing)

I have the feeling that indeed the clean handling of "state" vs "side-effects" (and what it implies for caching / retries / etc) is probably the real value here, but I fail to perceive it clearly (mostly because I probably don't (or not yet) have those issues in my build pipelines)

If you were to give a few examples / ELI5 of this, it would probably help convert more people (eg: I would definitely adopt a "clean by default" way of doing things if I knew it would help me down the road when some new complex-to-handle use-cases will inevitably pop up)

Post reply on HN