Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern CI is too complex and misdirected (2021)

gregoryszorc.com

Post reply on HN
Print view View original

101–110 of 207 posts

Re: Modern CI is too complex and misdirected (2021)

#101

Post by wink »

sluongng wrote: post #74

The most concerning part about modern CI to me is how most of it is running on GitHub Actions, and how GitHub itself has been deprioritizing GitHub Actions maintenance and improvements over AI features. Seriously, take a look at their pinned repo: https://github.com/actions/starter-workflows > Thank you for your interest in this GitHub repo, however, right now we are not taking contributions. > We continue to focus o…

The last time the company I worked for was hosting code on Github, Actions did not exist yet and for personal stuff copying some 3 liners was fine, I'd hardly call that "using".

"Github Actions might be over, so not worth engaging" was not on my bingo card.

Re: Modern CI is too complex and misdirected (2021)

#102

Post by woodruffw »

captn3m0 wrote: post #90
sluongng wrote: post #74

The most concerning part about modern CI to me is how most of it is running on GitHub Actions, and how GitHub itself has been deprioritizing GitHub Actions maintenance and improvements over AI features. Seriously, take a look at their pinned repo: https://github.com/actions/starter-workflows > Thank you for your interest in this GitHub repo, however, right now we are not taking contributions. > We continue to focus o…

They are instead focusing on Agentic Workflows which used natural language instead of YAML. https://github.com/githubnext/gh-aw

I personally find this pretty concerning: GitHub Actions already has a complex and opaque security model, and adding LLMs into the mix seems like a perfect way to keep up the recent streak of major compromises driven by vulnerable workflows and actions.

I would hope that this comes with major changes to GHA’s permissions system, but I’m not holding my breath for that.

Re: Modern CI is too complex and misdirected (2021)

#103

Post by mikepurvis »

DrBazza wrote: post #64
MathMonkeyMan wrote: post #15

I remember a Rich Hickey talk where he described Datomic, his database. He said "the problem with a database is that it's over there ." By modeling data with immutable "facts" (a la Prolog), much of the database logic can be moved closer to the application. In his case, with Clojure's data structures. Maybe the the problem with CI is that it's over there . As soon as it stops being something that I could set up and r…

Your build should be this: build.bash and that's it (and that can even trigger a container build). I've spent far too much time debugging CI builds that work differently to a local build, and it's always because of extra nonsense added to the CI server somehow. I've yet to find a build in my industry that doesn't yield to this 'pattern'. Your environment setup should work equally on a local machine or a CI/CD server,…

There are various things that can be a reasonable candidate for the "top level" build entrypoint, including Nix, bazel, docker bake, and probably more I'm not thinking of. They all have an entrypoint that doesn't have a ton of flags or nonsense, and operate in a pretty self contained environment that they set up and manage themselves.

Overall I'm not a fan of wrapping things; if there are flags or options on the top-level build tool, I'd rather my devs explore those and get used to what they are and can do, rather than being reliant on a project-specific script or make target to just magically do the thing.

Anyway, other than calling the build tool, CI config can have other steps in it, but it should be mostly consumed with CI-specific add-ons, like auth (OIDC handshake), capturing logs, uploading artifacts, sending a slack notification, whatever it is.

Re: Modern CI is too complex and misdirected (2021)

#104

Post by chubot »

nrclark wrote: post #89
DrBazza wrote: post #64

Earlier quoted context omitted.

Your build should be this: build.bash and that's it (and that can even trigger a container build). I've spent far too much time debugging CI builds that work differently to a local build, and it's always because of extra nonsense added to the CI server somehow. I've yet to find a build in my industry that doesn't yield to this 'pattern'. Your environment setup should work equally on a local machine or a CI/CD server,…

Agreed with this sentiment, but with one minor modification: use a Makefile instead. Recipes are still chunks of shell, and they don’t need to produce or consume any files if you want to keep it all task-based. You get tab-completion, parallelism, a DAG, and the ability to start anywhere on the task graph that you want. It’s possible to do all of this with a pure shell script, but then you’re probably reimplementing…

Make is not a general purpose parallel DAG engine. It works well enough for small C projects and similar, but for problems of even medium complexity, it falls down HARD

Many years ago, I wrote 3 makefiles from scratch as an exploration of this (and I still use them). I described the issues here: https://lobste.rs/s/yd7mzj/developing_our_position_on_ai#c_s...

---

The better style is in a sibling reply -- invoke Make from shell, WHEN you have a problem that fits Make.

That is, the "main" should be shell, not Make. (And it's easy to write a dispatcher to different shell functions, with "$@", sometimes called a "task file" )

In general, a project's CI does not fit entirely into Make. For example, the CI for https://oils.pub/ is 4K lines of shell, and minimal YAML (portable to Github Actions and sourcehut).

https://oils.pub/release/latest/pub/metrics.wwz/line-counts/...

It invokes Make in a couple places, but I plan to get rid of all the Make in favor of Python/Ninja.

Re: Modern CI is too complex and misdirected (2021)

#105

Post by kqr »

KronisLV wrote: post #33
MathMonkeyMan wrote: post #15

I remember a Rich Hickey talk where he described Datomic, his database. He said "the problem with a database is that it's over there ." By modeling data with immutable "facts" (a la Prolog), much of the database logic can be moved closer to the application. In his case, with Clojure's data structures. Maybe the the problem with CI is that it's over there . As soon as it stops being something that I could set up and r…

> Part of the problem, maybe the whole problem, is that we could get it all working and portable and optimized for non-blessed environments, but it still will only be expected to work over there, and so the frog keeps boiling. Build the software inside of containers (or VMs, I guess): a fresh environment for every build, any caches or previous build artefacts explicitly mounted. Then, have something like this, so tho…

This sounds a lot like "use Nix".

Re: Modern CI is too complex and misdirected (2021)

#106

Post by chubot »

sambuccid wrote: post #50

I'm not sure why no one mentioned it yet, but the CI tool of sourcehut ( https://man.sr.ht/builds.sr.ht/ ) simplifies all of this. It just spins a linux distro of your choice, and executes a very bare bone yml that essentially contains a lot of shell commands, so it's also easy to replicate locally. There are 12 yml keywords in total that cover everything. Other cool things are the ability to ssh in a build if it fai…

BTW if you follow the philosophy of using bash as your CI so it runs locally (mentioned by several people in this thread), then you can use the same CI logic on sourcehut and Github Actions.

Both of them provide VMs where you can run anything, and bash is of course there on every image.

We do that for https://oils.pub/

sourcehut yaml: https://github.com/oils-for-unix/oils/tree/master/.builds

github yaml: https://github.com/oils-for-unix/oils/tree/master/.github/wo...

They both call the same shell. The differences are:

* We use Github's API to merge on green; right now we don't have the same for sourcehut (since Github is the primary repo)

* Github Actions provides way more resources. They are kind of "locking projects in" by being free.

This post on NixOS gives a hint of htat

https://blog.erethon.com/blog/2025/07/31/how-nixos-is-built/

The monthly cost for all the actions in July of 2025 came out to a bit over 14500 USD which GitHub covers in its entirety.

So I think many projects are gradually sucked in to Github because it is indeed quite generous (including us, which annoys me -- we run more tasks on Github than sourcehut, even though in theory we could run all on sourcehut)

---

BUT I think it is a good idea to gradually consolidate your logic into shell, so you can move off Github in the future. Open source projects tend to last longer than cloud services.

This already happened to us -- we started using Travis CI in 2018 or so, and by 2021, it was acquired and the free tier was removed

Re: Modern CI is too complex and misdirected (2021)

#107

Post by kstrauser »

captn3m0 wrote: post #90
sluongng wrote: post #74

The most concerning part about modern CI to me is how most of it is running on GitHub Actions, and how GitHub itself has been deprioritizing GitHub Actions maintenance and improvements over AI features. Seriously, take a look at their pinned repo: https://github.com/actions/starter-workflows > Thank you for your interest in this GitHub repo, however, right now we are not taking contributions. > We continue to focus o…

They are instead focusing on Agentic Workflows which used natural language instead of YAML. https://github.com/githubnext/gh-aw

Know what I love in a good build system? Nondeterminism! Who needs coffee when you can get your thrills from stochastic processes. Why settle for just non-repeatable builds when you can have non-repeatable build failures!

Re: Modern CI is too complex and misdirected (2021)

#108

Post by stonemetal12 »

franga2000 wrote: post #91
mettamage wrote: post #85

Earlier quoted context omitted.

> you likely do need zero-downtime deployment I know SaaS businesses that don't as they operate in a single country, within a single timezone and the availability needs to be during business days and business hours. > easy rollbacks Yea, I haven't seen exceptions at all on this. So yea. > server fault tolerance That really depends. Many B2B or internal apps are fine with a few hours, or even a day, of downtime. > ser…

I'm definitely curious about alternatives for getting these features without k8s. Frankly, I don't like it, but I use it because it's the easiest way I've found to get all of these features. Every deployment I've seen that didn't use containers and something like k8s either didn't have a lot of these features, implemented them with a bespoke pile of shell scripts, or a mix of both. For context, I work in exactly that…

>I use it because it's the easiest way I've found to get all of these features. Every deployment I've seen that didn't use containers and something like k8s either didn't have a lot of these features, implemented them with a bespoke pile of shell scripts, or a mix of both.

Features aren't pokemon you don't have to catch them all.

Back when stackoverflow was cool and they talked about their infrastructure, they were running the whole site at 5 9s on 10-20 boxes. For a setup like that k8s would have A) required more hardware B) a complete rewrite of their system to k8sify it C) delivered no additional value.

k8s does good things if you have multiple datacenters worth of hardware to manage, for everyone else it adds overhead for features you don't really need.

Re: Modern CI is too complex and misdirected (2021)

#109

Post by kqr »

mettamage wrote: post #85
franga2000 wrote: post #38

Earlier quoted context omitted.

Requirements are complex too. Even if you don't need to scale at all, you likely do need zero-downtime deployment, easy rollbacks, server fault tolerance, service isolation... If you put your apps into containers and throw them onto Kubernetes, you get a lot of that "for free" and in a well-known and well-tested way. Hand-rolling even one of those things, let alone all of them together, would take far too much effort…

> you likely do need zero-downtime deployment I know SaaS businesses that don't as they operate in a single country, within a single timezone and the availability needs to be during business days and business hours. > easy rollbacks Yea, I haven't seen exceptions at all on this. So yea. > server fault tolerance That really depends. Many B2B or internal apps are fine with a few hours, or even a day, of downtime. > ser…

> I know SaaS businesses that don't as they operate in a single country, within a single timezone and the availability needs to be during business days and business hours.

This is a bad road to go down. Management will understand the implication that it's okay to reduce reliability requirements because "we'll just do the dangerous things on the weekends!"

After some time, developers are scheduled every other weekend and when something breaks during daytime, it's not going to be a smooth process to get it up again, because the process has always been exercised with 48 hours to spare.

Then at some point it's "Can we deploy the new version this weekend?" "No, our $important_customer have their yearly reporting next week, and then we have that important sales demo, so we'll hold off another month on the deployment." You get further and further away from continuous integration.

Re: Modern CI is too complex and misdirected (2021)

#110

Post by donatj »

homebrewer wrote: post #67
donatj wrote: post #59

Drone was absolutely perfect back when it was Free Software. Literally "run these commands in this docker container on these events" and basically nothing more. We ran the last fully open source version much longer than we probably should have. When they went commercial, GitHub Actions became the obvious choice, but it's just married to so much weirdness and unpredictability. Whole thing with Drone opened my eyes at…

It lives on as Woodpecker, the fork of the last truly free version. As simple as it gets, no CLAs required to contribute.

I saw that. We've sadly got a corporate mandate right now to move everyone to GitHub Actions
Post reply on HN