Live data from Hacker News

GitHub Actions is slowly killing engineering teams

iankduncan.com

41–50 of 219 posts

Re: GitHub Actions is slowly killing engineering teams

#41

Earlier quoted context omitted.

The worst part about CDK is, by far , that it's still backed by Cloudformation.

What pains are you experiencing? Cdk has far exceeded Ansible and Terraform in my experience.

Hooo boy where do I begin? Dependency deadlocks are the big one - you try to share resource attributes (eg ARN) from one stack to another. You remove the consumer and go to deploy again. The producer sees no more dependency so it prunes the export. But it can't delete the export, cause the consumer still needs it. You can't deploy the consumer, because the producer has to deploy first sequentially. And if you can't delete the consumer (eg your company mandates a CI pipeline deploy for everything) you gotta go bug Ops on slack, wait for someone who has the right perms to delete it, then redeploy.

You can't actually read real values from Parameters/exports (you get a token placeholder) so you can't store JSON then read it back and decode (unless in same stack, which is almost pointless). You can do some hacks with Fn:: though.

Deploying certain resources that have names specified (vs generated) often breaks because it has to create the new resource before destroying the old one, which it can't, because the name conflicts (it's the same name...cause it's the same construct).

It's wildly powerful though, which is great. But we have basically had to create our own internal library to solve what should be non-problems in an IaC system.

Would be hilarious if my coworker stumbled upon this. I know he reads hn and this has been my absolute crusade this quarter.

Re: GitHub Actions is slowly killing engineering teams

#43
post #8

> If you’re a small team with a simple app and straightforward tests, it’s probably fine. I’m not going to tell you to rip it out. > But if you’re running a real production system, if you have a monorepo, if your builds take more than five minutes, if you care about supply chain security, if you want to actually own your CI: look at Buildkite. Goes in line with exactly what I said in 2020 [0] about GitHub vs Self-hos…

I know this is off topic, but that homepage is a piece of work: https://buildkite.com I get it's quirky, but I'm at a low energy state and just wanted to know what it does... Right before I churned out, I happened to click "[E] Exit to classic Buildkite" and get sent to their original homepage: https://buildkite.com/platform/ It just tells you what it Buildkite does! Sure it looks default B2B SaaS, but more important…

Hello mate, Head of Brand and Design at BK here. Thanks for the feedback, genuinely; the homepage experiment has been divisive, in a great way. Some folk love it, some folk hate it, some just can't be bothered with it. All fair.

Glad that the classic site hit the mark, but a lot work to do to make that clearer than it is; we're working on the next iteration that will sunset the CLI homepage into an easter egg.

Happy to take more critique, either on the execution or the rabbit hole.

Re: GitHub Actions is slowly killing engineering teams

#44
post #21

I tend to disagree with this as it seems like an ad for Nix/Buildkite... If your CI invocations are anything more than running a script or a target on a build tool (make, etc.) where the real build/test steps exist and can be run locally on a dev workstation, you're making the CI system much more complex than it needs to be. CI jobs should at most provide an environment and configuration (credentials, endpoints, etc.…

Can 100% confirm this is not an ad (at least not for Buildkite) and was a lovely surprise to read for the team.

Re: GitHub Actions is slowly killing engineering teams

#45
post #26

I hope the author will check out RWX -- they say they've checked out most CI systems, but I don't think they've tried us out yet. We have everything they praise Buildkite for, except for managing your own compute (and that's coming, soon!). But we also built our own container execution model with CI specifically in mind. We've seen one too many Buildkite pipelines that have a 10 minute Docker build up front (!) and t…

Can you explain how your product solves this problem? I clicked around your site and couldn't figure it out.

As a (very happy) RWX customer:

- Intermediate tasks are cached in a docker-like manner (content-addressed by filesystem and environment). Tasks in a CI pipeline build on previous ones by applying the filesystem of dependent tasks (AFAIU via overlayfs), so you don't execute the same task twice. The most prominent example of this is a feature branch that is up-to-date with main passes CI on main as soon as it's merged, as every task on main is a cache-hit with the CI execution on the feature branch.

- Failures: the UI surfaces failures to the top, and because of the caching semantics, you can re-run just the failed tasks without having to re-run their dependencies.

- Debugging: they expose a breakpoint (https://www.rwx.com/docs/rwx/remote-debugging) command that stops execution during a task and allows you to shell into the remote container for debugging, so you can debug interactively rather than pushing `env` and other debugging tasks again and again. And when you do need to push to test a fix, the caching semantics again mean you skip all the setup.

There's a whole lot of other stuff. You can generate tasks to execute in a CI pipeline via any programming language of your choice, the concurrency control supports multiple modes, no need for `actions/cache` because of the caching semantics and the incremental caching feature (https://www.rwx.com/docs/rwx/tool-caches).

And I've never had a problem with the logs.

Re: GitHub Actions is slowly killing engineering teams

#46

I agree with all the points made about GH actions. I haven't used as many CI systems as the author, but I've used, GH actions, Gitlab CI, CodeBuild, and spent a lot of time with Jenkins. I've only touched Buildkite briefly 6 years ago, at the time it seemed a little underwhelming. The CI system I enjoyed the most was TeamCity, sadly I've only used it at one job for about a year, but it felt like something built by a…

tc is probably the best console runner there is and I agree, it made CI not suck. It is also possible to make it very fast, with a bit of engineering and by hosting it on your own hardware. Unfortunately it’s as legacy as Jenkins today. And in contrast to Jenkins it’s not open source or free, many parts of it, like the scheduler/orchestrator, is not pluggable.

But I don’t know about competent people, reading their release notes always got me thinking ”how can anyone write code where these bugs are even possible?”. But I guess that’s why many companies just write nonsense release notes today, to hide their incompetence ;)

Re: GitHub Actions is slowly killing engineering teams

#47
post #25

Dead on. GitHub Actions is the worst CI tool I’ve ever used (maybe tied with Jenkins) and Buildkite is the best. Buildkite’s dynamic pipelines (the last item in the post) are so amazingly useful you’ll wonder how you ever did without them. You can do super cool things like have your unit test step spawn a test de-flaking step only if a test fails. Or control test parallelism based on the code changes you’re testing.…

Jenkins had a lot of issues and I’m glad to not be using it overall, but I did like defining pipelines in Groovy and I’ll take Groovy over YAML all day.

Re: GitHub Actions is slowly killing engineering teams

#48
I just can't stand using a build system tied to the code host. And that is really because I have an aversion to vendor lock-in.

webhooks to an external system was such a better way to do it, and somehow we got away from that, because they don't want us to leave.

webhooks are to podcasts as github actions are to the things that spotify calls podcasts.

Re: GitHub Actions is slowly killing engineering teams

#49

Earlier quoted context omitted.

The worst part about CDK is, by far , that it's still backed by Cloudformation.

What pains are you experiencing? Cdk has far exceeded Ansible and Terraform in my experience.

I'll just echo the other poster with "deadlocks". It's obscene how slow CF is, and the fact that its failure modes often leave you in a state that feels extremely dangerous. I've had to contact AWS Support before due to CF locking up in an irrecoverable way due to cycles.

Re: GitHub Actions is slowly killing engineering teams

#50
Personally I like Drone more than Buildkite. It's as close to a perfect CI system as I've seen; just complex enough to do everything I need, with a design so stripped-down it can't be simpler. I occasionally check on WoodpeckerCI to see if it's reached parity with Drone. Now that AI coding is a thing, hopefully that'll happen soon
Post reply on HN