Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Actions could be so much better

blog.yossarian.net

Post reply on HN
Print view View original

211–220 of 238 posts

Re: GitHub Actions could be so much better

#211

Post by toastal »

Huggernaut wrote: post #192
solatic wrote: post #145

GitHub Actions is a horrible CI/CD system. You cannot run steps in parallel on the same VM; container-based workloads are a second-class citizen. The first problem means that setting up local credentials and other environment dependencies cannot be parallelized (I'm looking at you, google-github-actions/setup-gcloud, with your 1m+ runtime... grrr), the second makes it quite difficult to put a Dockerfile in a reposito…

Less Pivotal and more VMWare post acquisition dismantling I'd say. There was a lot of love internally for Concourse (I left before the acquisition though). SSH debugging and one off tasks absolutely dreamy.

Even SourceHut’s Spartan CI supports SSH debugging

Re: GitHub Actions could be so much better

#212

Post by solatic »

Huggernaut wrote: post #191
speed_spread wrote: post #187

Earlier quoted context omitted.

Concourse rocks. I didn't know the team had been dismantled, this sucks. Zito's communication style was the best.

Vito is at dagger.io now so hopefully we can expect some good stuff in the CI space there.

Sadly, Dagger doesn't get it either. It's so focused on portability between the underlying infrastructure providers, on not being the underlying infrastructure provider, and therefore it doesn't solve the real problem, which is the underlying infrastructure provider.

(a) Consider Dagger's integration with GitHub Actions: https://docs.dagger.io/cookbook#github-actions where you anyway need to run setup-node, npm ci, etc. just to start the Dagger pipeline. So Dagger isn't saving you from the having to deal with GitHub Actions' caching layer and always-blank starting point - it's unavoidable. Well, if I can't avoid it, why should I use Dagger in the first place - why not embrace it?

(b) Consider a usecase where I want to parallelize the computation onto a number of machines dynamically chosen at run-time. Maybe I want to allow a test suite to run on an increasing number of machines without needing to periodically manually increase the number of machines in a configuration file, or maybe I'm using Terraform workspaces where I want to run terraform apply for each workspace on a different VM to let the number of workspaces scale horizontally. This is fundamentally impossible with something like Dagger (also impossible in GitHub Actions) because it would require Dagger to communicate with the infrastructure provider to tell it to scale up compute to handle the parallel jobs, and then scale down once those jobs finish.

This was achievable with Concourse by having Concourse pipelines generate other Concourse pipelines, and running the underlying Concourse workers as an autoscaling Kubernetes statefulset/deployment, combined with other Kubernetes implements like cluster autoscaler.

Re: GitHub Actions could be so much better

#213

Post by solatic »

txtsd wrote: post #185
solatic wrote: post #145

GitHub Actions is a horrible CI/CD system. You cannot run steps in parallel on the same VM; container-based workloads are a second-class citizen. The first problem means that setting up local credentials and other environment dependencies cannot be parallelized (I'm looking at you, google-github-actions/setup-gcloud, with your 1m+ runtime... grrr), the second makes it quite difficult to put a Dockerfile in a reposito…

Try the sourcehut build server

It's a public alpha and clearly targeted for small hobbyists / FOSS work as a result. I'm looking for something where the creator has more confidence in its reliability...

Re: GitHub Actions could be so much better

#214

Post by globular-toast »

pxeger1 wrote: post #98

I would like to give a strong recommendation for https://pre-commit.ci (no relation, just a happy user). The idea is that you write hooks (or use pre-existing ones) that check (and fix, where possible) your code before you commit, and then once you've pushed, the CI will check again, including automatically committing fixes if necessary. Anyway, it works brilliantly - unbelievably fast thanks to good automatic cachin…

This combined with tox is great for Python projects in particular. Tox automates creating virtualenvs for the right Python versions you want to test with then running your tests in them. It can also run static checks by issuing `skip_install = True` because you want to test the source code itself. You just need to run this in a Python container that has tox installed as a globally available tool and all versions of Python available in it (like https://github.com/georgek/docker-python-multiversion Here's some boilerplate to do all that:

    [tox]
    envlist = py{310,311}

    [testenv]
    passenv =
            PIP_CACHE_DIR
    deps = coverage   # deps only for tox
    extras = testing  # testing extras include pytest
    commands = pytest ...

    [testenv:check]
    passenv =
            PIP_CACHE_DIR
            PRE_COMMIT_HOME
    skip_install = True
    deps = pre-commit
    commands = pre-commit run --all-files --show-diff-on-failure

Re: GitHub Actions could be so much better

#215

Post by flohofwoe »

MuffinFlavored wrote: post #179
MoreQARespect wrote: post #24

There are two types of github actions workflows you can build. 1) Program with github actions. Google "how can I send an email with github actions?" and then plug in some marketplace tool to do it. Your workflows grow to 500-1000 lines and start having all sorts of nonsense like conditionals and the YAML becomes disgusting and hard to understand. Github actions becomes a nightmare and you've invited vendor lock in. 2…

> our workflow ends up being about 50-60 lines as a result and very rarely needs to be changed once you've set up. As in, use GitHub Actions as a YAML wrapper around bash/zsh/sh scripts?

It can be any scripting language, Python or Typescript via Deno are good choices because they have batteries-included cross-platform standard libs and are trivial to setup.

Python is actually preinstalled on Github CI runners.

Re: GitHub Actions could be so much better

#216

Post by devoutsalsa »

kitallis wrote: post #106
andix wrote: post #64

Earlier quoted context omitted.

If I'm fixing CI I always put it on a feature branch and do a squash merge once I'm done. Because it's never just one quick fix, it's always 3-10 commits.

> If I'm fixing CI I always put it on a feature branch and do a squash merge once I'm done. Because it's never just one quick fix, it's always 3-10 commits. The problem is GA also does not allow you to commit a new workflow in a branch. It must first exist on your primary branch and then you may tweak it in another.

If I have to commit several trash commits, I’m happy to squash locally and then do

    git push —force-with-lease origin main

Re: GitHub Actions could be so much better

#217

Post by rijoja »

k8svet wrote: post #137

I mean, it's embarrassing how bad it is. - (unrelated) build failures just randomly notifies the latest maintainer who happened to merge something? (Imagine you finding this out when your newly added maintainer pings you on Matrix and tells you 1: about this behavior, and 2: that your update/builds have been failing for a week without you knowing?!?! ) - The cache action is horribly, trivially observably broken with…

Wasn't it obvious that something along these lines would happen when Microsoft took over Github?

Re: GitHub Actions could be so much better

#218

Post by lenkite »

lpapez wrote: post #197
lenkite wrote: post #176

I wish Github Actions were Lua instead of YAML. One could use a restricted environment populated with predefined functions, a restricted set of modules and all the advantages of syntax checking and language help.

Congratulations, you just reinvented Jenkins but using Lua instead of Groovy.

Not a bad thing. Also Lua is both smaller, saner and more performant than Groovy. (with LuaJIT)

Re: GitHub Actions could be so much better

#219

Post by lawnchair »

solatic wrote: post #145

GitHub Actions is a horrible CI/CD system. You cannot run steps in parallel on the same VM; container-based workloads are a second-class citizen. The first problem means that setting up local credentials and other environment dependencies cannot be parallelized (I'm looking at you, google-github-actions/setup-gcloud, with your 1m+ runtime... grrr), the second makes it quite difficult to put a Dockerfile in a reposito…

solatic -- I have an existing solution that accounts for a lot of these issues you bring up. Would it be possible to pick your brain? Can you share your email or shoot me an email? lawnchair@lawnchair.net.

Re: GitHub Actions could be so much better

#220

Post by cdaringe »

MoreQARespect wrote: post #24

There are two types of github actions workflows you can build. 1) Program with github actions. Google "how can I send an email with github actions?" and then plug in some marketplace tool to do it. Your workflows grow to 500-1000 lines and start having all sorts of nonsense like conditionals and the YAML becomes disgusting and hard to understand. Github actions becomes a nightmare and you've invited vendor lock in. 2…

There’s a curve. Stringy, declarative DSLs have high utility when used in linear, unconditional, stateless programming contexts.

Adding state? Adding conditionals? Adding (more than a couple) procedure calls?

These concepts perform poorly without common programming tools: testing (via compilation or development runtime), static analysis, intellisense, etc etc

Imagine the curve:

X axis is (vaguely) LinesOfYaml (lines of dsl, really) Y axis is tool selection. Positive region of axis is “use a DSL”, lower region is “use a GeneralPurposeProgrammingLanguage”

The line starts at the origin, has a SMALL positive bump, than plummets downwards near vertically.

Gets it right? Tools like ocurrent (contrasted against GH actions) [1], cdk (contrasted against TF yaml) [2]

Gets it wrong? Well, see parent post. This made me so crazy at work (where seemingly everyone has been drinking the yaml dsl koolaide) that i built a local product simulator and yaml generator for their systems because “coding” against the product was so untenable.

[1] https://github.com/ocurrent/ocurrent/blob/master/doc/example... [2] https://docs.aws.amazon.com/cdk/v2/guide/getting_started.htm...

Post reply on HN