Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run GitHub Actions locally

github.com

Post reply on HN
Print view View original

121–130 of 137 posts

Re: Run GitHub Actions locally

#121

Post by digianarchist »

mdaniel wrote: post #73
digianarchist wrote: post #60

Earlier quoted context omitted.

Like https://docs.gitlab.com/runner/ ?

I'm as big a GitLab fanboy as they come, but they recently axed the gitlab-runner binary's ability to execute local .gitlab-ci.yml files https://gitlab.com/gitlab-org/gitlab/-/issues/385235 >. It now only operates in "receive webhook from the mothership" mode, just like GHA Pour one out, I guess, but it's okay since I previously was super angry at it for languishing in the uncanny valley of "hello world works, practi…

Disappointing. That was very useful for troubleshooting issues.

Re: Run GitHub Actions locally

#122

Post by pseudocomposer »

hk1337 wrote: post #111
philistine wrote: post #110

Earlier quoted context omitted.

My experience with Ruby on Apple Silicon as been far from seamless. The only way I could thoroughly get rid of problems was by running the most recent Ruby release and dealing with what THAT causes. I’m pretty sure GH actions don’t run the latest Ruby version.

My experience with both Ruby and Python Apple Silicon has been hit or miss. Mise has made it a lot simpler and seamless but if you need an older version or the absolute latest when it is released, you might have some issues. Mise has helped a lot, I generally do not expect to get an error.

Are there specific problems? I’ve been primarily Ruby at work the last 6 or so years (before, also doing Ruby, but also lots of Java, Kotlin, Swift, FE stuff, and some Python)… exactly that period… and haven’t really noticed any issues. Granted, I don’t typically use a lot of obscure binary-only dependencies, or things whose C/C++/machine code wouldn’t mostly cross-compile fine.

Granted, all this time I’ve always only used rbenv-managed Ruby versions (and rvm before that). I’ve long disliked/avoided Python because its ecosystem lacked any equivalent, but I know “uv” has gained popularity there, and perhaps Mise is good in Python land as well.

Re: Run GitHub Actions locally

#123

Post by OptionOfT »

mdaniel wrote: post #119
OptionOfT wrote: post #80

Earlier quoted context omitted.

No, as their MDM solution only works on Macs.

In fairness, the opposite way usually ends up being much cheaper: make prod run arm64 I would guess that just `alias docker="docker --platform=linux/amd64"` type thing would go a long way toward evicting those platform mismatches[1]. I would also guess there's a way to switch off the binfmt "helper" in colima such that trying to run arm64 in the VM would immediately puke rather than silently helping you 1: or, of cou…

All of this is an effort to validate that building works on all environments.

> alias docker="docker --platform=linux/amd64"

I think that on its own breaks a Rust project which uses openssl with the vendored feature.

Re: Run GitHub Actions locally

#124

Post by nitwit005 »

jiggawatts wrote: post #54
cadamsdotcom wrote: post #51

Rather than tying CI & deployments to Github Actions, it is usually better to pull as much of it as possible out to shell scripts and call them in containers in GH actions.. There are optimizations you’ll want (caching downloaded dependencies etc); if you wait to make those after your build is CI-agnostic you’ll be less tempted by vendor specific shortcuts. Usually means more code - but, easier to test locally. Also,…

This is always the top comment in these kinds of threads, and I see this as an indication that the current state of CI/CD is pathetically propriety. It’s like the dark times before free and open source compilers. When are we going to push back and say enough is enough!? CI/CD desperately needs something akin to Kubernetes to claw back our control and ability to work locally. Personally, I’m fed up with pipeline devel…

Having a container makes debugging possible, but it's still generally going to be an unfriendly experience, compared to a script you can just run and debug immediately.

It's inevitable that things will be more difficult to debug once you're using a third party asynchronous tool as part of the flow.

Re: Run GitHub Actions locally

#125

Post by pxc »

manx wrote: post #43
Kinrany wrote: post #42

Every mention of Github Actions is an occasion to start looking for the best alternative in , let's go! Is Dagger usable yet? Is there still hope for Earthly? Are general purpose workflow systems winning any time soon, or are we still afraid of writing code? Any new systems out there that cover all the totally basic features like running locally, unlike Github Actions?

I think we could build something on top of nix that is as easy to use and powerful as earthly, but gets all the nice stuff from nix: reproducibility, caching, just use any command from any nix package, etc

I built something like this out using an old version of Dagger and found it enormously complicated, and the they rewrote everything and abandoned the version of Dagger I used.

When I they did, I said "fuck it" and just started distributing a Nix flake with wrappers for all the tools I want to run in CI so that at least that part gets handled safely and is easy to run locally.

The worst and most annoying stuff to test is the CI platforms' accursed little pseudo-YAML DSLs. I still would like that piece.

devenv.sh tasks might be a good basis for building on it. I think maybe bob.build also wants to be like this

Re: Run GitHub Actions locally

#126

Post by pxc »

OptionOfT wrote: post #75

I've had the displeasure of working with Macs on which we wrote code that would end up being a docker container. But what is compiled on a Mac is an aarch64 container. What is compiled on an AMD64 Linux machine is amd64. This is annoying when the code you write is native or reaches out to native stuff. Then all of the sudden your container that builds perfectly on the Mac doesn't build on Linux anymore. (and vice ver…

> And their Mac runners do not come with Docker, and it cannot be installed.

?????

If you have a Docker runner on macOS, you're just running a Linux runner in a VM. So just register a Linux runner in a VM, right?

Re: Run GitHub Actions locally

#127

Post by pxc »

droelf wrote: post #23

I think this is really cool. We're tackling this problem from the other side by building `pixi` (pixi.sh) which bundles project / package management with a task runner so that any CI should be as simple as `pixi run test` and easy to execute locally or in the cloud.

My team has a setup that sounds essentially the same using Nix via devenv.sh. We deterministically bundle and run everything from OpenTofu and all its providers to our programming languages runtimes to our pre-commit hooks this way, and it also features a task runner that builds a dependency graph and runs things in parallel and so on.

Our commands for CI are all just one liners that go to wrappers than pin all our dependencies.

Lately I've been working with a lot of cross-platform Bash scripts that run natively on macOS, WSL, and Linux servers, with little to no consideration for the differences. It's been good!

Re: Run GitHub Actions locally

#128

Post by hk1337 »

pseudocomposer wrote: post #122
hk1337 wrote: post #111

Earlier quoted context omitted.

My experience with both Ruby and Python Apple Silicon has been hit or miss. Mise has made it a lot simpler and seamless but if you need an older version or the absolute latest when it is released, you might have some issues. Mise has helped a lot, I generally do not expect to get an error.

Are there specific problems? I’ve been primarily Ruby at work the last 6 or so years (before, also doing Ruby, but also lots of Java, Kotlin, Swift, FE stuff, and some Python)… exactly that period… and haven’t really noticed any issues. Granted, I don’t typically use a lot of obscure binary-only dependencies, or things whose C/C++/machine code wouldn’t mostly cross-compile fine. Granted, all this time I’ve always onl…

Usually just errors building it. Mostly python but I think I had an error with 3.4.3 the other day.

Re: Run GitHub Actions locally

#129

Post by sea-gold »

pxc wrote: post #125
manx wrote: post #43

Earlier quoted context omitted.

I think we could build something on top of nix that is as easy to use and powerful as earthly, but gets all the nice stuff from nix: reproducibility, caching, just use any command from any nix package, etc

I built something like this out using an old version of Dagger and found it enormously complicated, and the they rewrote everything and abandoned the version of Dagger I used. When I they did, I said "fuck it" and just started distributing a Nix flake with wrappers for all the tools I want to run in CI so that at least that part gets handled safely and is easy to run locally. The worst and most annoying stuff to test…

In addition to https://devenv.sh and https://bob.build , here are 2 others that are based on Nix.

- https://flox.dev

- https://www.jetify.com/devbox

Re: Run GitHub Actions locally

#130

Post by z_mitchell »

manx wrote: post #43
Kinrany wrote: post #42

Every mention of Github Actions is an occasion to start looking for the best alternative in , let's go! Is Dagger usable yet? Is there still hope for Earthly? Are general purpose workflow systems winning any time soon, or are we still afraid of writing code? Any new systems out there that cover all the totally basic features like running locally, unlike Github Actions?

I think we could build something on top of nix that is as easy to use and powerful as earthly, but gets all the nice stuff from nix: reproducibility, caching, just use any command from any nix package, etc

That's mostly what we've built with Flox [1] (though I'm not exactly sure what you mean by run any command from any Nix package). It looks and feels like an amped up package manager, but uses Nix as kind of an infrastructure layer under the hood. Here's a typical workflow for an individual developer:

- cd into repo

- `flox activate` -> puts you into a subshell with your desired tools, environment variables, services, and runs any setup scripts you've defined

- You do your work

- `exit` -> you're back in your previous shell

Setting up and managing an environment is also super easy:

- cd into project directory

- `flox init` -> creates an "environment"

- `flox install python312` -> installing new packages is very simple

- `flox edit` -> add any environment variables, setup scripts, services in an editor

- `flox activate` -> get to work

The reason we call these "environments" instead of "developer environments" is that what we provide is a generalization of developer environments, so they're useful in more than just local development contexts. For example, you can use Flox to replace Homebrew by creating a "default" environment in your home directory [2]. You can also bundle an environment up into a container [3] to fit Flox into your existing deployment tools, or use Flox in CI [4].

All that stuff I described is free, but we have some enterprise features in development that won't be, and I think people are going to find those very appealing.

[1] https://flox.dev

[2] https://flox.dev/docs/tutorials/migrations/homebrew/

[3] https://flox.dev/docs/reference/command-reference/flox-conta...

[4] https://flox.dev/docs/tutorials/ci-cd/

Post reply on HN