Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Act: Run your GitHub Actions locally

github.com

Post reply on HN
Print view View original

81–90 of 161 posts

Re: Act: Run your GitHub Actions locally

#81

Post by stabbles »

FooBarWidget wrote: post #73
Arnavion wrote: post #38

Earlier quoted context omitted.

>With actions you can run multiple tasks in parallel, If you mean "I want to run one build with the foo feature and one build with the bar feature. Actions lets me run those in parallel", then that is the "strategy" part of the workflow, not the "tasks" part. My comment was about the latter. ("and make your GH Actions run `make` in a script task.") If you mean "I want to run two steps of a job in parallel and then ru…

> Running things in parallel is literally a single `&` character. Yes now try waiting for all parallel tasks, and error out if at least one errors. And try separating their output so that they don't get interleaved into a big mess. And try by default hiding the outputs of commands that succeeded, only showing those that failed, except when the user explicitly asks for it. Your "simple" shell script now suddenly isn't…

Using GNU make you can get very nice output sync:

    all: variant-1 variant-2
        @echo done with all
    
    variant-%:
        @echo starting $@
        @sleep 1
        @echo done $@
Use `make -j2 --output-sync=target`.

Re: Act: Run your GitHub Actions locally

#82

Post by 2253539242 »

jaxn wrote: post #63
ilammy wrote: post #49

Earlier quoted context omitted.

This is a lesson that I've learned after going all-out on actions once. Now my makefiles in addition to the usual "make" and "make test" also support "make prerequisites" to prepare a build environment by installing everything necessary and "make ci" to run everything that CI should check. With actual implementation being scripts placed under "scripts/ci". The scripts do provide some goodies when they are run by GitH…

What about caching to reduce ci time? GH setup scripts cache dependencies in a way that would seem hard to replicate in a make file.

om.

Re: Act: Run your GitHub Actions locally

#83

Post by cormacrelf »

FooBarWidget wrote: post #73
Arnavion wrote: post #38

Earlier quoted context omitted.

>With actions you can run multiple tasks in parallel, If you mean "I want to run one build with the foo feature and one build with the bar feature. Actions lets me run those in parallel", then that is the "strategy" part of the workflow, not the "tasks" part. My comment was about the latter. ("and make your GH Actions run `make` in a script task.") If you mean "I want to run two steps of a job in parallel and then ru…

> Running things in parallel is literally a single `&` character. Yes now try waiting for all parallel tasks, and error out if at least one errors. And try separating their output so that they don't get interleaved into a big mess. And try by default hiding the outputs of commands that succeeded, only showing those that failed, except when the user explicitly asks for it. Your "simple" shell script now suddenly isn't…

That’s what GNU Parallel is for. Or Pueue, which gives you a GH-Actions-level feature set but it’s less likely to be installed on any particular machine. Pretty sure you could fetch a pueue binary at the start of an actions script and do everything that way.

(These can’t project their tasks over multiple GH Actions runners, eg for multiple OSes. For that you will need to use the YAML. Good compilers will already do work in parallel and max out however many cores they are given, so multi machine is the main use case. Unfortunate.)

Re: Act: Run your GitHub Actions locally

#84

Post by MuffinFlavored »

FooBarWidget wrote: post #71
Arnavion wrote: post #33

Rather than replacing your Makefile with GH Actions, replace your GH Actions with a Makefile, and make your GH Actions run `make` in a script task. Do you really need that GH Action for pulling Docker images / installing $language_compiler / creating cloud resources ? A `docker` / `curl` / `sudo apt-get install` invocation in a Makefile / script needs to be written once and is the same in CI as on your dev machines.…

A Makefile really sucks at displaying outputs/logs of commands, especially when there are lots of commands and when they run concurrently. It also really sucks at communicating what the overall progress is: how many jobs have finished, how many left, how much time has elapsed. Heck make can make all this much better by just prepending each output line with some colored prefix and timestamp. But make hasn't changed in…

https://www.stevefenton.co.uk/blog/2022/06/run-a-bash-script...

Re: Act: Run your GitHub Actions locally

#85

Post by naikrovek »

Arnavion wrote: post #33

Rather than replacing your Makefile with GH Actions, replace your GH Actions with a Makefile, and make your GH Actions run `make` in a script task. Do you really need that GH Action for pulling Docker images / installing $language_compiler / creating cloud resources ? A `docker` / `curl` / `sudo apt-get install` invocation in a Makefile / script needs to be written once and is the same in CI as on your dev machines.…

forget make, just put it all in a docker container and let the docker container be your CI. that's what the tool linked in the post does, and if you have a docker container you can run it unmodified in just about any CI system.

Re: Act: Run your GitHub Actions locally

#86

Post by oweiler »

plonk wrote: post #16
pocketarc wrote: post #6

I’ve used this a bunch; I actually don’t know how else you’d write and test a GitHub action. Do people just push them and hope they work and push more commits with “fix” as the message until it works?

I add a workflow like this: - write the main job in a Python/PowerShell/Bash script that runs locally, - write a workflow that sets up the environment (AWS login), installs dependencies (using GitHub's facilities for that) and calls the script (often passing values using GitHub Secrets), - push the workflow with a "push" trigger on a feature branch (for fast testing), - push 30 fixup commits to fix all the small synt…

> I add a workflow like this:

> - write the main job in a Python/PowerShell/Bash script that runs locally,

> - write a workflow that sets up the environment (AWS login), installs dependencies (using GitHub's facilities for that) and calls the script (often passing values using GitHub Secrets),

> - push the workflow with a "push" trigger on a feature branch (for fast testing),

> - push 30 fixup commits to fix all the small syntax errors, logic mistakes, and GHA idiosyncrasies I hadn't thought of,

> - remove the push trigger,

> - do an interactive rebase to squash all the fixups,

> - force-push and merge.

> It works pretty well and can be easily used locally or in other CI/CD systems.

It saddens me that such madness is considered as "works pretty well"

Re: Act: Run your GitHub Actions locally

#87

Post by naikrovek »

goodpoint wrote: post #25
benrutter wrote: post #12

Finally! I've been wanting something like this for ages- generally speaking I don't consider myself an idiot, but I'm forced to pull that into questioning every time I test and debug ci/cd actions with an endless stream of pull request modifications

It's ridiculous how we still tolerate the CI vendor lock-in.

name one (1) CI system, open or closed, which shares enough with another CI system, open or closed, that there is no pain when changing from one to another.

they are all at least semi proprietary.

Re: Act: Run your GitHub Actions locally

#88

Post by nickjj »

Arnavion wrote: post #33

Rather than replacing your Makefile with GH Actions, replace your GH Actions with a Makefile, and make your GH Actions run `make` in a script task. Do you really need that GH Action for pulling Docker images / installing $language_compiler / creating cloud resources ? A `docker` / `curl` / `sudo apt-get install` invocation in a Makefile / script needs to be written once and is the same in CI as on your dev machines.…

This is what I do except I use a shell script instead of a Makefile.

A working example of this is at: https://github.com/nickjj/docker-flask-example/blob/912388f3...

Those ./run ci:XXX commands are in: https://github.com/nickjj/docker-flask-example/blob/912388f3...

I like it because if CI ever happens to be down I can still run that shell script locally.

Re: Act: Run your GitHub Actions locally

#89

Post by naikrovek »

letmeinhere wrote: post #62

Ability to run {X} locally is _the_ problem with building atop paid services, not the dreaded vendor lock-in.

do your builds and tests in a container.

run the container on your dev machine or your dreaded paid service. same every time no matter what.

Post reply on HN