Great idea. One question - what pain point is this solving? As a GitLab, GitLab Pipeline and Docker user I don't find running pipelines that slow to run or initiate
Why do you say great idea when you didn't even get the pain point it is solving?
Show HN: Test your Gitlab CI Pipelines changes locally using Docker
21–30 of 41 posts
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#22IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker.
If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you easily jump between CI providers since a majority of your logic is in a generic shell script.
For example in my CI specific files I typically only call `./run ci:install-deps && ./run ci:test`. Then there's a tiny bit of boiler plate in each CI provider's yml file to handle specific things for that provider.
If your app is running in Docker most of the heavy duty dependencies are all contained there. The dependencies I install directly in CI end up being things like installing shellcheck and helper scripts for CI (like tiny scripts that let you wait until a process is ready). Having WSL 2 is nice here because these are very small tools that I want locally installed anyways but even if you use macOS you could install these dependencies using brew instead.
A working example of this is here: https://github.com/nickjj/docker-flask-example
Check out the run script in the root of the project for "ci:" functions and the GH Actions yml file. This same strategy applies to GitLab or any other CI provider too.
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#23Excellent idea!
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#24IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you…
[deleted]
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#25IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you…
We do this but with makefiles to contain whatever CI logic which also makes it easy to switch.
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#26IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you…
I also like this approach, although I am using Nix instead of Docker for a slightly more lightweight way of managing dependencies of different stages.
If you also reduce the assumptions made by your CI scripts, as in the scripts themselves will authenticate to and set up any needed connectivity / port forwarding, or fetch secrets instead of being passed secrets from the CI system, then you end up with some really nice portable scripts that can be run from anywhere.
The only things that my CI scripts depend on is an authentication token to Hashicorp Vault, an internet connection and Nix to be installed on the machine.
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#27IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you…
Same, my CI has looked like this for years now:
./script/prepare-env
./script/test
./script/build
./script/deploy
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#28IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker. If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you…
urgh no please don't wrap simple commands like docker build in shell called from makefile it's so f'ing complex to follow when it falls apart.
If you're going to do script-in-script like this at least go to a proper cross-platform language like python, powershell or js anything but make+bash it's just the worst of all worlds.
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#29any chance of running this without installing yarn?
Re: Show HN: Test your Gitlab CI Pipelines changes locally using Docker
#30Why not just use GitLab's runner[0] directly for this? gitlab-runner exec docker [0]: https://docs.gitlab.com/runner/install/
Yeah I thought about that, but gitlab-runner doesn't allow to "share" cache or artifacts between jobs so it's not a good fit to test multiple-jobs workflows.
Sure it does. Jobs have a shared cache (although this can be slow) and artifacts are shared to downstream jobs.
Maybe this is a tiered feature?