Almost all our Github projects use Docker. We don't use automatic deploys at the moment, but CI basically consists of building a Dockerfile, then running tests with "docker-compose run", then pushing the image to GCR.
We do have some projects that consist of multiple dockerfiles, but they're outliers. Everything else follows the same convention.
We currently use Semaphore, which has several downsides:
* Every project needs to be created and configured individually. We have many, many small projects. On Semaphore, setting up a project requires clicking through 4 or 5 screens to configure. I know Semaphore has an API (two incompatible ones, actually), but that means we have to write code for something that should be included in the box.
* To alleviate this burden, we have a single shell script, hosted with HTTPS and basic auth, that every project runs to build. It does things like log into GCR, set up secret build args, then do the build. Again, stuff that's identical across all projects. It's a bit silly to fetch this script every time rather than have it be shared across the whole organization.
* Semaphore's Docker layer caching (achieved with docker pull + docker build --cache-from) seems inconsistent. Some builds are fast, some don't get any caching.
* Semaphore is terrible at describing test failures. You'd think that a CI system would be great at doing things like show "4 failures/2 skipped/123 successful" on some kind of board. But instead you just get success/failure, and the entire build log. If the test is something like "go test", then you get 99% noise, and it's hard to zero in on what the failure was. I'd love to get the actual failing test in the Slack notification.
We've tried a few alternatives:
* Self-hosted Drone: Great, but too immature when we ran it, and development seems slow, with the organization behind apparently focused on a SaaS solution?
* Google Cloud Container Builder: Better, but requires each project to be "mirrored" within Google Cloud, which is a pain, and the mirroring is flaky. Also, no layer caching. And apparently not appropriate for running Docker Compose tests, and secrets management requires more tendrils into GCP, whereas we prefer things to be ore self-contained.
* CircleCI: Similar problems as Semaphore. Seemed a bit messy.
* Jenkins: Nobody likes Jenkins, apparently.
Based on the above, would Gitlab be a better fit?