Ask HN: How Long Is Your CI Process?

#1

Post by chuckgreenman » Sat, May 01, 2021, 2:10 PM UTC

I've seen a couple of projects using CI to build the project and run their test suite.

All of them have been interpreted languages like PHP, Python and Ruby. Their builds and tests took between 30-45 minutes. As far as project size and complexity, these were projects that were built and maintained by four person teams over the course of 3-5 years, so it's not like they were massive services with hundreds of developers.

I'm still kind of new, I worked at a couple internships and I've been working full time for a year so I might be totally wrong but I feel like these CI pipelines could be optimized to run faster.

Re: Ask HN: How Long Is Your CI Process?

#2

Post by jareds » Sat, May 01, 2021, 2:29 PM UTC

That seems reasonable to me if you require heavy integration test coverage. I work on several applications. The ones that are message driven and don't require a database have test suites that run in a couple minutes. The one that has a large database component takes about 30 minutes to run the tests. This is because we actually run the tests against a real database which requires migrations, data loading, etc.

Re: Ask HN: How Long Is Your CI Process?

#3

Post by tomduncalf » Sat, May 01, 2021, 2:32 PM UTC

Depends on the complexity of what you are doing. For web stuff with unit tests I’ve seen it run in a few minutes.

Our current CI takes an hour because it has to build quite a complex app on iOS and Android, this happens in parallel but the Azure build nodes we use are pretty slow. Ideally it would be faster but it’s not too huge an issue in practice, we have the lint/unit tests etc. run first so the build will fail early for any glaring errors.

Re: Ask HN: How Long Is Your CI Process?

#4

Post by detaro » Sat, May 01, 2021, 2:32 PM UTC

CI generally is a topic where there is usually lots of potential for optimization, but many things are not easily done with common tooling. Some large companies put serious R&D effort into improving CI with custom tooling.

What is applicable to the specific project depends, same as to what is worth which effort. To a degree, of course throwing more resources at the problem helps - faster build workers, parallelized tests, ... but isn't always easily implemented on a chosen platform and costs money of course.

In projects I worked on, it varied greatly. From just a few minutes to cases where the full process took 6 hours (which then was only done as a nightly job, and individual merge requests only ran a subset of steps). I really would want <15 mins as the normal case, but it's often difficult to get the ability to do so.

Re: Ask HN: How Long Is Your CI Process?

#5

Post by jimmyvalmer » Sat, May 01, 2021, 2:35 PM UTC

You get what you pay for? Cloud CI bottom-feeds unused downmarket capacity like a magazine-stand calling card. And Cloud CI is still "Cloud", i.e., a mass-market, one-size-fits-all solution like the Department of Motor Vehicles.

Re: Ask HN: How Long Is Your CI Process?

#6

Post by ilmiont » Sat, May 01, 2021, 2:44 PM UTC

I don't see how anyone can give you useful information without knowing more about the pipeline and the projects, and as everyone's pipelines/projects are going to work differently (I do web dev work, so pipelines are relatively simple, I can imagine that a game dev team creating Windows/Mac/Linux builds might have multi-hour pipelines though).

Anyway as the question is "How Long Is Your CI Process", here we go!

I have two main types of pipelines, both running on a self-hosted GitLab instance which runs on an 8th-gen i3 Intel NUC. No project is particularly massive.

1. PHP Projects. Run PHPStan + unit tests on each branch. Most projects take 1-5 mins. On master, run PHPStan + unit tests, build a Docker image, and use Helm to deploy to managed Kubernetes on DigitalOcean. This takes 5-10 mins.

2. React Projects, again not massively huge, but sizable. Biggest time is to run ESLint on every branch. About 5 mins (due to very poor caching which I keep meaning to fix). On master, run ESLint, create a Docker image, and deploy to managed Kubernetes. 5-10 mins.

There are opportunities to improve this by fixing/optimising caching. Overall I'm reasonably happy with the pipeline performance. I'm also sure that upgrading the hardware would make a big difference, probably more so than fixing the caching; an i3 isn't really ideal but this machine does well overall for my small team.

Re: Ask HN: How Long Is Your CI Process?

#7

Post by jart » Sat, May 01, 2021, 2:45 PM UTC

2 minutes and 7 seconds https://github.com/jart/cosmopolitan/runs/2482398460

on travis for a repository that builds 14,479 objects, 67 libraries, and 456 static executables, 284 of which are test executables which are run too. If I want to run all the test binaries on freebsd openbsd netbsd rhel7 rhel5 xnu win7 win10 too, then it takes 15 additional seconds. On a real PC, building and testing everything from scratch takes 34 seconds instead of two minutes.

Re: Ask HN: How Long Is Your CI Process?

#8

Post by djxfade » Sat, May 01, 2021, 2:48 PM UTC

We use CircleCI to deploy PHP apps. It takes about 1-2 minutes to fetch and build the JavaScript and SCSS assets, and 1 minute to install dependencies from Composer. And about 1 minute to rsync the build artifact to the server.

Re: Ask HN: How Long Is Your CI Process?

#9

Post by wyc » Sat, May 01, 2021, 2:48 PM UTC

We have a CI pipeline for a cross-platform Rust library, and it currently takes an hour across C, Android, iOS, Java, WASM, etc. and different combinations of cryptographic libraries. This is probably something we’ll tune over this or next quarter, such as by throwing some beefy hardware at it and parallelizing. We also seem to be hitting some GitHub actions limits in terms of storage.

https://github.com/spruceid/didkit/runs/2468746631

Re: Ask HN: How Long Is Your CI Process?

#10

Post by lacker » Sat, May 01, 2021, 2:52 PM UTC

Probably your CI could be faster. Just take a look at where the time is going and see if you can speed it up. Here’s some tips that might be useful: https://charity.wtf/2020/12/31/why-are-my-tests-so-slow-a-li...