`make install && make test` is about 15-20 seconds on a decent machine.
A full GHA CI run is ~30 minutes, but that involves 3 platforms, a (short) ci-fuzz run, and running the full test suite through valgrind.
Ask HN: How Long Is Your CI Process?
31–40 of 81 posts
Re: Ask HN: How Long Is Your CI Process?
#32The latency of your CI process doesn't matter[1].
What matters is the development process - local build & test should be fast.
Otherwise, with CI/CD, it's a continually-moving release train where changes get pushed, built, tested, and deployed non-stop and automatically without human intervention. Once you remove humans from the process, and you have guard rails (quality) built into the process, it doesn't matter if your release process for a single change takes 1min, 1hour, or 1day.
Even if it takes 1 day to release commit A, that's OK b/c 10min later commit B has been released (because it was pushed 10min after commit A).
I've seen pipelines that take 2 weeks to complete because they are deploying to regions all over the world - the first region deploys within an hour, and the next 2weeks are spent serially (and automatically) rolling out to the remaining regions at a measured pace.
If any deployment fails (either directly, or indirectly as measured by metrics) then it's rolled back and the pipeline is stopped until the issue is fixed.
[1] Yes, even for fixing production issues. You should have a fast rollback process for fixing bad pushes and not rely on pushing new patches.
Re: Ask HN: How Long Is Your CI Process?
#33A lot of the drag on CI for complex projects is tests, which are hard to argue against. Complexity : need for tests isn't linear -- once you hit some critical mass of complexity where one person can't know the whole application, the need for tests skyrockets.
I joined a company last year that's trying to solve this [1] by tracing tests so it can skip any whose dependencies (functions, environment variables, etc.) haven't changed. It's amazing what "what if we don't run tests we know will pass?" can do to a CI pipeline.
[1]: https://yourbase.io
Re: Ask HN: How Long Is Your CI Process?
#34Automated tests that my team runs vary from ~seconds to multiple days , depending on what's being tested. Some of the tests involve compiling a multi-billion line repo using over 30+ languages, and doing some analysis on the resulting code graph. So that takes awhile. 30-45 minutes just for a simple test suite, even if it's PHP, Python, and Ruby - that sounds long. But without any details on exactly what's being test…
Multi-billion? Wow. Google? Microsoft?
Re: Ask HN: How Long Is Your CI Process?
#35I've setup multiple CI systems and it really depends on what you need to test. A long time ago I built a CI for a team that ran end to end integration tests and collected code coverage from each service running. This took between 4 minutes and 10 minutes to run for our 3 to 6 services. Another job I worked at I setup a git repo + CI for a team of about 15ish people. In the beginning we had no CI, then I containerized everything and the CI took at long time (~20 minutes). Then, I switched to a build/test system that was more in tune with what we needed and I ultimately (through some hacks) got the entire CI time for ~20ish microservices down to The main blockers I've seen to CI performance is:
1. Caching: Most build systems are intended to run on a developers laptop and do not cache things correctly. Because of this most CIs completely chuck all of your state out of the window. The only CI that I've found that lets you work around this is Gitlab CI (this is my secret for getting 2. What you do in CI: If you want to run end-to-end integration tests, it's going to be slow. Any time you're accessing a disk, accessing the network, anything that doesn't touch memory, it's slow. Make sure your unit tests are written to use Mocks/Fakes/Stubs instead of real implementations of DBs like sqlite or postgres or something.
3. The usage pattern: If you don't have developers utilizing your CI machines 100% of the time you are "wasting" those resources. People will often say "lets autoscale these nodes" and, when you do, you'll notice they scale down to 1 node when everyone is asleep, everyone starts work and pushes code, then the CI grinds to a halt. You can make a very inefficient CI just by having the correct number of runners available at the correct time.
Another thing to consider: anything you can make asynchronous doesn't need to be fast. If you setup a bot to automatically rebase and merge your code after code review then you don't really need to think about how fast the CI is.
Re: Ask HN: How Long Is Your CI Process?
#36Think of build and test times as being determined by what people are willing to put up with. If people only start getting annoyed at the runtime once it's past 45 minutes, it'll probably take about 45 minutes. People will keep adding things that slow it down, such as new dependencies.
Re: Ask HN: How Long Is Your CI Process?
#37Depending on the repo, between 3-4 minutes and 3 hours. Fast is some quick checks that the repo builds, slow is an FPGA synthesis and place/route. None of them are particularly large in term of LOC. Probably the slowest part of the process on the non-FPGA builds is installing python packages.
Re: Ask HN: How Long Is Your CI Process?
#38Does anyone here have any tips/tricks when it comes to iOS builds?
Currently experimenting with Travis-CI, but man it sure does take awhile at 45-60 minutes roughly in my personal case. Have heard a dedicated Mac of some kind to leave at the office may help. Overall, I am all ears to any advice.
Re: Ask HN: How Long Is Your CI Process?
#39Does anyone here have any tips/tricks when it comes to iOS builds? Currently experimenting with Travis-CI, but man it sure does take awhile at 45-60 minutes roughly in my personal case. Have heard a dedicated Mac of some kind to leave at the office may help. Overall, I am all ears to any advice.
I used Expo, which has a pretty good DX on iOS.
Re: Ask HN: How Long Is Your CI Process?
#40Deoends on what parts changed. 2 hours of tests for simple changes, 8 hours for the complex changed everything stuff. We have broken up the system so the first is far more common.
Note that half of the tests on the fast build are regression that can't possibly fail based on my changes... we run them anyway because about once a month something has a completely unexpected interaction and so a test fails that the developer didn't think to test.