For CI on large repos, you can do much better than this by using a persistent git cache. It takes a little finessing to destroy it if it's corrupt and avoid concurrent modifications, but it's extremely worth it.
A one-line change decreased our build times by 99%
251–256 of 256 posts
Re: A one-line change decreased our build times by 99%
#252Earlier quoted context omitted.
At my second job in the industry I worked on a Python project that had to be deployed in a kind of sandboxed production environment where we had no internet access. Deploys were painful, as any missing dependency had to be searched in our notebooks over 3G, then copied to an external storage, then plugged into a Windows machine, uploaded to the production server through SCP and then deployed manually over SSH. Someti…
I suppose no Docker containers were allowed in prod either?
Re: A one-line change decreased our build times by 99%
#253Earlier quoted context omitted.
I'd be interested to know how they came to realize what was missing. Did they read the Jenkins docs more thoroughly? Post on a mailing list? See something on StackOverflow? Hire a consultant?
I would bet on "the new recruit found it, 45 minutes in in the onboarding process".
It can see things that were there all along, but everyone who has been there has developed a blindness to.
Re: A one-line change decreased our build times by 99%
#254I sympathise a lot with this post! Git cloning can be shockingly slow. As a personal anecdote, clones of the Rust repository in CI used to be pretty slow, and on investigating we found out that one key problem was cloning the LLVM submodule (which Rust has a fork of). In the end we put in place a hack to download the tar.gz of our LLVM repo from github and just copy it in place of the submodule, rather than cloning i…
Out of curiosity, why not use the submodule. .shallow option in .gitmodules?
That said, I generally shy away from shallow clones and probably wouldn't use it here:
- it's a trap for people who ever want to work in that repo normally (we use the trick for more than just LLVM) - I believe shallow clones, over time (e.g. for contributors), are less nefficient than deep clones - I would expect shallow cloning to reuse fewer objects and benefit less from git's design. [0] describes a historic issue on this topic
[0] https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...
Re: A one-line change decreased our build times by 99%
#255Earlier quoted context omitted.
I think you live in a highly theoretical parallel universe. The one I occupy is the one where 'good teams' profile those things that take too long. Take yourself as an example: in spite of the wide availability of free certificates you are still hosting your domain without using a secure transport layer. Some would take that as incompetence. Others would assume you have more stuff on your plate rather than that you d…
And others would assume, that there are other philosophies out there regarding ssl everywhere. So the question is, who's POV has more validity and logical rigor attached to it. I actually can't see any side winning here on a purely logical level. Only on an ideological level. At least as long as we are talking about consuming public information. Am I in favor of the aggressiveness of OP in other posts? No. Am I using…
We take security very seriously. But we don't take anything too seriously.
Edit: by the way, persuade me that there's an upside and I'll turn it on.
Re: A one-line change decreased our build times by 99%
#256Earlier quoted context omitted.
I would bet on "the new recruit found it, 45 minutes in in the onboarding process".
I think everyone has done this, sometimes it really does take a second set of eyes
That tends to be the beginners during the onboarding weeks.