A one-line change decreased our build times by 99% • Hacker News Board

Re: A one-line change decreased our build times by 99%

#121

Post by jeromenerf » Mon, Oct 26, 2020, 8:20 AM UTC

dusted wrote: post #117

This is the most I've ever gotten out of pinterest, other than this, it's just the "wrong site that google turns up, that I can't use because it wants me to create an account just to watch the image I searched for"

This is one situation where a duckduckgo search is objectively of a better signal/noise ratio.

Re: A one-line change decreased our build times by 99%

#122

Post by jeromenerf » Mon, Oct 26, 2020, 8:23 AM UTC

projektfu wrote: post #27

JOnAgain wrote: post #2

I think it takes some real humility to post this. No doubt someone will follow up with an “of course...” or “if you don’t understand the tech you use...” comment. But thank you for this. It takes a bit of courage to point out you’ve been doing something grotesquely inefficient for years and years.

I'd be interested to know how they came to realize what was missing. Did they read the Jenkins docs more thoroughly? Post on a mailing list? See something on StackOverflow? Hire a consultant?

I would bet on "the new recruit found it, 45 minutes in in the onboarding process".

Re: A one-line change decreased our build times by 99%

#123

Post by csunbird » Mon, Oct 26, 2020, 8:24 AM UTC

dusted wrote: post #117

This is the most I've ever gotten out of pinterest, other than this, it's just the "wrong site that google turns up, that I can't use because it wants me to create an account just to watch the image I searched for"

I am not sure why google does not penalize this behavior in their search ranking.

Re: A one-line change decreased our build times by 99%

#124

Post by maccard » Mon, Oct 26, 2020, 8:25 AM UTC

marta_morena_28 wrote: post #76

pushrax wrote: post #37

Earlier quoted context omitted.

I would expect that internally someone profiled the build (i.e. looked at timestamps) and then either profiled git, or just looked at the logs and did some guessing/research. This didn't seem like it would be complicated to find once you realize the time is spent in git. Also, this probably has been an exponentially increasing problem, and wasn't really a priority to solve until relatively recently. I would bet there…

It really doesn't sound complicated to find, unless you just have a handsoff approach to building things and just don't care as long as "something" comes out on the other end. What makes me wonder however is this: 40 min made them look into this? I mean 40 min is crazy long. What builds this long? Chrome, Windows, Linux Kernel on a single core? This should have been raising red flags much earlier. The only explanatio…

I work in games, a clean sync of our project takes well over an hour (and if you're at home it takes multiple hours), and compiling takes a large amount of time. We use lots of tricks - "unity" builds with out build system detecting modified files and compiling them standalone as an example. On my last workstation (2x intel xeon gold) it took about 30 minutes to compile

Re: A one-line change decreased our build times by 99%

#125

Post by nathan_f77 » Mon, Oct 26, 2020, 8:25 AM UTC

Is there any way to do this for GitLab CI [1]? I'm using GIT_DEPTH=1, but I'm not sure how to set refspecs. It's not too important right now since it only takes about 11 seconds to clone the git repo, but maybe it's a quick win as well.

[1] https://docs.gitlab.com/ee/ci/large_repositories/

Re: A one-line change decreased our build times by 99%

#126

Post by yoz-y » Mon, Oct 26, 2020, 8:28 AM UTC

Because of strife with 99% claim. If the pull time took 39.9min (and thus build took 0.1min = 6sec) then a 99% decrease in pull time would result in 99% decrease of total time and you would get 30sec total time in the end. (Rounding to 0 decimal places).

Not that any of this is important for the article to be interesting. In a previous job we had to fight long pull times and we quickly created a git repo for CI that would sit on a machine next to the CI server and would periodically pull from GitHub to avoid the CI to do pulls over Internet.

Re: A one-line change decreased our build times by 99%

#127

Post by sercankd » Mon, Oct 26, 2020, 8:33 AM UTC

dusted wrote: post #117

This is the most I've ever gotten out of pinterest, other than this, it's just the "wrong site that google turns up, that I can't use because it wants me to create an account just to watch the image I searched for"

Yeah i always believed it was some kind of lone evil ai that lives through search results.

Re: A one-line change decreased our build times by 99%

#128

Post by sambe » Mon, Oct 26, 2020, 8:34 AM UTC

Ok, I'll ask the obvious question: why did setting the branches option to master not already do this?

EDIT

https://www.jenkins.io/doc/pipeline/steps/workflow-scm-step/ makes it sounds like the branches option specifies which branches to monitor for changes, after which all branches are fetched. This still seems like a counter-intuitive design that doesn't fit the most common cases.

Re: A one-line change decreased our build times by 99%

#129

Post by nathan_f77 » Mon, Oct 26, 2020, 8:35 AM UTC

andrelaszlo wrote: post #118

We recently reduced our build times by 5-10% or so by changing the default bcrypt iteration count (for tests). It also felt silly once we found it.

Thanks so much for this tip! I just made this change and some of my tests are now much faster. Here's the result for one of the affected tests (averaged across 5 runs):

Before: 1.39 seconds

After: 0.62 seconds

I have this default line in config/initializers/devise.rb:

    config.stretches = Rails.env.test? ? 1 : 11

So hashing user passwords was already very fast. But I'm also manually calling BCrypt in some other places, so these calls are now much faster as well.

Re: A one-line change decreased our build times by 99%

#130

Post by saagarjha » Mon, Oct 26, 2020, 8:36 AM UTC

dusted wrote: post #117

This is the most I've ever gotten out of pinterest, other than this, it's just the "wrong site that google turns up, that I can't use because it wants me to create an account just to watch the image I searched for"

Can we not do the thing where we pick an organization from an article and then bring up the most generic complaint you can about it in a way that is entirely irrelevant to the post? We get it, you don't like Pinterest showing up in search results, nobody does. But this has absolutely nothing to do with the article other than it being pattern matching on the word "Pinterest", which is about the least informative comment you can make aside from outright trolling or spam. There are threads that come up from time to time where such comments would be appropriate, if not particularly substantive.