Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A one-line change decreased our build times by 99%

medium.com

Post reply on HN
Print view View original

11–20 of 256 posts

Re: A one-line change decreased our build times by 99%

#11

Post by TylerE »

freedomben wrote: post #10
TylerE wrote: post #3

@Dang, can we get an edit? This did NOT slash build times 99%, but rather time to do a git pull.

From the article: > We found that setting the refspec option during git fetch reduced our build times by 99%. Seems pretty clear to me that build times were reduced by 99% as a result of cutting the git fetch times significantly (but exacyt number is not give). The headline looks correct to me.

FTA: "This simple one line change reduced our clone times by 99% and significantly reduced our build times as a result"

Unless their build is 100% git pull time, this did not reduce build time by 99%.

Re: A one-line change decreased our build times by 99%

#12

Post by mandeepj »

Looks like Pinterest’s team is confused about Git Branches. These are not real full copy versions of the main branch like in SVN or TFS. A branch in Git world is simply a pointer to a specific commit in the code push history.

Having said that, happy to be proven wrong, and learn about it.

Re: A one-line change decreased our build times by 99%

#13

Post by Aeolun »

hashkb wrote: post #5
JOnAgain wrote: post #2

I think it takes some real humility to post this. No doubt someone will follow up with an “of course...” or “if you don’t understand the tech you use...” comment. But thank you for this. It takes a bit of courage to point out you’ve been doing something grotesquely inefficient for years and years.

They are a publicly traded company. They have a team dedicated to engineering support. A better article would include a management and hiring postmortem. It's shocking, really. Humility is nice, but competency is also nice.

I don’t know, I consider myself fairly competent but I’d never even considered that. It’s just not so relevant until your repo is multiple gigabytes big.

Still, I’ll see if it works for our pipelines, and we can get our clone from 20s to 1s

Re: A one-line change decreased our build times by 99%

#14

Post by [deleted] »

freedomben wrote: post #10
TylerE wrote: post #3

@Dang, can we get an edit? This did NOT slash build times 99%, but rather time to do a git pull.

From the article: > We found that setting the refspec option during git fetch reduced our build times by 99%. Seems pretty clear to me that build times were reduced by 99% as a result of cutting the git fetch times significantly (but exacyt number is not give). The headline looks correct to me.

[deleted]

Re: A one-line change decreased our build times by 99%

#15

Post by Aeolun »

TylerE wrote: post #11
freedomben wrote: post #10

Earlier quoted context omitted.

From the article: > We found that setting the refspec option during git fetch reduced our build times by 99%. Seems pretty clear to me that build times were reduced by 99% as a result of cutting the git fetch times significantly (but exacyt number is not give). The headline looks correct to me.

FTA: "This simple one line change reduced our clone times by 99% and significantly reduced our build times as a result" Unless their build is 100% git pull time, this did not reduce build time by 99%.

To be fair, if their pull took 40 minutes, that’s a very real option :)

Re: A one-line change decreased our build times by 99%

#17

Post by hashkb »

Aeolun wrote: post #13
hashkb wrote: post #5

Earlier quoted context omitted.

They are a publicly traded company. They have a team dedicated to engineering support. A better article would include a management and hiring postmortem. It's shocking, really. Humility is nice, but competency is also nice.

I don’t know, I consider myself fairly competent but I’d never even considered that. It’s just not so relevant until your repo is multiple gigabytes big. Still, I’ll see if it works for our pipelines, and we can get our clone from 20s to 1s

Good teams profile everything. This team's only goal is to support other engineers. Build time is a huge issue for every ops team. Missing this for so long is wasted money that's easy to calculate. We can be nice to people while still having high standards. It's a missed opportunity for a deeper postmortem, and it's bland content at best.

Re: A one-line change decreased our build times by 99%

#18

Post by argc »

hashkb wrote: post #5
JOnAgain wrote: post #2

I think it takes some real humility to post this. No doubt someone will follow up with an “of course...” or “if you don’t understand the tech you use...” comment. But thank you for this. It takes a bit of courage to point out you’ve been doing something grotesquely inefficient for years and years.

They are a publicly traded company. They have a team dedicated to engineering support. A better article would include a management and hiring postmortem. It's shocking, really. Humility is nice, but competency is also nice.

This is neither incompetence nor surprising. Maybe you’ve only worked at large companies who have had time to optimize things for years (and even then, I see grotesque software decisions at my large company quite often). Try accepting that software is often written poorly optimized on the first pass, for good reason, and learn to celebrate the wins without needing to shame someone.

Re: A one-line change decreased our build times by 99%

#19

Post by ekimekim »

mandeepj wrote: post #12

Looks like Pinterest’s team is confused about Git Branches. These are not real full copy versions of the main branch like in SVN or TFS. A branch in Git world is simply a pointer to a specific commit in the code push history. Having said that, happy to be proven wrong, and learn about it.

IIUC the issue here is the depth option - they're telling it to only fetch the last 50 commits, but they were fetching the last 50 commits from EACH branch. In other words, they were fetching all commits that are within 50 commits of any branch head. By restricting the branches, they drastically reduce the set of commits to fetch.

Re: A one-line change decreased our build times by 99%

#20

Post by tracer4201 »

I truly appreciate articles like this — it’s warming to see other companies running into the kinds of issues I’ve ran into or had to deal with, and more so that their culture openly discusses and shares these learnings with the broader community.

The most effective organizations I’ve worked at built mechanisms and processes to disseminate these kinds of learnings and have regular brown bags on how a particular problem was solved or how others can apply their lessons.

Keep it up Pinterest engineering folks.

Post reply on HN