Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook hit git performance issue on large repository

thread.gmane.org

Post reply on HN
Print view View original

51–60 of 217 posts

Re: Facebook hit git performance issue on large repository

#51

Post by lbrandy »

Wow. I was expecting an interesting discussion. I was disappointed. Apparently the consensus on hacker news is that there exists a repository size N above which the benefits of splitting the repo _always_ outweigh the negatives. And, if that wasn't absurd enough, we've decided that git can already handle N and the repository in question is clearly above N. And I guess all along we'll ignore the many massive organizations who cannot and will not use git for precisely the same issue.

So instead of (potentially very enlightening conversation) identifying and talking about limitations and possible solutions in git, we've decided that anyone who can't use git because of its perf issues is "doing it wrong".

Re: Facebook hit git performance issue on large repository

#53

Post by xxpor »

wmf wrote: post #14
courtewing wrote: post #3

This was actually pretty fascinating to me. On one hand, I am astonished at how long it takes to perform seemingly trivial git operations on repositories at this scale. On the other hand, I'm utterly mystified that a company like Facebook has such monolithic repositories. Even back when I was using SVN a lot, I relied on externals and such to break up large projects into their smaller service-level components. I'd be…

Given that Facebook is compiled into a single 1 GB executable, a git repo with 1.3 M files doesn't really surprise me.

Oh God, it's like Amazon was 10 years ago.

Re: Facebook hit git performance issue on large repository

#54

Post by jamespo »

lbrandy wrote: post #51

Wow. I was expecting an interesting discussion. I was disappointed. Apparently the consensus on hacker news is that there exists a repository size N above which the benefits of splitting the repo _always_ outweigh the negatives. And, if that wasn't absurd enough, we've decided that git can already handle N and the repository in question is clearly above N. And I guess all along we'll ignore the many massive organizat…

I would have liked this comment better if it came up with some solutions itself, although maybe it's not easy to solve?

Re: Facebook hit git performance issue on large repository

#55

Post by lincolnq »

lbrandy wrote: post #51

Wow. I was expecting an interesting discussion. I was disappointed. Apparently the consensus on hacker news is that there exists a repository size N above which the benefits of splitting the repo _always_ outweigh the negatives. And, if that wasn't absurd enough, we've decided that git can already handle N and the repository in question is clearly above N. And I guess all along we'll ignore the many massive organizat…

I had the same reaction as you.

Stat'ing a million files is going to take a long time. Perforce doesn't have this problem because you explicitly check out files (p4 edit). (Perforce marks the whole tree read-only, as a reminder to edit the file before you save.)

It seems like large-repo git could implement the same feature. You would just disable (or warn) for operations which require stat'ing the whole tree.

Then the question is how to make the rest of the operations perform well -- git add taking 5-10 seconds seems indicative of an interesting problem, doesn't it?

Re: Facebook hit git performance issue on large repository

#57

Post by brown9-2 »

InclinedPlane wrote: post #18
yuvadam wrote: post #10

While I'd be interested in seeing this issue further unfold, just the prospect of a 1.3M-file repo gives me the creeps. I'm not sure what the exact situation at Facebook is with this repository, but I'm positive that if they had to start with a clean slate, this repo would easily find itself broken up into at least a dozen different repos. Not to mention the fact that if _git_ has issues dealing with 1.3M files, I wo…

A lot of big companies have repos 10 or 100 times that size. With tens of millions of files, sometimes up to 100 gigs or more of data under source control.

True, but don't most places organize one git repo per project, rather than one for the entirety of the company's source code?

Re: Facebook hit git performance issue on large repository

#58

Post by sek »

lbrandy wrote: post #51

Wow. I was expecting an interesting discussion. I was disappointed. Apparently the consensus on hacker news is that there exists a repository size N above which the benefits of splitting the repo _always_ outweigh the negatives. And, if that wasn't absurd enough, we've decided that git can already handle N and the repository in question is clearly above N. And I guess all along we'll ignore the many massive organizat…

You have a point.

It is just surprising when git was designed for the Linux kernel and we all here have a Github mindset.

Re: Facebook hit git performance issue on large repository

#59

Post by nbm »

marginalboy wrote: post #45
losvedir wrote: post #28

Huh, fascinating. git was initially created for the Linux kernel development, and I haven't heard of any issues there. Offhand I would have said, as a codebase, the Linux kernel would be larger and more complex than facebook, but I don't have a great sense of everything involved in both cases. So what's the story here: kernel developers put up with longer git times, the kernel is better organized, the scope of facebo…

It isn't surprising if Facebook has a large, highly coupled code base. Given their reputation for tight timelines and maverick-advocacy, I'm continually surprised the thing works at all.

I wouldn't say that a large repository implies that the code is highly-coupled. There are advantages for keeping certain code together in a single repo. Being able to easily discover users of functions of a library, being able to "transactionally" commit an update to a library (or set of libraries) and the code that uses it, being able to do code review over changes of code in various places, being able to discover if someone else has solved this problem before, and so forth. If you only have your project and its libraries checked out, you don't serendipitously discover things in other projects.

As mentioned in this talk on how Facebook worked on visualizing interdependence between modules to drive down coupling at https://www.facebook.com/note.php?note_id=10150187460703920 , there are at least 10k modules with clear dependency information in a front-end repo, and the situation probably is a lot better now that they have that information-dense representation to work from (I don't work on the PHP/www side of things, I spend most of my time in back-end and operations repos).

Re: Facebook hit git performance issue on large repository

#60

Post by [deleted] »

sek wrote: post #29
dblock wrote: post #20

Others have tried and keep throwing more and more smart people at the problem they just shouldn't have. MSFT with Windows codebase that runs out of several labs. Crazy branching and merging infrastructure. They use source-depot, originally a clone of perforce. Google with all their source code in one Perforce repo. Facebook will be on perforce before we know it. The solution is an internal Github, not one giant proje…

Google has everything in one Perforce repo? You mean the search engine, do you? I agree btw, the Github mindset is the best one. Create for every project a new repo and connect them with build tools. But why not hire 100 SOA-Consultants, they have enough money now.

[deleted]
Post reply on HN