That's projected growth for two of their projects. Sounds like they have something brewing... Still amazed that breaking it up would do more harm than good when the code isn't even written yet...
Facebook hit git performance issue on large repository
11–20 of 217 posts
Re: Facebook hit git performance issue on large repository
#12While I'd be interested in seeing this issue further unfold, just the prospect of a 1.3M-file repo gives me the creeps. I'm not sure what the exact situation at Facebook is with this repository, but I'm positive that if they had to start with a clean slate, this repo would easily find itself broken up into at least a dozen different repos. Not to mention the fact that if _git_ has issues dealing with 1.3M files, I wo…
Re: Facebook hit git performance issue on large repository
#13While I'd be interested in seeing this issue further unfold, just the prospect of a 1.3M-file repo gives me the creeps. I'm not sure what the exact situation at Facebook is with this repository, but I'm positive that if they had to start with a clean slate, this repo would easily find itself broken up into at least a dozen different repos. Not to mention the fact that if _git_ has issues dealing with 1.3M files, I wo…
Re: Facebook hit git performance issue on large repository
#14This was actually pretty fascinating to me. On one hand, I am astonished at how long it takes to perform seemingly trivial git operations on repositories at this scale. On the other hand, I'm utterly mystified that a company like Facebook has such monolithic repositories. Even back when I was using SVN a lot, I relied on externals and such to break up large projects into their smaller service-level components. I'd be…
Re: Facebook hit git performance issue on large repository
#15Re: Facebook hit git performance issue on large repository
#16While I'd be interested in seeing this issue further unfold, just the prospect of a 1.3M-file repo gives me the creeps. I'm not sure what the exact situation at Facebook is with this repository, but I'm positive that if they had to start with a clean slate, this repo would easily find itself broken up into at least a dozen different repos. Not to mention the fact that if _git_ has issues dealing with 1.3M files, I wo…
If you read further down the thread they say that that's already had the non-interlinked files split out. What they've got left isn't easily broken up.
Our source repo at work (a C++ compiler with full commit history going back to the early 90s...) is smaller and more componentized!
Re: Facebook hit git performance issue on large repository
#17While I'd be interested in seeing this issue further unfold, just the prospect of a 1.3M-file repo gives me the creeps. I'm not sure what the exact situation at Facebook is with this repository, but I'm positive that if they had to start with a clean slate, this repo would easily find itself broken up into at least a dozen different repos. Not to mention the fact that if _git_ has issues dealing with 1.3M files, I wo…
Pretty sure Perforce performs fine with that.
It's even worse that just disk space and performance issues.
I can totally imagine a huge, busy repository where by the time you've pulled and rebased/merged your stuff, the repo has already been committed to again, invalidating your fast-forward commit and forcing you to pull again and again before you have any chance of pushing back your changes.
This is an inherent problem with DVCS that just can't be solved (trivially) when working on huge repositories that span millions of files and involve thousands of developers.
Re: Facebook hit git performance issue on large repository
#18While I'd be interested in seeing this issue further unfold, just the prospect of a 1.3M-file repo gives me the creeps. I'm not sure what the exact situation at Facebook is with this repository, but I'm positive that if they had to start with a clean slate, this repo would easily find itself broken up into at least a dozen different repos. Not to mention the fact that if _git_ has issues dealing with 1.3M files, I wo…
Re: Facebook hit git performance issue on large repository
#19They keep every project in a single repo, mystery solved.
Edit:
> We already have some of the easily separable projects in separate repositories, like HPHP.
Yeah, because it makes no sense, it's C++. They probably use for everything PHP i assume then. Is there no good build management tool for it?
Re: Facebook hit git performance issue on large repository
#20MSFT with Windows codebase that runs out of several labs. Crazy branching and merging infrastructure. They use source-depot, originally a clone of perforce.
Google with all their source code in one Perforce repo.
Facebook will be on perforce before we know it.
The solution is an internal Github, not one giant project.