For some context of why Facebook choose Hg over Git, here's the mailing list thread where Facebook initially reached out to the Git developers: http://thread.gmane.org/gmane.comp.version-control.git/18977...
[deleted]
Scaling Mercurial at Facebook
161–170 of 245 posts
Re: Scaling Mercurial at Facebook
#162For some context of why Facebook choose Hg over Git, here's the mailing list thread where Facebook initially reached out to the Git developers: http://thread.gmane.org/gmane.comp.version-control.git/18977...
That link appears dead. Here is another: http://git.661346.n2.nabble.com/Git-performance-results-on-a...
Re: Scaling Mercurial at Facebook
#163I wrote a comment about the scaling of repositories (and specifically Facebook's issues) a few days ago that was wiped out by the HN crash, but I've managed to recover it from HNSearch: Facebook's problem is that they were trying to scale with Git improperly. With conventional CVS[sic] systems like Perforce, you can scale a single repo nearly as large as any company will need. Emphasis on that nearly. At a certain po…
I get this impression every time I see a "look what neat scalability thing we did" post from Facebook Engineering. It's great that they're able to achieve such technical feats, but they refuse to acknowledge that maybe they're using the technology wrong. I'm reminded of the time they hacked the Dalvik VM on Android because apparently they had too many method names for the Dalvik VM to handle. http://jaxenter.com/face…
I can't agree. Instead of taking everyone's advice of either switching to perforce (less user-friendly/frustrating repo) or splitting their repo into parts, facebook built a solution where they don't have to make either compromise.
Just because their solution is complex doesn't mean they're using the technology wrong. They set out their requirements and they met all of them. All their engineers are more efficient now and all this complexity happens behind the scenes.
Re: Scaling Mercurial at Facebook
#164I wrote a comment about the scaling of repositories (and specifically Facebook's issues) a few days ago that was wiped out by the HN crash, but I've managed to recover it from HNSearch: Facebook's problem is that they were trying to scale with Git improperly. With conventional CVS[sic] systems like Perforce, you can scale a single repo nearly as large as any company will need. Emphasis on that nearly. At a certain po…
I get this impression every time I see a "look what neat scalability thing we did" post from Facebook Engineering. It's great that they're able to achieve such technical feats, but they refuse to acknowledge that maybe they're using the technology wrong. I'm reminded of the time they hacked the Dalvik VM on Android because apparently they had too many method names for the Dalvik VM to handle. http://jaxenter.com/face…
It is insane, but it never comes all at once. I have been at a company that had this type of internal insanity, and it is a bit of a "boil the frog" thing.
Each individual step seems resonable and sane thing -- but when you look at the result of all those steps -- you stand back and just stare in awe at the horror you have created... it hits you just how far you have drifted.
In a recent meeting, we had a corporate redirection a bit -- and a co-worker simply asked the question "Is this a sane approach to what our actual problem is?" -- glad to be working with a crew who askes those kinds of questions.
Re: Scaling Mercurial at Facebook
#165Earlier quoted context omitted.
I work in several branches in the same file system directory. Branching in mercurial is as simple as "hg branch branchname", and switching branches is just "hg update branchname". As long as you make sure you commit everything you can easily work in one directory. Merging changes across branches is easy too, as you only need to specify the revision number.
If you push while having open local branches, they get pushed to the remote (or at least that's how it was last time I used them). That's not what you want, in most cases (git branches are local).
Mercurial won't push new branches unless you specify -f or --new-branch. You can selectively push only the branches you want with hg push -r branch_or_revision (hg push -r . for the current revision). You can also hide branches using phases, so they won't be pushed even with -f or --new-branch.
Re: Scaling Mercurial at Facebook
#166Earlier quoted context omitted.
They're not exactly as supported, because they're not on by default , and they're not the thing everyone in the Mercurial culture tells you to use as the obvious solution to problems. Version control is as much about what other people do as what you do, and what other people do tends to align most closely with the defaults and what the tools encourage.
> They're not exactly as supported, because they're not on by default I really don't get this. In almost any software there are features that are not 'on by default', but it's still implemented and supported (except explicitly stated otherwise). How can this be a reason to not use the functionality when you need it?
It's like using Outlook to respond to mail on a mailing list. Sure, it's theoretically possible to use some combination of manual configuration and extra work to construct a post that won't violate social norms, but the tool doesn't encourage it, so mailing lists with a significant number of Outlook users on them tend to fail to follow the standard mailing list conventions of other lists, and develop their own conventions.
The same holds for any tool used in a workflow that involves anyone other than yourself: any tool with a default workflow tends to encourage people to follow that default workflow.
Re: Scaling Mercurial at Facebook
#167Can anyone explain to me how Mercurial is used at Facebook in conjunction with Murder (https://github.com/lg/murder)?
Re: Scaling Mercurial at Facebook
#168Earlier quoted context omitted.
> Modularity tends to obviate the need for large, atomic refactorings. "Tends to". But when you're dealing with code at Facebook's scale, things that "tend not to happen" actually happen quite a lot. In fact, you must plan for them as a matter of course. So yes, modularity is great, and I because I'm a nice guy I assume Facebook aren't a pack of idiots and that they're writing nice modular code. But even if that's th…
Perhaps I don't understand the whole situation here. I hear "all of our code is in one repository" and I think "GMail and Google Maps are in the same repository, in the same repository with GoLang, in the same repository with AdWords." The more I think about it, the more I think your post reveals a lack of maturity in our industry that lends credence to the pro-engineering-licensing argument that I've argued against…
I think you have this picture in your mind of just one big pile of spaghetti code. The truth is way more nuanced. All the code may be in one big repository, but that doesn't mean it is not well-managed. The code is still modular; code is managed in libraries with clean APIs, and so on.
But whether you keep your code in one big repository or many small repositories, you still need to track and manage those the dependencies between the various parts.
For instance, when a bug is discovered in library X, you need to know which binaries running in production were compiled against a buggy version of that library. At Google we can say "A bug was introduced at revision NNNNN and fixed at revision MMMMM. Please recompile and redeploy any binaries that were built within that range." (And we have tools to tell us exactly which binaries those are.) This is something that using One Giant Repository gives us for free.
If you were taking the many-small-repos approach, for any given binary you'd need to track the various versions of each of its dependencies. You'd also need to manage release engineering for each and every one of those projects, which slows progress a lot (although we do have a release process for really critical components).
But like I said, there are relative advantages and disadvantages to either approach. To write software at this scale requires tools, processes, and good communication. Where you keep your code, at the end of the day, is actually a pretty minor concern compared to all the other stuff you need to do to ship quality products.
Re: Scaling Mercurial at Facebook
#169Earlier quoted context omitted.
> Does the standard branch workflow still expect you to have a separate repository and directory per branch? You're probably confusing mercurial with bazaar, mercurial has always had branches (though they're not quite the same as git, mercurial's bookmarks are more closely related to git branches) and anonymous heads (contrary to git, an unnamed head is not stuck in limbo). > I don't care about plugins That's stupid,…
Mercurial's branches are server side (learned that the hard way the first time I made a branch), git's are client side.
I think you are confused, git and hg branches are very similar (I'd try to help but I'm not sure what the confusion is).
Re: Scaling Mercurial at Facebook
#170Earlier quoted context omitted.
Does the standard branch workflow still expect you to have a separate repository and directory per branch? I don't care about plugins, here; if the standard workflow doesn't include incredibly lightweight branches, I'll stick with a version control system that does. Likewise, does the standard workflow still intentionally make it painful to rearrange changes in your local repository to construct a series of patches?…
Why do you think mercurial branches are not lightweight? Which operation (create, close, push, pull) on branches is noticeably slower than git's?
According to #mercurial, branches (even closed ones) begin to noticeably slow down the repository when you reach around 2000. That's not at all impossible to reach with fine-grained feature branches. Bookmarks are really just pointers to commits, no branches in any sense, so they cannot be used for feature branches either. As a result, we simply don't use feature branches in Mercurial. It's literally the only issue I can think of.