Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Mercurial at Facebook

code.facebook.com

Post reply on HN
Print view View original

221–230 of 245 posts

Re: Scaling Mercurial at Facebook

#221

Post by Crito »

dubcanada wrote: post #120
Crito wrote: post #24

I wrote a comment about the scaling of repositories (and specifically Facebook's issues) a few days ago that was wiped out by the HN crash, but I've managed to recover it from HNSearch: Facebook's problem is that they were trying to scale with Git improperly. With conventional CVS[sic] systems like Perforce, you can scale a single repo nearly as large as any company will need. Emphasis on that nearly. At a certain po…

I think they just want to modify Git and don't have any solid C developers that can make such things. So they turned to a python solution which is perfectly fine. But webkit, and chromium, as well as other GIANT projects which as far as I know are larger then Facebook seem to work fine on Git.

The idea is that you should not have to modify git or mercurial. Instead of modifying git or mercurial to handle a single repo for your entire company, you should create a system that uses them to handle many repos in your company. You don't need C programmers to do this with git.

Re: Scaling Mercurial at Facebook

#222

Post by Crito »

Shamanmuni wrote: post #160
Uchikoma wrote: post #46

Earlier quoted context omitted.

90% of people choose Git because everyone else uses it. Noone has been fired for buying IBM.

I suspect many of us use Git because it was made by Linus Torvalds ( I think everyone agrees that he is a great developer) and is used in the Linux kernel. If it can handle that, then you can be pretty sure it will handle whatever you throw at it (unless you are Facebook, it seems). Add Github to that mix and you can see why so many developers sleep like babies at night. Of course it's not rigorous, but the choice is…

The linux kernel is much smaller than the entire codebase of Facebook.

Re: Scaling Mercurial at Facebook

#223

Post by Crito »

igravious wrote: post #98
ZenoArrow wrote: post #78

Earlier quoted context omitted.

To be honest, whilst we have no way to accurately determine whether the code is a mess without a chance to see it, the most surprising line of this article (in my opinion) was that the code base was larger than the Linux kernel. I'm not seeing anything on the front end that would warrant such complexity, guessing a large chunk of the code base is server code. Would be interested in reading a summary of the components…

This rather surprised me as well. I tend to think of the Linux kernel as one of the larger single code-bases out there. Am I wrong?

The kernel is measured in hundreds of megabytes. There are companies with codebases measured in terabytes.

Re: Scaling Mercurial at Facebook

#224

Post by twic »

dgesang wrote: post #149
novaleaf wrote: post #114

Earlier quoted context omitted.

I personally love mercurial (simpler than git), and have been a bit nervious over the last year or so with the mindshare shift to git. So hearing about this (FB all in with hg) ensures that hg won't be falling behind... at least in the nearterm.

Don't believe the hype. hg will be around for some time.

Mercurial is the new FreeBSD.

Re: Scaling Mercurial at Facebook

#225

Post by hibbelig »

VladRussian2 wrote: post #28
LukeHoersten wrote: post #3

I love the Mercurial community. We use Mercurial at work and I'm able to get instant support in IRC for any issue we have with an awesome signal/noise ratio. I'm glad Facebook is contributing back so much as well. My suspicion is that open source projects tend towards Git because of GitHub but I think a lot of companies who don't have the option of external code hosting lean towards Mercurial. All anecdotal observati…

>My suspicion is that open source projects tend towards Git because of GitHub but I think a lot of companies who don't have the option of external code hosting lean towards Mercurial. GitHub is a consequence not the cause (ponder for a moment why there is no MercurialHub...) It is about ability to choose best source control tool for multi-versioned distributed concurrent development. Open source devs have such choice…

Isn't MercurialHub called Bitbucket?

Re: Scaling Mercurial at Facebook

#226

Post by michal8181 »

fhd2 wrote: post #170
k_bx wrote: post #21

Earlier quoted context omitted.

Why do you think mercurial branches are not lightweight? Which operation (create, close, push, pull) on branches is noticeably slower than git's?

According to #mercurial, branches (even closed ones) begin to noticeably slow down the repository when you reach around 2000. That's not at all impossible to reach with fine-grained feature branches. Bookmarks are really just pointers to commits, no branches in any sense, so they cannot be used for feature branches either. As a result, we simply don't use feature branches in Mercurial. It's literally the only issue I…

Not true any longer.

As described in http://mercurial.808500.n3.nabble.com/named-branches-vs-book...

> We have users with thousands of named branches in production and have > done tests on up to 10k branches and the performance impact is fairly > minimal.

Re: Scaling Mercurial at Facebook

#227

Post by michal8181 »

fhd2 wrote: post #171
masklinn wrote: post #68

Earlier quoted context omitted.

Mercurial's equivalent to git branches (movable pointer to a commit rather than embedded commit metadata) is bookmarks.

From what I've seen, bookmarks are not "branches" in any sense, really just pointers to commits. I've tried several times to use bookmarks for feature branches (read: branches developed in parallel to each other and the default branch). I thought I just can't figure it out, but it really seems impossible at this point.

Branches in git are exactly pointers to graph. If you delete such a pointer you loose your branch.

Even worse - with next git gc you will loose your data as parts of graph with no pointer is assumed to be dead by git.

Re: Scaling Mercurial at Facebook

#228

Post by michal8181 »

igravious wrote: post #98
ZenoArrow wrote: post #78

Earlier quoted context omitted.

To be honest, whilst we have no way to accurately determine whether the code is a mess without a chance to see it, the most surprising line of this article (in my opinion) was that the code base was larger than the Linux kernel. I'm not seeing anything on the front end that would warrant such complexity, guessing a large chunk of the code base is server code. Would be interested in reading a summary of the components…

This rather surprised me as well. I tend to think of the Linux kernel as one of the larger single code-bases out there. Am I wrong?

Yes.

As described here http://www.informationisbeautiful.net/visualizations/million... Facebook code base (~60 MLOC) if almost 4 times bigger than Linux 3.1 (a mere 15 MLOC).

Re: Scaling Mercurial at Facebook

#229

Post by shoover »

yonran wrote: post #213
shoover wrote: post #209

Earlier quoted context omitted.

Nice list. What then are the mechanics of making a breaking change in a library, file format, network protocol, etc. in the single repo system?

Ideally, to make a breaking API change, you change the function and all the references within a single commit. Since this repository is only used for statically compiled programs, there is no need to keep the old API anymore. For protocols and file formats, Google universally uses protocol buffers with many optional fields. The protocol buffer library’s default is that when you read a protocol buffer, modify it, and…

Useful flexibility from protocol buffers there.

To me it seems like having to change all references for a breaking API change could be a debilitating amount of work in some cases. Do you then make your breaking change to a branch and lobby for other teams to catch up before merging to the main branch? What about situations where you have a legion of stable legacy applications that may not be worth updating for any reason other than critical bugs?

Re: Scaling Mercurial at Facebook

#230

Post by Pxtl »

com2kid wrote: post #218
ed_blackburn wrote: post #37

I wonder what they use at Microsoft. For their sake I hope they don't subject their own engineers to TFS.

We eat our own dog food, most teams have moved over to TFS by now. I don't know about the larger orgs (Windows, Office) but for smaller groups TFS is how it is. MS doesn't have a unified build environment, every team generally does their own thing. It has its pluses and minuses. Shared code would occasionally be useful at Microsoft, but not as often as you'd think. Generally when relying on another team's code, it is…

> We eat our own dog food.

Except for that whole ".NET framework" thing.

Post reply on HN