Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Mercurial at Facebook

code.facebook.com

Post reply on HN
Print view View original

201–210 of 245 posts

Re: Scaling Mercurial at Facebook

#201

Post by exDM69 »

greatsuccess wrote: post #128

Partitioning is the answer. In a repo that size 99% of the history is useless to anyone. You wouldnt manage a database like this so why force SCM down this path? If they used git with say only the last year of history in it they would be having zero issues.

Partitioning may be the answer but this is a huge problem for corporations like Facebook (and the one I am working for). If things have been done with one giant repo from day one, splitting it is going to be a major engineering/political/social problem when there's thousands of engineers working on the code base and you can't just shut down business for the duration of the migration.

Re: Scaling Mercurial at Facebook

#202

Post by oblio »

enneff wrote: post #115
X-Istence wrote: post #97

Earlier quoted context omitted.

You don't need a single repo to be able to run tests across all existing tools. If you have proper dependency management set up, you make the change, push it and a CI server goes off and builds it, then all dependent projects get rebuilt and tested...

No, but you do need a single repo if you want to make the API change and update all the dependencies in one fell swoop.

Does this matter that much if you version everything?

Commit in library -> successful library build, failed client build. Commit in client -> successful library build, successful client build.

In both situations you don't really care about the intermediary broken build - you still have the previous library/client versions and can use those. Once everything is committed & fixed you have the new versions and you can upgrade.

The only problem I see if there's a long delay before the second commit. But this can be prevented by a fast CI cycle (always a good idea) and sending notifications for failures across teams (i.e. the library committer is notified that the client build for his commit failed).

Re: Scaling Mercurial at Facebook

#203

Post by Touche »

thrownaway2424 wrote: post #159
Touche wrote: post #66

Earlier quoted context omitted.

> * You immediately get improvements from upstream projects without having to get them manually. You also immediately get regressions. Not trying to be dismissive, but we fundamentally have different software philosophies if you think this point (which is the essence of most of your points) is a good thing that should be encouraged.

Immediate regressions are good! If someone at Google breaks my code, I will know within half an hour at the latest and I will tell them to go fix it or just revert their changes myself. Immediate regressions also go perfectly with daily (or hourly!) releases. If there's a performance problem it will be identified early and I will only have thousands of changes to investigate instead of tens of millions. Imagine if I…

I don't just mean performance regressions. Someone upstream can change an API in a way that doesn't fit well with your use-case, goes in and "fixes" your code (makes sure all the tests pass) to fit the new API but makes it less maintainable in the process.

Re: Scaling Mercurial at Facebook

#204

Post by jordigh »

ibrahima wrote: post #157
Sssnake wrote: post #109

Earlier quoted context omitted.

No, git runs into issues before mercurial does. Just saying the opposite of reality doesn't make it so.

Did you even read the post? Stock git is significantly faster than stock mercurial, it's only after they hack it up that they make it faster.

There are some operations in which git is way slower than Mercurial. Two examples are cloning and blame/annotate. git's data structures are optimised for some operations, but not for others.

Re: Scaling Mercurial at Facebook

#205

Post by jordigh »

CmonDev wrote: post #177
boklm0 wrote: post #40

According to this page, Mercurial project leader is currently working for Facebook: http://mercurial.selenic.com/wiki/mpm

Google bought Git devs, Facebook bought Mercurial devs - wise choice for companies that even order custom CPUs.

There are some hg devs at Google too. :-)

Re: Scaling Mercurial at Facebook

#206

Post by moron4hire »

thrownaway2424 wrote: post #152
moron4hire wrote: post #130

Earlier quoted context omitted.

Perhaps I don't understand the whole situation here. I hear "all of our code is in one repository" and I think "GMail and Google Maps are in the same repository, in the same repository with GoLang, in the same repository with AdWords." The more I think about it, the more I think your post reveals a lack of maturity in our industry that lends credence to the pro-engineering-licensing argument that I've argued against…

The fact that Google is one of the largest, most successful software companies in history and you are arguing on the internet using the handle "moron4hire" just about sums up the merits of your position.

And your name is "thrownaway2424".

Just because a company is big doesn't mean they are working in the best way, or working in a way that is to the best benefit of the public. Might does not make right. We don't let large architectural engineering firms get away with doing whatever the hell they want just because they should have a proprietary interest in doing the best job possible, and we shouldn't be letting banks do it, either.

Yes, it's hard. Boo hoo. So is making safe cars. But you don't get the option to take the easy way out. Solve the hard problem, it's the job.

Re: Scaling Mercurial at Facebook

#207

Post by moron4hire »

enneff wrote: post #168
moron4hire wrote: post #130

Earlier quoted context omitted.

Perhaps I don't understand the whole situation here. I hear "all of our code is in one repository" and I think "GMail and Google Maps are in the same repository, in the same repository with GoLang, in the same repository with AdWords." The more I think about it, the more I think your post reveals a lack of maturity in our industry that lends credence to the pro-engineering-licensing argument that I've argued against…

I think you have this picture in your mind of just one big pile of spaghetti code. The truth is way more nuanced. All the code may be in one big repository, but that doesn't mean it is not well-managed. The code is still modular; code is managed in libraries with clean APIs, and so on. But whether you keep your code in one big repository or many small repositories, you still need to track and manage those the depende…

No, you're giving me the right picture, and it's mostly the picture I thought it was.

These issues are the same issues the rest of us in the world have to deal with when working with your APIs. Someone in one of the sibling comments has linked to an article discussing Bezos giving the command from on-high that Amazon would dog-food all of its APIs.

And apparently it isn't so minor of a concern if it warrants the first blog post out of Facebook in the last 3 weeks. Maybe that's just a coincidence that this is the first blog post of the year. It seems like they are trying to say "it's a big enough deal that we have and we're going to spend a lot of money on it."

Maybe the problem is that Facebook and Google are just too big. They might have to be as big as they are to be doing the work that they are doing, but is that really the best thing for the rest of the world?

Re: Scaling Mercurial at Facebook

#208

Post by moron4hire »

Crito wrote: post #56
VladRussian2 wrote: post #45

Earlier quoted context omitted.

>With perforce, you can reasonably expect to run into this brick wall somewhere in the neighborhood of terabytes of metadata and dozens of transactions per second. That changes depending on what sort of beastly hardware you are willing to throw at your version control team. >Git of course hits a brick wall much sooner, somewhere around single-digit gigabytes of data (depending heavily on the average size of every obj…

That is certain. If you are trying to run at a very large scale with a single repo, Perforce (or apparently Mercurial) is the way to go. The problem is "how do you grow even further ?" There isn't a perfect solution to that (yet), but I think the only existing workable (although absolutely imperfect) solution is splitting your codebase into many small repos. Thankfully very few companies need to worry about this prob…

At what point do we say "enough" though? I mean, start from the absolute maximum: how would one make a system to control all of the source code in the world?

For all of the source code in the world that is source-controlled, it exists in separate repos. It doesn't exist in one repo, and where there are inter-project dependencies, it is the dependency consumer's responsibility to keep abreast of the changes in the dependency and integrate them as necessary. It is self-organizing.

So, if Google and Facebook have grown beyond what can be done with a single repo, either they are not being mindful of their engineering practices (which puts the public at significant risk), or they are approaching a scale that is more similar to "world scale" than it is "corporate scale".

Which I would also say is putting the public at significant risk.

Re: Scaling Mercurial at Facebook

#209

Post by shoover »

yonran wrote: post #41
Touche wrote: post #9

> We could have spent a lot of time making it more modular in a way that would be friendly to a source control tool, but there are a number of benefits to using a single repository. Pray tell?

I worked at Google (in a team using Perforce) and now work at a different company that uses multiple interdependent projects using Maven. Using a single monolithic codebase along with a build tool that statically builds everything at trunk has its advantages: * You immediately get improvements from upstream projects without having to get them manually. * You can unambiguously answer the question, What code am I using…

Nice list. What then are the mechanics of making a breaking change in a library, file format, network protocol, etc. in the single repo system?

Re: Scaling Mercurial at Facebook

#210

Post by rossjudson »

ahomescu1 wrote: post #138
mushishi wrote: post #55

Earlier quoted context omitted.

Could you elaborate on why to avoid mq?

mq keeps its patches as an internal per-repo stack. This is much weaker than git's tree branches. Here's an example: let's say I've been working on a repo, and have 2 current mq patches called A and B pushed. There's no way for me to pop A and B, then apply another patch called D in the current repo (at least as far as I could figure out). Also, even if there were, I'd have no way of then pushing A and B on top of D.…

I do this all the time. Pop B, pop A, reorder patch queue to put D first, push D, push A, push B. Check the docs.
Post reply on HN