Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The largest Git repo

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

201–210 of 416 posts

Re: The largest Git repo

#201

Post by holtalanm »

cryptonector wrote: post #194

At Sun Microsystems, Inc., (RIP) we have many "gates" (repos) that made up Solaris. Cross-gate development was somewhat more involved, but still not bad. Basically: you installed the latest build of all of Solaris, then updated the bits from your clones of the gates in question. Still, a single repo is great if it can scale, and GVFS sounds great! But that's not what I came in to say. I came in to describe the rebase…

we use a rebase workflow in git at my current employer, and it is amazing.

previous employer used a merge workflow (primarily because we didnt understand git very well at the time), and there were merge conflicts all the time when pulling new changes down or merging new changes in.

It was a headache to say the least. As the integration manager for one project, I usually spent the better part of an hour just going through the pull requests and merge conflicts from the previous day. I managed a team that was on the other side of the world, so there were always new changes when I started working in the morning.

Re: The largest Git repo

#202

Post by manyoso »

I would pay money to see a video camera of Linus' face reading this article. I think we'd probably get impossible new shades of the color red heretofore unknown to humanity.

Re: The largest Git repo

#203

Post by Jach »

Eyas wrote: post #180
erikpukinskis wrote: post #126

Earlier quoted context omitted.

> - Becomes hard to make cross-cutting changes This does seem like a negative, doesn't it? But it's not. Making it hard to make cross-cutting changes is exactly the point of splitting up a repo. It forces you to slow down, and—knowing that you can only rarely make cross-cutting changes—you have a strong incentive to move module boundaries to where they should be. It puts pressure on you to really, actually separate c…

You say that, but it is very telling that every large company out there (Google and Facebook come to mind) go for the single-repository approach. I'm sure that, when dealing with stakeholder structures where different organizations can depend on different bits and pieces, having multiple repositories with difficulty of making breaking and cross-cutting changes, becomes good. From the view of a single organization whe…

It is very telling. Google has a cloud supercomputer doing nothing but building code to support their devs. I don't know about Facebook. (I really don't -- I'm constantly amazed that they have as many engineers that they do, what do they all work on?) Where I work (https://medium.com/salesforce-engineering/monolith-to-micros...) there's a big monolith but with more push towards breaking things up, at the architecture level and also on the code organization level. We also use and commit to open source projects (that use git), so to integrate those with the core requires a bit more effort than if they were there already but it's not a big burden and the benefits of having their tendrils being self-contained are big.

Which brings me to the point that in the open source world, you can't get away with a single-repository approach for your large system. And that also is telling, along with open source's successes. So which approach is better in the long run? I'd bet on the open source methods.

Re: The largest Git repo

#204

Post by rbanffy »

vtbassmatt wrote: post #25
tobyhinloopen wrote: post #5

I wonder why Windows is a single repository - Why not split it in separate modules? I can imagine tools like Explorer, Internet Explorer/Edge, Notepad, Wordpad, Paint, etc. all can stay in its own repository. I can imagine you can even further split things up, like a kernel, a group of standard drivers, etc. If that is not already the case (separate repos, that is), are the plans to separate it in the future?

Really good question. Actually, splitting Windows up was the first approach we investigated. Full details here: https://www.visualstudio.com/learn/gvfs-design-history/ Summary: - Complicates daily life for every engineer - Becomes hard to make cross-cutting changes - Complicates releasing the product - There's a still a core of "stuff" that's not easy to tease apart, so at least one of the smaller Windows repos would…

I remember that, in the 90's, you'd often get new UI elements in Office releases that then would eventually move into Windows. There was a technical reason - the cross-cutting - but there also seemed to be a marketing reason - the moment those UI elements became part of core Windows, all developers (yours truly included) would be able to use those elements, effectively negating Office the fresh look before the competition.

Re: The largest Git repo

#205

Post by dom0 »

Peaker wrote: post #167
js2 wrote: post #145

Windows, because of the size of the team and the nature of the work, often has VERY large merges across branches (10,000’s of changes with 1,000’s of conflicts). At a former startup, our product was built on Chromium. As the build/release engineer, one of my daily responsibilities was merging Chromium's changes with ours. Just performing the merge and conflict resolution was anywhere from 5 minutes to an hour of my t…

When you have that many conflicts, it's often due to massive renames, or just code moves. If you use git-mediate[1], you can re-apply those massive changes on the conflicted state, run git-mediate - and the conflicts get resolved. For example: if you have 300 conflicts due to some massive rename, you can type in: git-search-replace.py[2] -f oldGlobalName///newGlobalName git-mediate -d Succcessfully resolved 377 confl…

Also git rerere

When maintaining multiple release lines and moving fixes between them:

Don't use a bad branching model. Things like "merging upwards" (=committing fixes to the oldest branch requiring the fix, then merging the oldest branch into the next older branch etc.), which seems to be somewhat popular, just don't scale, don't work very well, and produce near-unreadable histories. They also incentivise developing on maintenance branches (ick).

Instead, don't do merges between branches. Everything goes into master/dev, except stuff that really doesn't need to go there (e.g. a fix that only affects a specific branch(es)). Then cherry pick them into the maintenance branches.

Re: The largest Git repo

#206

Post by rbanffy »

The pain increases with the square of the number of files and to the fourth power of the dependencies between specific versions of them.

I'm not sure a big repo is a wise thing, even though I understand it may make sense for multiple reasons for a company, understanding it may damage brains far more sophisticated than mammalian ones.

Re: The largest Git repo

#207

Post by vtbassmatt »

YeGoblynQueenne wrote: post #191

So, if windows engineers are using git now, who is using TFVC? That's Team Foundation Version Control- the original TFS version control engine.

Lots and lots of external customers, and a handful of internal folks. FWIW Windows was never on TFVC (at least not the main development group).

Re: The largest Git repo

#208

Post by cryptonector »

holtalanm wrote: post #201
cryptonector wrote: post #194

At Sun Microsystems, Inc., (RIP) we have many "gates" (repos) that made up Solaris. Cross-gate development was somewhat more involved, but still not bad. Basically: you installed the latest build of all of Solaris, then updated the bits from your clones of the gates in question. Still, a single repo is great if it can scale, and GVFS sounds great! But that's not what I came in to say. I came in to describe the rebase…

we use a rebase workflow in git at my current employer, and it is amazing. previous employer used a merge workflow (primarily because we didnt understand git very well at the time), and there were merge conflicts all the time when pulling new changes down or merging new changes in. It was a headache to say the least. As the integration manager for one project, I usually spent the better part of an hour just going thr…

Yes! One of the most important advantages of a rebase workflow is that you can see immediately what upstream commits your conflict with, as opposed to some massive merge you have to go chasing branch history to figure out the semantics of the change in question.

"Amazing" is right. Sun was doing rebases in the 90s, and it never looked back.

Re: The largest Git repo

#209

Post by flamedoge »

Peaker wrote: post #167
js2 wrote: post #145

Windows, because of the size of the team and the nature of the work, often has VERY large merges across branches (10,000’s of changes with 1,000’s of conflicts). At a former startup, our product was built on Chromium. As the build/release engineer, one of my daily responsibilities was merging Chromium's changes with ours. Just performing the merge and conflict resolution was anywhere from 5 minutes to an hour of my t…

When you have that many conflicts, it's often due to massive renames, or just code moves. If you use git-mediate[1], you can re-apply those massive changes on the conflicted state, run git-mediate - and the conflicts get resolved. For example: if you have 300 conflicts due to some massive rename, you can type in: git-search-replace.py[2] -f oldGlobalName///newGlobalName git-mediate -d Succcessfully resolved 377 confl…

or worse, formatting changes

Re: The largest Git repo

#210

Post by hinkley »

daxfohl wrote: post #197
ericfrederich wrote: post #168

Earlier quoted context omitted.

This may be the thing that gets Google to switch. They like having every piece of code in a single repository which Git cannot handle. Now that it is somewhat proven, maybe Google will leverage GVFS on Windows and create a FUSE solution for Linux.

I'd rather see google open up their monorepo as a platform, and compete with github. git is fine, but there's something compelling about a monorepo. Whether they do it one-monorepo-per-account, or one-global-monorepo, or some mix of the two, would be interesting to see how it shapes up.

This is the main thing I miss about subversion. You could check out any arbitrary subdirectory of a repository. On two projects the leads and full stack people had the whole thing checked out, everybody else just had the one submodule they were responsible for. Worked fairly well.
Post reply on HN