Live data from Hacker News

The largest Git repo

blogs.msdn.microsoft.com

361–370 of 416 posts

Re: The largest Git repo

#362
post #314

Earlier quoted context omitted.

This just blew my mind. I'm gonna go home and see about combining all my projects. That seems very useful!

It makes things tricky if you want to opensource just one of your projects.

Well, there's always "git filter-branch".

Not that I'd want to run it on such a mega-repository; it takes long enough running it on an average one with a decade of history.

Re: The largest Git repo

#363

A handful of us from the product team are around for a few hours to discuss if you're interested.

"A handful of us from the product team are around for a few hours to discuss if you're interested." Thanks! This is a little off-topic, but why can't Windows 10 users conclusively disable all telemetry? (I consider the question only a little off-topic, because I have the impression that this story is part of an ongoing Microsoft charm-offensive.)

Haha, no answer, as expected. HN got butthurt as well lol.

Re: The largest Git repo

#364
The day has come that Microsoft employees are celebrating how good they're getting at running Linus Torvalds' source code management tool.

Cats and dogs, flying pigs.

Might be good to start work on a compatible client and server for FUSE-based systems (Linux, OpenBSD, macOS [with a FUSE kernel module]).

Re: The largest Git repo

#365

Earlier quoted context omitted.

I'd rather see google open up their monorepo as a platform, and compete with github. git is fine, but there's something compelling about a monorepo. Whether they do it one-monorepo-per-account, or one-global-monorepo, or some mix of the two, would be interesting to see how it shapes up.

"one-global-monorepo" caused me to envision a beautiful/horrifying Borg-like future where all code in the universe was in a single place and worked together.

This is how Linux (and BSD, and so on) distributions work. Of course there are proprietary and niche outliers, but you can't forbid those in the first place.

Re: The largest Git repo

#366

Earlier quoted context omitted.

Windows developed an extension that lets them do conflict resolution in the web. We have a server-side API that it calls into, but the extension isn't fundamentally different from using BeyondCompare or $YOUR_FAVORTE_MERGETOOL.

An extension to what? Could you open source it?

Extension to VSTS [1], sorry. We're working with them on making the extension available to the public. It's possible we could open source it as well; I'll poke around.

[1] https://www.visualstudio.com/en-us/docs/integrate/extensions...

Re: The largest Git repo

#367
post #145

Windows, because of the size of the team and the nature of the work, often has VERY large merges across branches (10,000’s of changes with 1,000’s of conflicts). At a former startup, our product was built on Chromium. As the build/release engineer, one of my daily responsibilities was merging Chromium's changes with ours. Just performing the merge and conflict resolution was anywhere from 5 minutes to an hour of my t…

I spent the better part of a year in a team that was merging and bug fixing our companies engine releases into the customer (and owning company) code base. We also had to deal with different code repositories and version control systems.

We ended up with a script that created a new git repository, checked out the base version of the code there. Then created a branch for our updated release and another for their codebase. Then attempted to do a merge between the two. For any files which couldn't be automatically merged it created a set of 4 files, the original then a triplet for doing a 3-way merge.

This is also when I bought myself a copy of Beyond Compare 4, which fits perfectly for the price and feature set for what we needed.

Re: The largest Git repo

#368
post #27

Earlier quoted context omitted.

Why split it into separate modules? Seeing that big companies are very successful with monorepos (Google, Facebook, Microsoft), has made me reconsider if repository modularization is actually worth it. There are a host of advantages to not modularizing repos, and I'm beginning to believe they outweigh those of modular repos.

> Why split it into separate modules? Well, because of the unbelievable amount of engineering work involved in trying to get Git to operate at such insane scale? To say nothing of the risk involved in the alternative. This project in particular could easily have been a catastrophe.

Microsoft has about 50k developers. When you're dealing with an engineering organization of this size, you're looking at a run rate of $5B a year, or about $20M a day. It's a no-brainer to spend tens or even hundreds of millions of dollars on projects like this if you're going to get even a few percentage points of productivity.

It can be hard to understand this as an individual engineer, but large tech companies are incredible engineering machines with vast, nearly infinite, capacity at their disposal. At this scale, all that matters is that the organization is moving towards the right goals. It doesn't matter what's in the way; all the implementation details that engineers worry about day-to-day are meaningless at this scale. The organization just paves over over any obstacles in pursuit of its goal.

In this case Microsoft decided that Git was the way to go for source control, that it's fundamentally a good fit for their needs. There was just implementation details in the way. So they just... did it. At their scale, this was not an incredible amount of work. It's just the cost of doing business.

Re: The largest Git repo

#369
post #308

Earlier quoted context omitted.

A problem with rebase workflows that I don't see addressed (here or in the replies) is: if I have, say, 20 local commits and am rebasing them on top of some upstream, I have to fix conflicts up to 20 times; in general I will have to stop to fix conflicts at least as many times as I would have to while merging (namely 0 or 1 times). Moreover, resolution work during a rebase creates​ a fake history that does not reflec…

Well, git rerere helps here, though, honestly, this never happens to me even when I have 20 commits. Also, this is what you want, as it makes your commits easier to understand by others. Otherwise, with thousands of developers your merge graph is going to be a pile of incomprehensible spaghetti, and good luck cherry-picking commits into old release patch branches! Ah, right, that's another reason to rebase: because y…

rebase : centralized repo :: merge : decentralized repo

rebase : linked-list :: merge : DAG

If the work/repo is truly distributed and there isn't a single permanently-authoritative repo, a "clean, linear" history is nonsensical to even try to reason about.

In all cases it is a crutch: useful (and nice, and sufficient!) in simple settings, but restricting/misleading in more complex ones (to the point of causing many developers to not see the negative space).

You can get very far thinking of a project as a linked list, but there is a lot to be gained from being able to work effectively with DAGs when a more complex model would better fit the reality being modeled.

It's harder to grok the DAG world because the tooling is less mature, the abstractions are more complex (and powerful!), and almost all the time and money up to now has explored the hub-and-spoke model.

In many areas of technology, however, better tooling and socialization around moving from linked-lists (and even trees) to DAGs is going to unlock more advanced capabilities.

Final point: rebasing is just glorified cherry-picking. Cherry-picking definitely also has a role in a merge-focused/less-centralized world, but merges add something totally new on top of cherry-picking, which rebase does not.

Re: The largest Git repo

#370
post #350

Earlier quoted context omitted.

Lots and lots of external customers, and a handful of internal folks. FWIW Windows was never on TFVC (at least not the main development group).

What is the opposite of dogfooding?

That wouldn't be dogfooding - the windows team is not responsible for TFVC, innit.
Post reply on HN