This is so awesome. Brilliant move MS! In addition to enabling Windows engineers to be significantly more productive (eventually), it will go a long way to enabling engineers in other departments to contribute to Windows. For example, I used to work in the Azure org and once noticed a relatively simple missing feature in Windows. I filed a bug and was in contact with a PM who suggested if I wanted I could work on adding it myself. I dipped my toe in, but the onboarding costs were just too high and I quickly decided against it. With Windows on git, much more likely to have dived in.
The largest Git repo
11–20 of 416 posts
Re: The largest Git repo
#12Earlier quoted context omitted.
The article mentions relying on a windows filesystem driver. Two questions about that: 1) Why include it in default windows? It seems that 99.99% of users would never even know it existed, let alone use it 2) Does that mean GVFS isn't useable on *nix systems? Any plans to make it useable, if so?
It's not included in Windows, which is why they have a signed drop of the driver for you to install. Even internally we have to install the driver. E- oops, missed that line. Cool, wonder if it will show up in more than Git
Yes, I was referring to the plan to include it in future Windows builds
Re: The largest Git repo
#1319 seconds for a commit (add + commit) might be long but the new improvements look promising (down to ~10s).
(Please correct me if the COMMIT column in the perf table includes the staging operations.)
This looks awesome. I just wish Facebook would also share some perf and time statistics on their own extensions for Mercurial, last time I checked their graphs were unitless.
Re: The largest Git repo
#14I wonder why Windows is a single repository - Why not split it in separate modules? I can imagine tools like Explorer, Internet Explorer/Edge, Notepad, Wordpad, Paint, etc. all can stay in its own repository. I can imagine you can even further split things up, like a kernel, a group of standard drivers, etc. If that is not already the case (separate repos, that is), are the plans to separate it in the future?
So, this is actually pretty common. I know that both Google and Facebook use a huge mono-repo for literally everything (except I think Facebook split out their Android code into a separate repo?). So, all of Facebook's and Google's code for front-end, back-end, tools, infrastructure, literally everything, lives in one repo.
It's news to me that Windows decided to go that route too. Personally, I think submodules and git sub-trees suck, so I'm all for putting things in a monorepo.
Re: The largest Git repo
#15A handful of us from the product team are around for a few hours to discuss if you're interested.
Sure, a couple of questions: 1. How do you measure "largeness" of a git repo? 2. How are you confident that you have the largest? 3. How much technical debt does that translate to?
1. Saeed is writing a really nice series of articles starting here: https://www.visualstudio.com/learn/git-at-scale/ In the first one, he lays out how we think about small/medium/large repos. Summary: size at tip, size of history, file count at tip, number of refs, and number of developers.
2. Fairly confident, at least as far as usable repos go. Given how unusable the Windows repo is without GVFS and the other things we've built, it seems pretty unlikely anyone's out there using a bigger one. If you know of something bigger, we'd love to hear about it and learn how they solved the same problems!
3. Windows is a 30 year old codebase. There's a lot of stuff in there supporting a lot of scenarios.
Re: The largest Git repo
#16I wonder why Windows is a single repository - Why not split it in separate modules? I can imagine tools like Explorer, Internet Explorer/Edge, Notepad, Wordpad, Paint, etc. all can stay in its own repository. I can imagine you can even further split things up, like a kernel, a group of standard drivers, etc. If that is not already the case (separate repos, that is), are the plans to separate it in the future?
their earlier blogpost goes into it a bit https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-g... :
The first big debate was – how many repos do you have – one for the whole company at one extreme or one for each small component? A big spectrum. Git is proven to work extremely well for a very large number of modest repos so we spent a bunch of time exploring what it would take to factor our large codebases into lots of tenable repos. Hmm. Ever worked in a huge code base for 20 years? Ever tried to go back afterwards and decompose it into small repos? You can guess what we discovered. The code is very hard to decompose. The cost would be very high. The risk from that level of churn would be enormous. And, we really do have scenarios where a single engineer needs to make sweeping changes across a very large swath of code. Trying to coordinate that across hundreds of repos would be very problematic.
After much hand wringing we decided our strategy needed to be “the right number of repos based on the character of the code”. Some code is separable (like microservices) and is ideal for isolated repos. Some code is not (like Windows core) and needs to be treated like a single repo.
Re: The largest Git repo
#17Archive Team is making a distributed backup of the Internet Archive. http://archiveteam.org/index.php?title=INTERNETARCHIVE.BAK Currently the method getting the most attention is to put the data into git-annex repos, and then have clients just download as many files as they have storage space for. But because of limitations with git, each repo can only handle about 100,000 files even if they are not "hydrated". http://git-annex.branchable.com/design/iabackup/ If git performance were improved for files that have not been modified, this restriction could be lifted and the manual work of dividing collections up into repos could be a lot lower.
Edit: If you're interested in helping out, e.g. porting the client to Windows, stop by the IRC channel #internetarchive.bak on efnet.
Re: The largest Git repo
#18I wonder why Windows is a single repository - Why not split it in separate modules? I can imagine tools like Explorer, Internet Explorer/Edge, Notepad, Wordpad, Paint, etc. all can stay in its own repository. I can imagine you can even further split things up, like a kernel, a group of standard drivers, etc. If that is not already the case (separate repos, that is), are the plans to separate it in the future?
It'd probably be hell on earth for the developers/engineers if it was more than one repo.
I'm sure the build process is non-trivial and with 4,000 people working on it, the amount of updates it gets daily is probably insane. Any one person across teams trying to keep this all straight would surely fail.
Having done a lot of small git repos, I'm a big fan of one huge repo. It makes life easier for everyone, especially your QA team as it's less for them to worry about. In the future, anywhere I'm the technical lead I'm gonna push for one big repo. No submodules either. They're a big pain in the ass too.
Re: The largest Git repo
#19I don't know much about Windows development, but I'm sure the system is modularized in some way. Why wouldn't you want to break up the project into multiple repos for different parts of the system? That would let you work on and test each part independent of the rest. Each part should be able to function on its own, right? Of course some engineers would need to build and test the entire OS as a whole, but I'd wager that (for example) the team working on visual design of the settings app doesn't need to have the source code of how the login screen verifies passwords.
Clearly Microsoft's process works well enough for them, so I wonder what benefits there are to using the monolithic repo choice over many smaller repos.
Re: The largest Git repo
#20Non-dev here, but does this replace/overlap with TFS? What was the driver to adopt Git?