Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The largest Git repo

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

101–110 of 416 posts

Re: The largest Git repo

#101

Post by wilatmsft »

mschuster91 wrote: post #78
taylorlafrinere wrote: post #60

Earlier quoted context omitted.

Most of that 300GB isn't text. There are test assets, images, videos, built binaries, vhd's, etc. Also, I should be clear that that 300GB is just at tip (no history). We can debate about whether or not those things should be checked into the repo but they are there now.

Whoa. 300GB with a shallow clone?! What size does the whole repo use on the server side?

The pack file size for a full clone is 187GB. The 300GB is the working directory. We did not import the history of the code base, so the current repo only has about 5 months of history. As others have called out, there are a lot of assets in the repo that don't compress.

Re: The largest Git repo

#102

Post by StavrosK »

sp332 wrote: post #17

Archive Team is making a distributed backup of the Internet Archive. http://archiveteam.org/index.php?title=INTERNETARCHIVE.BAK Currently the method getting the most attention is to put the data into git-annex repos, and then have clients just download as many files as they have storage space for. But because of limitations with git, each repo can only handle about 100,000 files even if they are not "hydrated". http:…

Wouldn't IPFS be much, much more suitable for this purpose?

Re: The largest Git repo

#103

Post by smilekzs »

maxxxxx wrote: post #73
Kenji wrote: post #30

I am kinda surprised that Microsoft doesn't use tfs - after all, it's their own version control system. But then again, we use tfs at work and not a day goes by on which I do not long for git.

Here is my cynical view: From what I know they have a history of not using their own tools. They didn't use SourceSafe, but Perforce. Then they made an effort to switch to TFS, realized that it sucks and moved on to git. You can see the same pattern in Windows desktop apps. They didn't use MFC for themselves, didn't use Winforms, used WPF only a little.

UWP everywhere now though.

Re: The largest Git repo

#104

Post by planteen »

cosinetau wrote: post #99
vtbassmatt wrote: post #25

Earlier quoted context omitted.

Really good question. Actually, splitting Windows up was the first approach we investigated. Full details here: https://www.visualstudio.com/learn/gvfs-design-history/ Summary: - Complicates daily life for every engineer - Becomes hard to make cross-cutting changes - Complicates releasing the product - There's a still a core of "stuff" that's not easy to tease apart, so at least one of the smaller Windows repos would…

This was a very interesting point. It sounds like there are some serious architectural limitations on Windows, and this makes me believe the same might be true for the NT kernel, and that MS might not be interested in doing heavy refactoring of it. I'm not a frequent Windows user, or a Windows dev at all. Does anyone know of any consequences that MS's decision might mean, if this hypothesis is true?

Google also uses a single giant repo...

Re: The largest Git repo

#105

Post by pvg »

Jyaif wrote: post #51
howinator wrote: post #14

Earlier quoted context omitted.

So, this is actually pretty common. I know that both Google and Facebook use a huge mono-repo for literally everything (except I think Facebook split out their Android code into a separate repo?). So, all of Facebook's and Google's code for front-end, back-end, tools, infrastructure, literally everything, lives in one repo. It's news to me that Windows decided to go that route too. Personally, I think submodules and…

pros to big repo: -dont have to spend time to think about defining interfaces cons: -history is full of crap you dont care about -tests take forever to run -tooling breaks down completely, though thanks to MS the limit was increased seriously

-dont have to spend time to think about defining interfaces

That seems like a design and policy choice, orthogonal to repos.

Re: The largest Git repo

#106

Post by eyq1 »

maxxxxx wrote: post #73
Kenji wrote: post #30

I am kinda surprised that Microsoft doesn't use tfs - after all, it's their own version control system. But then again, we use tfs at work and not a day goes by on which I do not long for git.

Here is my cynical view: From what I know they have a history of not using their own tools. They didn't use SourceSafe, but Perforce. Then they made an effort to switch to TFS, realized that it sucks and moved on to git. You can see the same pattern in Windows desktop apps. They didn't use MFC for themselves, didn't use Winforms, used WPF only a little.

If they didn't use MFC, WinForms and WPF what exactly were they using? Flash?

Re: The largest Git repo

#107

Post by breck »

hyperrail wrote: post #29
breck wrote: post #11

This is so awesome. Brilliant move MS! In addition to enabling Windows engineers to be significantly more productive (eventually), it will go a long way to enabling engineers in other departments to contribute to Windows. For example, I used to work in the Azure org and once noticed a relatively simple missing feature in Windows. I filed a bug and was in contact with a PM who suggested if I wanted I could work on add…

I'm not so sure moving to Git alone would have helped your case. Getting an enlistment is only a small part of contributing to Windows.

Agreed, but probably one of the top 5 road blocks.

Re: The largest Git repo

#108

Post by vtbassmatt »

maxxxxx wrote: post #73
Kenji wrote: post #30

I am kinda surprised that Microsoft doesn't use tfs - after all, it's their own version control system. But then again, we use tfs at work and not a day goes by on which I do not long for git.

Here is my cynical view: From what I know they have a history of not using their own tools. They didn't use SourceSafe, but Perforce. Then they made an effort to switch to TFS, realized that it sucks and moved on to git. You can see the same pattern in Windows desktop apps. They didn't use MFC for themselves, didn't use Winforms, used WPF only a little.

To be somewhat less cynical, VSS and TFVC were not intended to scale to the size of Windows's codebase, thus they weren't used. And instead of inventing our own thing this time around, we went with (and make contributions back to) the de facto standard.

When I was in Xbox, we had a lot of things in TFVC: all the services code, most of the console and Windows apps, and many of the tools. Only the Windows-related bits were in SD.

Re: The largest Git repo

#110

Post by asp2insp »

jonathanpoulter wrote: post #94
howinator wrote: post #14

Earlier quoted context omitted.

So, this is actually pretty common. I know that both Google and Facebook use a huge mono-repo for literally everything (except I think Facebook split out their Android code into a separate repo?). So, all of Facebook's and Google's code for front-end, back-end, tools, infrastructure, literally everything, lives in one repo. It's news to me that Windows decided to go that route too. Personally, I think submodules and…

How does a mono-repo company manage open sourcing a single part of their infrastructure if things are in one large repo? For example, if everything lived in one repo, how does Facebook manage open sourcing React? Or if I personally wanted to switch to one private mono-repo, how would I share individual projects easily?

We have an internal tool that allows us to mirror subdirectories of our monorepo into individual github repositories, and another tool that helps us sync our internal source code review tool with PRs etc.
Post reply on HN