Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The largest Git repo

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

131–140 of 416 posts

Re: The largest Git repo

#131

Post by monksy »

planteen wrote: post #104
cosinetau wrote: post #99

Earlier quoted context omitted.

This was a very interesting point. It sounds like there are some serious architectural limitations on Windows, and this makes me believe the same might be true for the NT kernel, and that MS might not be interested in doing heavy refactoring of it. I'm not a frequent Windows user, or a Windows dev at all. Does anyone know of any consequences that MS's decision might mean, if this hypothesis is true?

Google also uses a single giant repo...

Why is that a "good reason" to do it?

Re: The largest Git repo

#132

Post by hyperrail »

MikusR wrote: post #98
vtbassmatt wrote: post #15

Earlier quoted context omitted.

1. Saeed is writing a really nice series of articles starting here: https://www.visualstudio.com/learn/git-at-scale/ In the first one, he lays out how we think about small/medium/large repos. Summary: size at tip, size of history, file count at tip, number of refs, and number of developers. 2. Fairly confident, at least as far as usable repos go. Given how unusable the Windows repo is without GVFS and the other thing…

Is it possible to checkout (if not build) something like Windows 3.11 or NT 4?

As far as I can recall, this is not possible using Windows source control, as its history only goes back to the lifecycle of Windows XP (when the source control tool prior to GVFS was adopted).

Microsoft does have an internal Source Code Archive, which does the moral equivalent of storing source code and binary artifacts for released software in a underground bunker. I used to have a bit of fun searching the NT 3.5 sources as taken from the Source Code Archive...

Re: The largest Git repo

#133

Post by euyyn »

jonathanpoulter wrote: post #94
howinator wrote: post #14

Earlier quoted context omitted.

So, this is actually pretty common. I know that both Google and Facebook use a huge mono-repo for literally everything (except I think Facebook split out their Android code into a separate repo?). So, all of Facebook's and Google's code for front-end, back-end, tools, infrastructure, literally everything, lives in one repo. It's news to me that Windows decided to go that route too. Personally, I think submodules and…

How does a mono-repo company manage open sourcing a single part of their infrastructure if things are in one large repo? For example, if everything lived in one repo, how does Facebook manage open sourcing React? Or if I personally wanted to switch to one private mono-repo, how would I share individual projects easily?

So open sourcing can mean three different things:

The bad one is just dumping snapshots of the code into a public repo every so often. You need to make sure your dependencies are open source, have tools that rewrite your code and build files accordingly, and put them in a staging directory for publishing.

The good one is developing that part publicly, and importing it periodically into your internal monorepo with the same (or similar) process to the one you use for importing any other third-party library.

There's also a hybrid approach which is to try and let internal developers use the internal tooling against the internal version of the code, and also external developers, with external tooling, against the public version. That one's harder, and you need a tool that does bidirectional syncing of change requests, commits, and issues.

Re: The largest Git repo

#134

Post by uiri »

komali2 wrote: post #63
arjie wrote: post #27

Earlier quoted context omitted.

Why split it into separate modules? Seeing that big companies are very successful with monorepos (Google, Facebook, Microsoft), has made me reconsider if repository modularization is actually worth it. There are a host of advantages to not modularizing repos, and I'm beginning to believe they outweigh those of modular repos.

Wait so like, at google, Inbox and Android are in the same repo as ChromeOS and oh I dunno, Google Search? That doesn't make any sense at all...

Yes, Inbox, Maps, Search, etc. are all in one repo in a specialized version control system.

Android, Chromium, and Linux (among others, I'm sure) are different in that they use git for version control so they are in their own separate repos.

Re: The largest Git repo

#135

Post by tonmoy »

sriram_sun wrote: post #127
garyclarke27 wrote: post #45

Linus must be very proud - his favourite software Windows - now depends on GIT.

Well how the tables have turned! Only about 3 yrs back I was having a conversation with a Microsoft engineer about them evaluating a closed source Hadoop clone because Microsoft policy prohibited them from using open source.

Using open source in their product is different from using it for development. I wonder which one you talking about with that employee

Re: The largest Git repo

#136

Post by emodendroket »

lloeki wrote: post #125

Coming from the days of CVS and SVN, git was a freaking miracle in terms of performance, so I have to just put things into perspective here when the topmost issue of git is performance . It's just a testament how huge are the codebases we're dealing with (Windows over there, but also Android, and surely countless others), the staggering amount of code we're wrangling around these days and the level of collaboration i…

At the risk of sounding like a downer, this was a migration of an existing codebase.

Re: The largest Git repo

#137

Post by cosinetau »

Analemma_ wrote: post #112
cosinetau wrote: post #99

Earlier quoted context omitted.

This was a very interesting point. It sounds like there are some serious architectural limitations on Windows, and this makes me believe the same might be true for the NT kernel, and that MS might not be interested in doing heavy refactoring of it. I'm not a frequent Windows user, or a Windows dev at all. Does anyone know of any consequences that MS's decision might mean, if this hypothesis is true?

It seems presumptive to say that "If you can't use multiple repos, your architecture must be bad". I could just as easily counter with, "If you're using multiple repos, it must mean you have an unnecessarily complex and fragile microservice architecture"

I'm sorry that was implied. I simply want better insight into the kernel from people who have experience developing large kernels, and the decisions that are made as a consequence of architectural choices.

I feel like that those questions are valid, and are important in this field, not just kernel development. As someone who desires to continue learning, I will not yield to your counter.

Re: The largest Git repo

#138

Post by bitJericho »

TylerE wrote: post #72
komali2 wrote: post #63

Earlier quoted context omitted.

Wait so like, at google, Inbox and Android are in the same repo as ChromeOS and oh I dunno, Google Search? That doesn't make any sense at all...

It makes total sense when the expectation is that any engineer in the company can build any part of the stack at any time with a minimum of drama.

This just blew my mind. I'm gonna go home and see about combining all my projects. That seems very useful!

Re: The largest Git repo

#140

Post by ansible »

zanny wrote: post #69
kk1274 wrote: post #53

300GB of code WOW! Just for comparison the entire English Wikipedia dump including all media is about 50-60GB. What are you guys doing there and how large do you see this growing?

A lot of that is probably static art assets. A lot of big corporate projects are guilty of just throwing everything in one tree like this where for some bizarre reason you are checking out jpgs and video files. It is part of the reason why the Chrome or Unreal Engine repos are preposterously large (hundreds of megs to gigabytes each).

And there was an article a few weeks back about how certain content creation tools (Adobe) stuff in lots of metadata into these assets by default.

Some of the MS teams were good about stripping those out for release, others, didn't seem to notice.

Post reply on HN