Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The largest Git repo

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

31–40 of 416 posts

Re: The largest Git repo

#31

Post by sseveran »

Willamin wrote: post #19

I don't know much about Windows development, but I'm sure the system is modularized in some way. Why wouldn't you want to break up the project into multiple repos for different parts of the system? That would let you work on and test each part independent of the rest. Each part should be able to function on its own, right? Of course some engineers would need to build and test the entire OS as a whole, but I'd wager t…

If you have many small repos for a large interconnected project you simply move the complexity of managing a commit that requires changes into another tool that can manage cross repo changes and dependencies. With a single repo you can change something and build it, fix any breaks and then commit it with just source source control and build system. The many small repos has in my experience been driven by either poor processes or tooling limitations.

Re: The largest Git repo

#32

Post by saeednoursalehi »

rajathagasthya wrote: post #4
vtbassmatt wrote: post #2

A handful of us from the product team are around for a few hours to discuss if you're interested.

Very cool blog! As I understand, you dynamically fetch a file from the remote git server once for the first time I open the file. Do you do any sort of pre-fetching of files? For example, if a file has an import and uses a few symbols from that file, do you also fetch the imported file beforehand or just fetch it when you access it first time?

We don't currently do that sort of predictive prefetching, but it's a feature we've thought a lot about. For now, users can explicitly call "gvfs prefetch" if they want to, or just allow files to be downloaded on demand.

Re: The largest Git repo

#33

Post by vtbassmatt »

rajathagasthya wrote: post #4
vtbassmatt wrote: post #2

A handful of us from the product team are around for a few hours to discuss if you're interested.

Very cool blog! As I understand, you dynamically fetch a file from the remote git server once for the first time I open the file. Do you do any sort of pre-fetching of files? For example, if a file has an import and uses a few symbols from that file, do you also fetch the imported file beforehand or just fetch it when you access it first time?

For now, we're not that smart and simply fetch what's opened by the filesystem. With the cache servers in place, it's plenty fast. We do also have an optional prefetch to grab all the contents (at tip) for a folder or set of folders.

Re: The largest Git repo

#34

Post by monksy »

rubyn00bie wrote: post #18
tobyhinloopen wrote: post #5

I wonder why Windows is a single repository - Why not split it in separate modules? I can imagine tools like Explorer, Internet Explorer/Edge, Notepad, Wordpad, Paint, etc. all can stay in its own repository. I can imagine you can even further split things up, like a kernel, a group of standard drivers, etc. If that is not already the case (separate repos, that is), are the plans to separate it in the future?

It'd probably be hell on earth for the developers/engineers if it was more than one repo. I'm sure the build process is non-trivial and with 4,000 people working on it, the amount of updates it gets daily is probably insane. Any one person across teams trying to keep this all straight would surely fail. Having done a lot of small git repos, I'm a big fan of one huge repo. It makes life easier for everyone, especially…

Most of the work that is the issue would be solved by meta repos and tools to keep components up to date and integrated upward.

Re: The largest Git repo

#38

Post by vtbassmatt »

criddell wrote: post #28

At one point, I believe Microsoft was using a modified Perforce server for source code. Is that completely gone now?

Most of the large Source Depot users have moved to Git or, like Windows, are in the process of moving to Git. Legacy stuff will probably live on in SD for a long time, possibly forever, for maintenance work.

Re: The largest Git repo

#39

Post by UK-AL »

Kenji wrote: post #30

I am kinda surprised that Microsoft doesn't use tfs - after all, it's their own version control system. But then again, we use tfs at work and not a day goes by on which I do not long for git.

All the modern development on VSTS is focused on git as well.

Re: The largest Git repo

#40

Post by bluejekyll »

saeednoursalehi wrote: post #23
bluejekyll wrote: post #8

Earlier quoted context omitted.

I'm currently investigating using GitLFS for a large repo that has many binary and other large artifacts. I'm curious, did you experiment with LFS for prior to building GitVFS? Also, I know that there is an (somewhat) active effort to port GitVFS to Linux, do you know if any of the Git vendors (GitLab and/or GitHub) are planning to support GitVFS in their enterprise products?

Yes we did evaluate LFS. The thing about LFS is that while it does help reduce the clone size, it doesn't reduce the number of files in the repo at all. The biggest bottleneck when working with a repo of this size is that so many of your local git operations are linear on the number of files. One of the main values of GVFS is that it allows Git to only consider the files you're actually working with, not all 3M+ file…

> One of the main values of GVFS is that it allows Git to only consider the files you're actually working with, not all 3M+ files in the repo.

That is an excellent point. Thanks!

Post reply on HN