Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The largest Git repo

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

221–230 of 416 posts

Re: The largest Git repo

#221

Post by MikeHolman »

js2 wrote: post #145

Windows, because of the size of the team and the nature of the work, often has VERY large merges across branches (10,000’s of changes with 1,000’s of conflicts). At a former startup, our product was built on Chromium. As the build/release engineer, one of my daily responsibilities was merging Chromium's changes with ours. Just performing the merge and conflict resolution was anywhere from 5 minutes to an hour of my t…

Generally, it's the responsibility of whoever made the changes to resolve conflicts (basically, git blame conflicting lines and the people who changed them get notified to resolve conflicts in that file). Distributing the work like this makes the merges more reasonable.

Re: The largest Git repo

#222

Post by wfunction »

cubano wrote: post #211

Isn't this perhaps the greatest validation of Linus's design genius that what was initially a weekend project[0] has successfully scaled to this? They could no longer use their revision control system BitKeeper and no other Source Control Management (SCMs) met their needs for a distributed system. Linus Torvalds, the creator of Linux, took the challenge into his own hands and disappeared over the weekend to emerge th…

> Isn't this perhaps the greatest validation of Linus's design genius that what was initially a weekend project has successfully scaled to this?

I thought the entire point of the article was to show how git didn't scale, and how they're basically rewriting the project and changing it as much as necessary to make it scale. It's not like Linus designed git to scale as O(modified).

Re: The largest Git repo

#223

Post by aanm1988 »

Eyas wrote: post #180
erikpukinskis wrote: post #126

Earlier quoted context omitted.

> - Becomes hard to make cross-cutting changes This does seem like a negative, doesn't it? But it's not. Making it hard to make cross-cutting changes is exactly the point of splitting up a repo. It forces you to slow down, and—knowing that you can only rarely make cross-cutting changes—you have a strong incentive to move module boundaries to where they should be. It puts pressure on you to really, actually separate c…

You say that, but it is very telling that every large company out there (Google and Facebook come to mind) go for the single-repository approach. I'm sure that, when dealing with stakeholder structures where different organizations can depend on different bits and pieces, having multiple repositories with difficulty of making breaking and cross-cutting changes, becomes good. From the view of a single organization whe…

Amazon doesn't.

Re: The largest Git repo

#224

Post by tormeh »

qznc wrote: post #175
CiPHPerCoder wrote: post #3

Earlier quoted context omitted.

Sure, a couple of questions: 1. How do you measure "largeness" of a git repo? 2. How are you confident that you have the largest? 3. How much technical debt does that translate to?

I recently heard a story that someone tried to push a 1TB repo to our university Gitlab which then ran out of disk space. Sure, that might have been not be a usable repo but only an experiment. Still, I would bet against the claim that 300GB is the largest one.

1 TB of code?

Re: The largest Git repo

#225

Post by vec »

Goronmon wrote: post #153
erikpukinskis wrote: post #126

Earlier quoted context omitted.

> - Becomes hard to make cross-cutting changes This does seem like a negative, doesn't it? But it's not. Making it hard to make cross-cutting changes is exactly the point of splitting up a repo. It forces you to slow down, and—knowing that you can only rarely make cross-cutting changes—you have a strong incentive to move module boundaries to where they should be. It puts pressure on you to really, actually separate c…

No, it should be hard to make arbitrary changes. If you change the word "arbitrary" to "necessary" (implying a different bias than the one you went with) then all of a sudden this attitude sounds less helpful. Similarly "easy to limp along with a bad architecture" could be re-written as "easy to work with the existing architecture". At the end of the day, it's about getting work done, not making decisions that are th…

Arguing about purity is only pointless and sanctimonious if the water isn't contaminated. Being unable to break a several hundred megabyte codebase into modules isn't a "tap water vs bottled" purity argument, it's a "lets not all die of cholera" purity argument.

Re: The largest Git repo

#226

Post by nthcolumn »

Linus Torvalds rocks. Windows sucks. Subversion was crap so he just made git instead. git beat out svn, TFS and all that other crap legions of overpaid engineers came up with (or what they didn't get source control???) because unix design philosophy and therein lies the lesson still unlearned for they hath loaded all their bloat into one repo.

Windows. It sucks and it will forever suck because it sucks by design. Bill say 'Thank you Linus - I owe you sooo much because git is way better than the best I could do'

I mean has there ever been worse software ever written than the stuff being loaded into git right now? Awful, awful garbage, creaking and reeking of dirty hacks, different for the sake of it designs, misshapen, bolted together, bloated, willfully annoying, antisocial, phone home, locking-in, full of resolutely, defiant ancient unfixed bugs, butt ugly, horrible UI, full of errors and meaningless error messages, incessant nagging and weird quirks, wtf folders, command line from hell and urgh... note pad ... and oh dear god I almost forgot mmc consoles and visual studio and inconsistent flows, viral load by the galactic shit tonne, complete and utter drivel makes me want to vomit every time I hear that sickening jingle and after all those gazillions of engineering hours an absolute world wonder of fail?

Two guys working out of a garage could do better.

:P (Windows sucks btw)

Re: The largest Git repo

#227

Post by amag »

eridius wrote: post #176
Peaker wrote: post #167

Earlier quoted context omitted.

When you have that many conflicts, it's often due to massive renames, or just code moves. If you use git-mediate[1], you can re-apply those massive changes on the conflicted state, run git-mediate - and the conflicts get resolved. For example: if you have 300 conflicts due to some massive rename, you can type in: git-search-replace.py[2] -f oldGlobalName///newGlobalName git-mediate -d Succcessfully resolved 377 confl…

I just looked at git-mediate and I'm very confused. It appears that all it does is remove the conflict markers from the file after you've already manually fixed the conflict. Except you need to do more work than normal, because you need to apply one of the changes not only to the other branch's version but also to the base. What am I missing here, why would I actually want to use git-mediate when I'm already doing al…

It looks like git-mediate does one more important thing; it checks that the conflict is actually solved. In my experience it's very easy to miss something when manually resolving a conflict and often the choices the merge tools give you are not the ones you want.

Re: The largest Git repo

#228

Post by skybrian »

cubano wrote: post #211

Isn't this perhaps the greatest validation of Linus's design genius that what was initially a weekend project[0] has successfully scaled to this? They could no longer use their revision control system BitKeeper and no other Source Control Management (SCMs) met their needs for a distributed system. Linus Torvalds, the creator of Linux, took the challenge into his own hands and disappeared over the weekend to emerge th…

No, not really. When it scaled enough for the Linux kernel he lost interest in scaling any further, and that's nowhere near what's needed for a monorepo (or a Linux distribution).

And I recall that there are was fairly infamous tech talk at Google where he basically dismissed Google's concerns in an arrogant way.

So I'd say it's mostly a validation of git and Github's popularity with developers, that other people were willing to put in so much work to improve it.

Re: The largest Git repo

#229

Post by zmj »

wilatmsft wrote: post #128
ATsch wrote: post #91

Earlier quoted context omitted.

What are your thoughts on implementing something more general like linux's FUSE[1] instead? A general Virtual Filesystem driver in Windows could be used for a wide range of things and means you don't just have a single-purpose driver sitting around. [1] https://en.m.wikipedia.org/wiki/Filesystem_in_Userspace

A general FUSE-like API would be very useful to have, but unfortunately it can't meet our performance requirements. The first internal version of GVFS was actually based on a 3rd party driver that looks a lot like FUSE. But because it is general purpose, and requires all IO to context switch from the kernel to user mode, we just couldn't make it fast enough. Remember that our file system isn't going to be used just f…

For use cases that fit that limited virtualization, is GvFlt something to consider? Or is that a bad idea?

Re: The largest Git repo

#230

Post by Thaxll »

farresito wrote: post #185
ericfrederich wrote: post #168

Earlier quoted context omitted.

This may be the thing that gets Google to switch. They like having every piece of code in a single repository which Git cannot handle. Now that it is somewhat proven, maybe Google will leverage GVFS on Windows and create a FUSE solution for Linux.

They use mercurial (or were), which is as good as git. In fact, I bet a lot of people at Google are happy to use mercurial instead of git, given git's bad reputation with its command line interface.

They don't use mercurial.
Post reply on HN