The largest Git repo • Hacker News Board

Re: The largest Git repo

#371

Post by zeckalpha » Thu, May 25, 2017, 2:54 PM UTC

rdubz wrote: post #369

cryptonector wrote: post #326

Earlier quoted context omitted.

Well, git rerere helps here, though, honestly, this never happens to me even when I have 20 commits. Also, this is what you want, as it makes your commits easier to understand by others. Otherwise, with thousands of developers your merge graph is going to be a pile of incomprehensible spaghetti, and good luck cherry-picking commits into old release patch branches! Ah, right, that's another reason to rebase: because y…

rebase : centralized repo :: merge : decentralized repo rebase : linked-list :: merge : DAG If the work/repo is truly distributed and there isn't a single permanently-authoritative repo, a "clean, linear" history is nonsensical to even try to reason about. In all cases it is a crutch: useful (and nice, and sufficient!) in simple settings, but restricting/misleading in more complex ones (to the point of causing many d…

Decentralization at scale can result in a linear chain, too.

Re: The largest Git repo

#372

Post by Luuseens » Thu, May 25, 2017, 3:07 PM UTC

vtbassmatt wrote: post #172

js2 wrote: post #145

Windows, because of the size of the team and the nature of the work, often has VERY large merges across branches (10,000’s of changes with 1,000’s of conflicts). At a former startup, our product was built on Chromium. As the build/release engineer, one of my daily responsibilities was merging Chromium's changes with ours. Just performing the merge and conflict resolution was anywhere from 5 minutes to an hour of my t…

Windows developed an extension that lets them do conflict resolution in the web. We have a server-side API that it calls into, but the extension isn't fundamentally different from using BeyondCompare or $YOUR_FAVORTE_MERGETOOL.

Is this CodeFlow you're talking about?

Re: The largest Git repo

#373

Post by WorldMaker » Thu, May 25, 2017, 3:08 PM UTC

wilatmsft wrote: post #128

ATsch wrote: post #91

Earlier quoted context omitted.

What are your thoughts on implementing something more general like linux's FUSE[1] instead? A general Virtual Filesystem driver in Windows could be used for a wide range of things and means you don't just have a single-purpose driver sitting around. [1] https://en.m.wikipedia.org/wiki/Filesystem_in_Userspace

A general FUSE-like API would be very useful to have, but unfortunately it can't meet our performance requirements. The first internal version of GVFS was actually based on a 3rd party driver that looks a lot like FUSE. But because it is general purpose, and requires all IO to context switch from the kernel to user mode, we just couldn't make it fast enough. Remember that our file system isn't going to be used just f…

I'm curious what the cross-over between GvFlt may be and the return of virtual files for OneDrive in the Fall Creators Update? Is the work being coordinated between the efforts?

From your description it sounds like there could be usefulness in coordinating such efforts.

Re: The largest Git repo

#374

Post by [deleted] » Thu, May 25, 2017, 3:19 PM UTC

cryptonector wrote: post #329

emmelaich wrote: post #298

Earlier quoted context omitted.

Are you saying that you use Git instead of Mercurial these days? Not necessarily implied by you; just checking.

Me personally? Yes, I use git whenever I can. I still have to use Mercurial for some things. I don't know what Oracle does nowadays with the gates that make up Solaris. My guess is that they still have a hodge podge, with some gates using git, some Mercurial, and some Teamware still. But that's just a guess. For all I know they may have done what Microsoft did and gone with a single repo for the whole thing.

[deleted]

Re: The largest Git repo

#375

Post by WorldMaker » Thu, May 25, 2017, 3:30 PM UTC

maxxxxx wrote: post #129

smilekzs wrote: post #103

Earlier quoted context omitted.

UWP everywhere now though.

Is that true? Which larger app is written in UWP? Something like Office, Skype or Visual Studio.

The preferred Windows client for Skype has been UWP for a while now (though up until recently it was still often referred to as "Skype Preview").

Re: The largest Git repo

#376

Post by ubolonton_ » Thu, May 25, 2017, 3:35 PM UTC

richardwhiuk wrote: post #360

cryptonector wrote: post #194

At Sun Microsystems, Inc., (RIP) we have many "gates" (repos) that made up Solaris. Cross-gate development was somewhat more involved, but still not bad. Basically: you installed the latest build of all of Solaris, then updated the bits from your clones of the gates in question. Still, a single repo is great if it can scale, and GVFS sounds great! But that's not what I came in to say. I came in to describe the rebase…

Why is a clean linear history desirable? It's not reflective of how the product was built? Is it just for some naive desire of purity?

It's easier to see commits of a branch grouped together in most history viewers. Even though sorting commits topologically can help, most history viewers don't support that option.

When there is an undesired behavior that is hard to reason about, git-bisect can be used to determine the commit that first introduced it. With a normal merge, it will point to the merge commit, because it was the first time the 2 branches interacted. With a rebase, git bisect will point to one of the rebased commits, each of which already interacted with the branch coming before.

Resolving conflicts in a big merge commit vs in small rebased commits is like resolving conflicts in a distributed system by comparing only the final states, vs inspecting at the actual sequences of changes.

Re: The largest Git repo

#377

Post by DoofusOfDeath » Thu, May 25, 2017, 3:42 PM UTC

gear54rus wrote: post #363

DoofusOfDeath wrote: post #70

Earlier quoted context omitted.

"A handful of us from the product team are around for a few hours to discuss if you're interested." Thanks! This is a little off-topic, but why can't Windows 10 users conclusively disable all telemetry? (I consider the question only a little off-topic, because I have the impression that this story is part of an ongoing Microsoft charm-offensive.)

Haha, no answer, as expected. HN got butthurt as well lol.

I knew there was a risk of getting downvoted, but I was surprised that it went to "-4".

Re: The largest Git repo

#378

Post by cryptonector » Thu, May 25, 2017, 4:00 PM UTC

nerdponx wrote: post #334

cryptonector wrote: post #324

Earlier quoted context omitted.

No no, you never force push to "published" branches, e.g., upstream master. What you're doing when you rebase onto the latest upstream is this: you're making your local history the _same_ as the upstream, plus your commits as the latest commits, which means if you push that, then you're NOT rewriting the upstream's history. (In the Sun model one does rewrite project branch history, but one also leaves behind tags, an…

That's what I thought as well. But what happens when you need to rebase the feature branch against master? Won't you have to force push that rebase?

I've described this. Downstreams of the feature branch rebase from their previous feature branch merge base (a tag for which is left behind to make it easy to find it) --onto the new feature branch head.

E.g., here's what the feature branch goes through:

feature$ git tag feature_05

feature$ git fetch origin feature$ git rebase origin/master feature$ git tag feature_06

And here's what a downstream of the feature branch goes through:

downstream$ git fetch feature_remote

downstream$ git rebase --onto feature_remote/feature_06 feature_remote/feature_05

Easy peasy. The key is to make it easy to find the previous merge base and then use git rebase --onto to rebase from the old merge base to the new merge base.

Everybody rebases all the time. Everybody except the true master -- that one [almost] never rebases (at Sun it would happen once in a blue moon).

Re: The largest Git repo

#379

Post by cryptonector » Thu, May 25, 2017, 4:04 PM UTC

rdubz wrote: post #369

cryptonector wrote: post #326

Earlier quoted context omitted.

Well, git rerere helps here, though, honestly, this never happens to me even when I have 20 commits. Also, this is what you want, as it makes your commits easier to understand by others. Otherwise, with thousands of developers your merge graph is going to be a pile of incomprehensible spaghetti, and good luck cherry-picking commits into old release patch branches! Ah, right, that's another reason to rebase: because y…

rebase : centralized repo :: merge : decentralized repo rebase : linked-list :: merge : DAG If the work/repo is truly distributed and there isn't a single permanently-authoritative repo, a "clean, linear" history is nonsensical to even try to reason about. In all cases it is a crutch: useful (and nice, and sufficient!) in simple settings, but restricting/misleading in more complex ones (to the point of causing many d…

As @zeckalpha says, rebase != centralized repo.

You can have a hierarchical repo system (as we did at Sun).

Or you can have multiple hierarchies, contributing different series of rebased patches up the chain in each hierarchy.

Another possibility is that you are not contributing patches upstream but still have multiple upstreams. Even in this case your best bet is as follows: drop your local patches (save them in a branch), merge one of the upstreams, merge the other, re-apply (cherry-pick, rebase) your commits on top of the new merged head. This is nice because it lets you merge just the upstreams first, then your commits, and you're always left in a situation where your commits are easy to ID: they're the ones on top.

Re: The largest Git repo

#380

Post by cryptonector » Thu, May 25, 2017, 4:07 PM UTC

richardwhiuk wrote: post #360

cryptonector wrote: post #194

At Sun Microsystems, Inc., (RIP) we have many "gates" (repos) that made up Solaris. Cross-gate development was somewhat more involved, but still not bad. Basically: you installed the latest build of all of Solaris, then updated the bits from your clones of the gates in question. Still, a single repo is great if it can scale, and GVFS sounds great! But that's not what I came in to say. I came in to describe the rebase…

Why is a clean linear history desirable? It's not reflective of how the product was built? Is it just for some naive desire of purity?

Who cares about how a product's sub-projects were put together? What one should care about is how those sub-projects were put together into a final product. To be sure, the sub-projects' internal history can be archived, but it needn't pollute the upstream's history.

Analogously: who cares how you think? Aside from psychologists and such, that is. We care about what you say, write, do.