Ask HN: How are pull requests integrated in a repo with high commit frequency?
51–60 of 71 posts
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#52Earlier quoted context omitted.
I'd be curious to know what you find limited in off-the-shelf solutions? (disclaimer: I'm from the Mergify team )
One question and one comment. Question: You list Uber (my employer) as a user in your homepage, but searching internally, the only search result for mergify is a mergify.yml in some random debian module for lua, so it doesn't seem like we actually use it for anything (and we built an in-house solution for the repos w/ serious commit traffic). I'm curious how you determine who your corporate users are? Comment: we end…
We know Uber has some great internal tooling (there's even a paper published on their merge queue: https://eng.uber.com/research/keeping-master-green-at-scale/). We had various Uber engineers using Mergify on open source repositories those last years. I didn't check recently if that was still the case, I'll make sure our lis is not too much outdated.
I definitely agree with your view on how pipelines can be different. I think none of our customers has something that is exactly the same. However, many of them don't have the workforce to build in house or even to optimize as far as you would do in a (very) large company.
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#53Earlier quoted context omitted.
One question and one comment. Question: You list Uber (my employer) as a user in your homepage, but searching internally, the only search result for mergify is a mergify.yml in some random debian module for lua, so it doesn't seem like we actually use it for anything (and we built an in-house solution for the repos w/ serious commit traffic). I'm curious how you determine who your corporate users are? Comment: we end…
We know Uber has some great internal tooling (there's even a paper published on their merge queue: https://eng.uber.com/research/keeping-master-green-at-scale/ ). We had various Uber engineers using Mergify on open source repositories those last years. I didn't check recently if that was still the case, I'll make sure our lis is not too much outdated. I definitely agree with your view on how pipelines can be differen…
Ah, good point, our internal search tool only indexes internal code, so that makes sense, thanks!
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#54Earlier quoted context omitted.
Another question I got on this. What do you do with merge conflicts between the branches? Do you then provide the system with patches for the merge conflicts somehow, so that it can resolve them as needed? E.g. when merging A, B and C. C might have no conflict with master or A but it has a conflict with B. Now for the queuing to work A -> B -> C it will have to know somehow how to patch C to fit on top of B. Maybe B…
The developer whose task branch had merge conflicts was responsible for resolving the merge conflicts, and doing sidebars as needed where a discussion of the approach was warranted. They would resolve those conflicts in their branch before the PR was submitted. If the conflicts happened between submission and approval then the conflicts would still be resolved on the task branch before merge.
>The developer whose task branch had merge conflicts was responsible for resolving the merge conflicts
But that's exactly my point. The first PR to merge cleanly will determine whether the next PR causes a merge conflict or not. At the time of merging, none of these PRs had a merge conflict.
We often have 10 conflicts between PR admission and approval.
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#55Our full acceptance test run in 3 hrs (on build servers with 64 cores and 3xx GB memory) and we have PRs coming up every hour or so. We run the acceptance tests for all PRs and also the main branch after each change. So a given PR needs to be 'green', and if we merge the PR then we run the tests on the integration branch too. If the integration branchs gets broken somehow, then all merges stop -- we don't merge any f…
1000 CPU for 30 developers sounds enormous. What kind of software are you working on?
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#56What we do at my job is just merge it anyway, but when failures happen, we work backwards to find faulting commit(s). This is also necessary because certain tests take too long to block PRs for. For people creating new branches, we also keep track of the last known good commit since this almost always lags compared to the latest commit.
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#57Earlier quoted context omitted.
The developer whose task branch had merge conflicts was responsible for resolving the merge conflicts, and doing sidebars as needed where a discussion of the approach was warranted. They would resolve those conflicts in their branch before the PR was submitted. If the conflicts happened between submission and approval then the conflicts would still be resolved on the task branch before merge.
>The developer whose task branch had merge conflicts was responsible for resolving the merge conflicts But that's exactly my point. The first PR to merge cleanly will determine whether the next PR causes a merge conflict or not. At the time of merging, none of these PRs had a merge conflict. We often have 10 conflicts between PR admission and approval.
> We often have 10 conflicts between PR admission and approval.
I don't think a devops solution can remove those conflicts. At least not all of them.
If you have devs in the same hot-spots of the code, you're going to get conflicts. Maybe refactoring can address some of this core issue.
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#58Earlier quoted context omitted.
Does bors stand for something? what does the ng stand for? (I wasn't able to find out while googling.)
-ng is a suffix sometimes used for "reboots" of abandoned or poorly maintained software; I believe it stands for "next generation".
Thanks, I assumed that's what it meant. It's not terribly common, but there are a few packages using this naming convention: aircrack-ng, bwm-ng, crosstool-ng, libcap-ng, lincity-ng, minizip-ng, proxychains-ng, raxml-ng, recon-ng, stress-ng, zlib-ng, macfusion-ng, shadowsocksx-ng
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#59The technology you're looking for goes by many names, "submit queue", "merge queue", "merge train", etc. In a nutshell, the idea is to keep track of currently running jobs, then any time a new commit enters the queue, you merge all the running commits into that and test that as a bundle. If merging fails, bail out. If that bundle job fails, bail out. If one of the previously running commits fail, you bail out of the…
That's amazing, I haven't heard of these terms before. I looked through some of the solutions posted here (GitLab Merge Trains, mergify, ZUUL Gate) and they are awesome.
I just felt that merging into main without having 100% confidence that acceptance tests would pass means that true continuous delivery isn't achieved. But with these tools, we can make sure that our software is always in a releasable state (according to our test suite). I learned a lot today.
Re: Ask HN: How are pull requests integrated in a repo with high commit frequency?
#60Earlier quoted context omitted.
does bors limit the commit frequency / PR merge frequency to one merge every test runtime (5 minutes in OP), or does it do something more complex and speculate / run some stuff in parallel? OP’s numbers are 120 commits per hour and 5 minute test run, I’m wondering if adopting bors would necessarily reduce that to 12 commits per hour.
It tries to build all queued changes together at first. So if you don't have any conflicts, there is no throughput limit.
What about if something in the middle causes a failure? Does it have a git-bisect-like workflow to figure out the bad change?