Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The origin story of merge queues

mergify.com

Post reply on HN
Print view View original

21–30 of 34 posts

Re: The origin story of merge queues

#21

Post by tantalor »

Despite the article, I'm not quite sure I understand exactly what this entails.

Mainly I'm confused what this check is gating. Based on the article it's hard to tell what they mean.

1. Code changes that may conflict with each other in the repo, in the sense of a merge conflict.

2. Regressions (test failure, build breakage) caused by recently checked-in code.

3. Preparing and verifying a new release prior to deployment.

4. Monitoring/canary a release candidate with real users.

In my mind, these are all very different things, but the article seems to mix them up.

Re: The origin story of merge queues

#22

Post by kevincox »

kccqzy wrote: post #18

> The motivation was to avoid "merge skew," where changes appear compatible when reviewed in isolation but break once merged into an updated main. My opinion is that this situation of a merge skew happens rarely enough not to be a major problem. And personally, I think instead of the merge queues described in the article, it would be overall more beneficial to invest in tooling to automatically revert broken commits…

> it is overly strict to require your main branch to be fast forwarded

But merge queues (talking in general, IDK about the mergify.com product specifically) don't require fast-forwarding as far as the developer is aware. In the simplest case it looks like merging (non-fast-forward) to a temporary branch, then only updating the main branch after tests pass. This is very similar to your auto-revert except the main branch is never broken, so no wasted developer time and confusion when they pull a bad commit to start their PR.

IMHO it is a real shame that all CI doesn't work like this. It should be the default. Just this basic delay and auto-revert is already a nice boost to developer productivity. Not to mention that blocking a merge in the original PR is much less confusing than reverting and requiring a fresh PR to make the change. It adds basically no complexity other than the fact that our tools aren't set up to work this way by default which ends up requiring extra tools which are not as well integrated.

On top of this you can add batching which can be incredibly useful when your CI is slow (including things like deploying to a staging environment and letting it soak for a few hours) which isn't feasible to do per-PR even for fairly small teams.

Re: The origin story of merge queues

#24

Post by rufo »

jes5199 wrote: post #9

I don't know what it's like now, but GitHub's internal merge queue circa 2017 was a nightmare . Every PR required you to set aside a full day of babysitting to get it getting merged/deployed - there were too many nondeterministic steps. You'd join the queue, and then you'd have to wait for like 12 other people in front of you who would each spend up to a couple hours trying to get their merge branch to go green so it…

It got a little bit better, first with trains (bundling together PRs so they weren't going out one at a time), and then the merge queue started automating most of the testing and fitting together PRs into bundles that could go out together. But by the time I left GH last year it had devolved into roughly the same amount of hassle; I had multiple days where I could queue a PR for deploy mid-morning and not have the deploy containing it go out until dinnertime, and I'd need to keep an eye out in Slack in case merge or test conflicts arose.

Re: The origin story of merge queues

#25

Post by jes5199 »

rufo wrote: post #24
jes5199 wrote: post #9

I don't know what it's like now, but GitHub's internal merge queue circa 2017 was a nightmare . Every PR required you to set aside a full day of babysitting to get it getting merged/deployed - there were too many nondeterministic steps. You'd join the queue, and then you'd have to wait for like 12 other people in front of you who would each spend up to a couple hours trying to get their merge branch to go green so it…

It got a little bit better, first with trains (bundling together PRs so they weren't going out one at a time), and then the merge queue started automating most of the testing and fitting together PRs into bundles that could go out together. But by the time I left GH last year it had devolved into roughly the same amount of hassle; I had multiple days where I could queue a PR for deploy mid-morning and not have the de…

well, that’s horrifying! I appreciate the update though

Re: The origin story of merge queues

#27

Post by w10-1 »

Roguelazer wrote: post #8

I think this is ignoring a lot of prior art. Our deploys at Yelp in roughly 2010 worked this way -- you flagged a branch as ready to land, a system (`pushmaster` aka `pushhamster`) verified that it passed tests and then did an octopus merge of a bunch of branches, verified that that passed tests, deployed it, and then landed the whole thing to master after it was happy on staging. And this wasn't novel at Yelp; we in…

> we inherited the practice from PayPal

Paypal got it from eBay, which in 2000's was rolling out 20M LOC worldwide every week or two on "release trains". There, a small team of kernel engineers rotated doing the merging -- two weeks of clearcase hell when it was your turn.

And, since eBay wrote their own developer tools, you'd have to deploy different tooling depending on the branch you were on. But because of their custom tooling, if there was a problem in the UI, in debug mode you could select an element in the browser UI and navigate to the java class in a particular component and branch that produced that element.

Re: The origin story of merge queues

#28

Post by pcthrowaway »

fosterfriends wrote: post #26

Pretty direct rip of this blog post I wrote a while back: https://graphite.dev/blog/bors-google-tap-merge-queue

I've only skimmed both, but I'm under the impression that your article lays out the problem much clearly. For example, your article makes it much clearer than OP's that merge skew refers to the situation where two PRs submitted closer together than the test suite takes to run may indicate the tests pass for both branches, when both merged together would not pass.

Another user suggested that the OP article may be AI-generated or AI-assisted; I'm not confident one way or another, but it does have me questioning whether HN has any AI detection mechanisms (though I'm not sure how effective these will be as AI keeps evolving)

Re: The origin story of merge queues

#29

Post by sandstrom »

Does anyone working at Github know why 'semi-linear' merge isn't supported as a merge strategy in the merge queue (and regular PRs)?

I Gitlab and Azure DevOps (also owned by MS) supports it, and even talked to an employee now working at Github, that implemented this in Azure DevOps.

More background: https://github.com/orgs/community/discussions/14863

With a semi-linear merge strategy, you rebase (without --fast-forward) before merging, so the history ends up looking like this:

    *   c8be0c632 Merge pull request #1538 from my-org/api-error-logging
    |\  
    | * 80ecc4985 Fix security warning, bump nokogiri
    | * 750613638 Log and respond with more detailed validation errors in the API
    | * 0165d6812 Log code and details when rendering an API error response.
    | * 1d4daab48 Refactor email validation result to include a descriptive message
    | * 635214092 Move media_type logging into context_logging
    |/  
    *   1cccd4412 Merge pull request #1539 from my-org/profile-clarify
    |\  
    | * 87b342a32 Rename profile default to migration target
    | * 2515c1e59 Fix disallow removing last profile in company
    |/  
    *   b8f3f1658 Merge pull request #1540 from my-org/customer
    |\  
    | * 064b31232 Add customer-specific taxed allowance reduction
    |/  
    *   3cf449f94 Merge pull request #1528 from my-org/console-logging
    |\  
    | * 99657f212 Don't log to rails console in production
    |/  
    *   8c72e7f19 Merge pull request #1527 from my-org/gemfile
It makes it easy to look at the Git history both at the 'PR level' kind of like a change log (`git log --merges --decorate --oneline`) or dig down into each PR to see all commits.

Re: The origin story of merge queues

#30

Post by jd__ »

sandstrom wrote: post #29

Does anyone working at Github know why 'semi-linear' merge isn't supported as a merge strategy in the merge queue (and regular PRs)? I Gitlab and Azure DevOps (also owned by MS) supports it, and even talked to an employee now working at Github, that implemented this in Azure DevOps. More background: https://github.com/orgs/community/discussions/14863 With a semi-linear merge strategy, you rebase (without --fast-forwa…

FWIW, Mergify's Merge Queue supports this as long as you run it without batching or parallel checks. We expose a config option for semi-linear history while still keeping CI green.
Post reply on HN