Ask HN: How are pull requests integrated in a repo with high commit frequency?

The key is to reduce the number of developers touching a specific section of code, and line up all tickets and hand them over to a single developer (or two).

And yeah, management won't do it. Fred Brooks wrote an entire book about this in 1975 that no one reads today and everyone would certainly ignore if they did read it. Because it tells you the unvarnished truth about the nature of communication and information flow within an organization. Such is the state of things in our industry. Sweet little lies.

We use bors as well. Our repo isn’t particularly active, a few dozen committers at any one time, but it lets us have pretty aggressive checks on master without having to run them separately for each PR before it lands.

Definitely recommended. We run ours on a Heroku instance it’s cheap and almost entirely maintenance free.

Do you really need to solve this problem?

You should definitely make sure that every PR is being tested against a merge commit to current main and not just tested against the code in the PR that may have forked off weeks ago. If the PR has been sitting you may need to recycle the tests to test against current main before merging.

This should take care of most problems, but doesn't guarantee 100% that main is never broken.

I'd argue that isn't likely to be possible, and you need gates in between main and whatever production is and that code needs to be retested before deployment actions happen.

When you do this you should then address how often main actually breaks and what the root cause (or what the accident chain is for people who deeply hate that term). If it is really breaking a lot due to races in different orthogonal commits touching the same sensitive locations (somehow without merge conflicts), I'd argue the correct course of action might be to refactor the code rather than the pipeline, and worrying about the pipeline is the last thing.

And your pipeline probably should break and hold up the tests from time to time, that probably costs less than designing a complex solution which tries to be perfect for the sake of being perfect.

Of course for the FAANG-scale readers it is probably worth it, but most devs aren't actually FAANG-scale, and there'll be some fuzzy line in the middle where it really starts to matter.

But if something breaks once in a blue moon that doesn't necessarily mean you should always fix it, as long as you can always contain the damage. So how often is this really happening that you think you need to fix it?

Patient portal + electronic health records.

1000 CPU cores, not CPUs. :)

We call ours the "stage". Merge conflicts are just a "please wait" signal. The way it works is:

  - the stage is always up-to-date with the target branch (so if it updates, we re-merge all topics on top of the new target branch head)
  - topics should pass CI standalone before being staged
  - the stage is first-come first-served, so "later" topics have to wait
  - updating a topic puts it at the end of the line
  - topics should really go through the stage before landing in the target branch

Conflicts (content or logical) will cause a topic to have to wait until the conflicting topic has landed in the target branch before it can participate in the stage. This usually only affects topics working in the same area.

Our library implementing this: https://gitlab.kitware.com/utils/rust-git-topic-stage

However, given your later metric of 10+ conflicts per topic…I suspect your project is still in the "getting off the ground" phase where a stage is awfully heavy process because there aren't "bright line" distinct sections of the code yet. Or your topics are too big. Hard to say.