Live data from Hacker News

A successful Git branching model (2010)

nvie.com

81–90 of 121 posts

Re: A successful Git branching model (2010)

#81
post #77
post #74

Earlier quoted context omitted.

Interesting. How did you do signed code reviews?

We used PRs with BitBucket for all code reviews. The reviewer(s) had to digitally sign their final approval of the review comments+answers and of the related code changes, if any. The only way to merge a feature branch into `develop` was via the PR + code review process.

Was it something like exporting PR history to a file and then signing (X.509/PGP)? Thanks for answers, it looks like a nice, lightweight auditable system.

Re: A successful Git branching model (2010)

#82
post #11
post #3

The advice here given to avoid using "master" branch for development, and advice to create non-default branch named "develop" (or variations thereof) is quite harmful. If you must have a "release" branch or "stable" branch, ok, go for it, but leave the "master" for developing. Why? Strive to have sane defaults. Frankly, the idea that somebody must check out some extra special branch after cloning repo in order to sta…

> Leave the "master" for developing. IMHO do developing always on topic branches, never on master. This keeps master available for fully-working software, such as for the most-recent successful build using continuous integration, or for always-available deployment, or for external users, etc. To create topic branches, here are git alias commands that you can customize as you like for your git workflows: topic-start =…

Or most importantly for us, master needs to always work, because that's where you start new development on.

You can't expect to fix another bug somewhere else in the codebase if you are starting from a point where the code may not work, or features may only be partially complete.

Re: A successful Git branching model (2010)

#84
post #15

I've got a much nicer branching model- try not to have one. Everyone works off master, and you aren't allowed to check in code that won't run in production. Hide unfinished features behind feature flags, and never merge/push a change that won't pass tests/CI. The chaos of huge feature merges (a key source of bugs I've experienced) is minimized. You deploy fixes hourly, not weekly (or later monthly when it just won't…

Here's the caveats I have with that, although I do trend towards only having a master central branch whenever possible:

1) Small, incremental commits tend to be way more reviewable than big "finished" blobs of work. Local/small branches prevent me from getting blocked on waiting for review, and allows me to continue working, although you don't exactly need a "branching model" for it.

2) Tests/CI are insufficient. For games, you'll want QA hammering on a release for days, weeks, or months with only the most conservative changes applied, in an effort to shake out any remaining weird heisenbugs. To do otherwise invites the specter of failing certification, delaying your release, or burning some pretty bad bugs to hundreds of thousands of physical disks. Do you, effectively, shut down the studio when you've dialed up the stability demands that much - or do you branch so work can continue?

The worst case consequences are no longer quite so bad as burning bugs into unpatchable ROMs, but they can still be pretty bad.

> The time between code being written and a bug being seen can be reduced to minutes and hours, making finding the root cause a breeze.

This is, unfortunately, merely the ideal happy path. Great when it happens, but fails to account for the worst case.

It can take weeks, months, to find the heisenbugs that slip through CI/tests/initial QA. If you have a CI/testing setup that can catch, say, a title exit(3)ing when the charm bar is opened for more than 10 seconds, but only on the main menu and without a debugger attached - without writing a test to catch that extremely specific edge case once you have the benefit of hindsight - I'm begging you to share! Just nailing down the exact repro steps took days.

When you have no relevant callstack (even when you do eventually figure out how to dump the exit(3) event), no relevant logs, the fundamental bug resides in third party code you don't have the source code to, and fully rebuilding takes hours because you're on a large C++ codebase with even larger asset building requirements - well, for me, it took me a week or two to root cause (or more accurately, get fed up to the point that I spent a day manually bisecting version history to track down a totally unrelated and innocuous looking changelist that ultimately made us hit the bug.)

In the weeks of turnaround time fixing such a bug, a sufficiently active dev branch will have acquired another. A release branch won't. Bam, branching model.

3) Plenty of codebase-wide refactoring is hard or impossible to feature-flag.

Re: A successful Git branching model (2010)

#85
post #15

I've got a much nicer branching model- try not to have one. Everyone works off master, and you aren't allowed to check in code that won't run in production. Hide unfinished features behind feature flags, and never merge/push a change that won't pass tests/CI. The chaos of huge feature merges (a key source of bugs I've experienced) is minimized. You deploy fixes hourly, not weekly (or later monthly when it just won't…

Here's the caveats I have with that, although I do trend towards only having a master central branch whenever possible: 1) Small, incremental commits tend to be way more reviewable than big "finished" blobs of work. Local/small branches prevent me from getting blocked on waiting for review, and allows me to continue working, although you don't exactly need a "branching model" for it. 2) Tests/CI are insufficient. For…

Tests/CI are insufficient. For games, you'll want QA hammering on a release for days, weeks, or months with only the most conservative changes applied, in an effort to shake out any remaining weird heisenbugs. To do otherwise invites the specter of failing certification, delaying your release, or burning some pretty bad bugs to hundreds of thousands of physical disks.

Eh.. Game studios aren't generally still in the dark ages, are they? Physical disks went the way of the dodo. (Every game now basically requires updates on first use, right? So even if consumers get a disc[o ball], they won't really be affected by a bug burned to it.)

But yes, solid points.

Re: A successful Git branching model (2010)

#86
post #80
post #63

Earlier quoted context omitted.

> The dev source code isn't stripped of if-statements, right? When I use a feature flag as an alternative to branching, then I delete the if statements once the feature becomes permananent and gets released to all users. It's analogous to merging a branch. Obviously I'd leave it there during QA and A/B testing, but once toggling is no longer necessary, I remove the toggle. > Ok, but how? Assume an express server. You…

> then I delete the if statements once the feature becomes permanent and gets released to all users Doesn't this throw out some of the point of feature flags? That if your production servers are getting hammered, and you're having trouble scaling (for whatever reason), that you can degrade service by toggling off various features which are more resource-intensive so that your service doesn't crash entirely. Feature t…

> Doesn't this throw out some of the point of feature flags?

Not if you're using feature flags as an alternative to branching, nor if your feature flag is intentionally temporary. A branch is temporary, and merging the branch doesn't eliminate the point of branches, right?

For performance toggles you want to keep and use, then definitely just keep those.

> Feature toggles implemented through conditional logic seems like an anti-pattern to me. Developers make mistakes, and it's very easy to forget to wrap work around the necessary toggles

This is a valid concern generally, but in my experience doesn't detract much from the reasons to use feature toggles. In particular, the good examples you brought up (performance features & API feature toggles) you can't have without conditional logic, so there isn't any choice or alternative, right?

> It seems to me like a better pattern would be to feature-toggle on an API level

That's a good idea when the feature in question is an API feature. Are you a backend kind of person? ;) Lots of features are frontend, lots of features are full-stack. It all just depends on which feature, but generally speaking having API level feature toggles in your toolbelt is a great idea.

Re: A successful Git branching model (2010)

#87
post #81
post #77

Earlier quoted context omitted.

We used PRs with BitBucket for all code reviews. The reviewer(s) had to digitally sign their final approval of the review comments+answers and of the related code changes, if any. The only way to merge a feature branch into `develop` was via the PR + code review process.

Was it something like exporting PR history to a file and then signing (X.509/PGP)? Thanks for answers, it looks like a nice, lightweight auditable system.

No, simpler than that; we used the BitBucket web interface to enter the approval message and click the approved button to allow for merge. These actions are recorded and visible in the overview page of the PR.

However the BitBucket server's web interface was not approved/validated for long-term storage and evidence for the audit trail, so the PR owner was responsible (before triggering the merge) for saving a PDF copy of that PR page and committing the PDF file into a git-controlled code-review directory.

So it's digitally signed to the extent that your account/identity is recorded in the approval step and in the collection of PDFs. I did ask about a more systematic export method but it was not considered important given the PDF-based approach.

Re: A successful Git branching model (2010)

#88
post #15

I've got a much nicer branching model- try not to have one. Everyone works off master, and you aren't allowed to check in code that won't run in production. Hide unfinished features behind feature flags, and never merge/push a change that won't pass tests/CI. The chaos of huge feature merges (a key source of bugs I've experienced) is minimized. You deploy fixes hourly, not weekly (or later monthly when it just won't…

You're not the only who thinks this is a great idea.

https://www.thoughtworks.com/insights/blog/enabling-trunk-ba...

Re: A successful Git branching model (2010)

#89

Earlier quoted context omitted.

Here's the caveats I have with that, although I do trend towards only having a master central branch whenever possible: 1) Small, incremental commits tend to be way more reviewable than big "finished" blobs of work. Local/small branches prevent me from getting blocked on waiting for review, and allows me to continue working, although you don't exactly need a "branching model" for it. 2) Tests/CI are insufficient. For…

Tests/CI are insufficient. For games, you'll want QA hammering on a release for days, weeks, or months with only the most conservative changes applied, in an effort to shake out any remaining weird heisenbugs. To do otherwise invites the specter of failing certification, delaying your release, or burning some pretty bad bugs to hundreds of thousands of physical disks. Eh.. Game studios aren't generally still in the d…

> Eh.. Game studios aren't generally still in the dark ages, are they? Physical disks went the way of the dodo.

I wish! Turns out enough of the world still has terrible enough internet for sneakernet to still have it's advantages. It's not the game studios that are in the dark ages ;)

> Every game now basically requires updates on first use, right?

Sadly. It's a pretty terrible experience. Ideally it's a small optional patch, or maybe only required for online play - but your ability to do that depends in part on how bad the bugs are without it. The ability to do day one patches doesn't translate into a rubber stamp either - it's easier to get a waiver on a few heisenbugs than a truckload. And this is in the "okay okay, we'll backpedal on requiring our console to be always online" world of console dev - I imagine mandatory day 1 patches are even worse on handhelds.

And if you don't want that day 1 patch to introduce more bugs than it fixes (souring your launch, reviews, and ultimately sales) - or a release delay messing up your marketing plans (to potentially similar effect and/or extra costs) - you still need as stable a build as you can get by some hard cutoff X, so you're back to having a release branch, freezing master, or some combination thereof.

Re: A successful Git branching model (2010)

#90
Do not use Git Flow for a web application deployed on your own infrastructure (SaaS, microservice, mobile backend, etc.). It will slow down development and make your software less reliable.

The entire purpose of Git Flow is saving up changes to release later, e.g., saving up for a weekly release event. Don't do that! Deploy your changes as soon as they are ready, if they aren't ready don't merge them into a shared branch. If you do Continues Delivery you don't need "hotfix" branches because every changes goes out as soon as it is ready, so you don't need any of the complexity of Git Flow.

By saving up changes for a release event it means more things are getting released at once. If there is a problem after deployment it will be harder to narrow down the cause. Git Flow fosters a harmful development mentality where developers merge untested changes to the develop branch, then move on, and expect someone to test and stabilize their changes before release. With trunk-based development (https://trunkbaseddevelopment.com/) or GitHub Flow (https://guides.github.com/introduction/flow/) developers take ownership of their code, and only merge to master after they have tested it. With a good deployment pipeline they can own their code all the way to production.

Git Flow also encourages humans to think about and make up version numbers, like 15.0.5. This is a pointless waste of brain power, web apps don't need version numbers. The artifact systems (packages, containers, etc.) may need something, but it can just be an incrementing number that no on thinks about.

Git Flow wastes so much time, and makes everything it touches so complex, all to enable the harmful behavior of saving up changes for later, and enabling the pointless use of version numbers.

Trunk-based development and Continues Delivery is the default way people develop, it is how you would work if you had a one person company with one customer. It also is how the biggest web companies in the world work. It scales from smallest to largest. Just use trunk-based development. Stay away from Git Flow.

Edit: Fixed spelling of incrementing.

Post reply on HN