Trunk-Based Development • Hacker News Board

Re: Trunk-Based Development

#161

Post by dang » Sun, Jan 29, 2017, 10:23 PM UTC

whyileft wrote: post #70

Yup, as the other comment stated, this is par for the course in modern quality software development. Its actually rather shocking to me to see my comment get downvoted. The quality of the HN audience is declining dramatically. Feels like all we have is a bunch of web agency lifers at this point.

A microcode assembler has been in the top 5 for hours now. I suspect we're ok.

It's tempting to draw general conclusions about HN from specific things you don't like, but that's sample bias.

We detached this subthread from https://news.ycombinator.com/item?id=13514804 and marked it off-topic.

Re: Trunk-Based Development

#162

Post by glenjamin » Sun, Jan 29, 2017, 10:34 PM UTC

cygned wrote: post #86

We currently use short-lived feature branches, merged via Pull Requests (+ review / automated testing) into the main development line. This way, we can communicate changes in a detailed manner before they are added to the product and make sure there is no unfished or bad code in the main branch. (The dev team is small, 5-8 devs). I don't see (yet?), what benefits TBD would provide in such a setup.

The linked website includes this workflow under their definition of trunk-based-development. As long as your branches are short-lived and make it into the main line rapidly, it appears to be considered equivalent.

I've seen some commentators say that this doesn't pass their bar for TBD, but I think it's effectively the same thing.

The site does call out "GitHub flow" as being slightly different, but that's because GitHub's description of their model includes deploying the branch to production before merging to master.

Re: Trunk-Based Development

#163

Post by alkonaut » Sun, Jan 29, 2017, 10:39 PM UTC

sydd wrote: post #151

alkonaut wrote: post #148

In subversion the only way I can possible develop with sanity is in an "unstable trunk". Branch off releases N weeks before shipping, and have only the requirement that trunk passes automated tests while release branches are manually tested. Obviously you don't release more than say once every month or two, but that's more than enough for most. I'm honestly not sure if the site is satire? No one develops in long live…

If you release only a month or 2 you are doing something really wrong (or need to work based on some insane spec). Read the basics about CI and CD, read the clean code book, read about modern organisation practices (e.g. Github releases several times a day). Releasing every month or so is (thank God) a relic of the old waterfall development times.

We do CI, not CD. Lots of software doesn't do CD and lots of it can't. Games. Embedded. Desktop. Store-apps.

In my case it's large desktop software for structural engineering. Customers don't want new versions more than 1-2 times per year because of deployment, training and code compliance requirements. If we bork a release we have had users make a long download+install again. We don't just deploy the fixed one in prod to solve problems. Thats why a release goes through hundreds of hours of manual testing.

Not to mention data: a feature is a new file format. If we ship one new feature we basically ship a new format. Now you know why Autocad, Word and Illustrator ships once per year and not once per commit.

I think what a lot of people (who presumably do web dev only) forget is that web dev is just one of many software development disciplines, and a very young discipline too.

I know of the practices you mention. I like clean code (both the book and the idea). I'd do CD if I had a project some day that runs on a server. I hope I won't have to do web dev any day soon though.

Re: Trunk-Based Development

#164

Post by TylerE » Sun, Jan 29, 2017, 10:42 PM UTC

stcredzero wrote: post #144

kosma wrote: post #68

This workflow, like all others, is just a formalization of some reality. The reality is that many organizations and teams run on nothing but trunk - and it more-or-less works on them. Some teams work by building blocks separately and then joining them together; other prefer to hammer away on a problem all together. Both approaches work. What worries me are emotional claims that the other approach is fundamentally fla…

Trunk-only is a reality. Not a reality some of us would want to live it - but one that nevertheless exists. My background in Smalltalk made me accustomed to a style where everyone would continuously merge everyone else's changes as they worked. This style makes you aware of what your team mates are doing. In fact, it facilitates communication pretty much when communication is most called for. As always: context. The…

This is basically how the team I work on does it. We have wrapper scripts around git. Commiting automatically merges from master first.

Re: Trunk-Based Development

#165

Post by clusmore » Sun, Jan 29, 2017, 10:50 PM UTC

nhumrich wrote: post #3

hashkb wrote: post #2

The suggestion that feature branches hinder CD is just untrue, at least in my experience. To me, this seems like a silly restriction to put on such a critical tool (the scm) which is designed to support unlimited flexibility.

You must not understand what CD really means then, because full CD is impossible with feature branches. CD means every commit gets built and delivered all the way prod if it passes all tests. The key word is "continuous" as in, every single commit. Branch based means you only deliver once you merge. Merging is a manual step, so your not doing full-CD. The merge is basically your "trigger" and your doing "delivery whe…

Why do we have this obsession with religiously following methodologies rather than focusing on the goals they attempt to achieve?

>CD means every commit gets built and delivered all the way prod if it passes all tests. The key word is "continuous" as in, every single commit.

In my experience one of two things happen with this approach. Either the functionality is hidden behind flags and intentionally doesn't impact prod, in which case there's no point in it being there. Or you only commit when the feature is complete, which is effectively the same "delivery when I am ready" as with merges.

The goal of CD is not to push every commit to prod -- this is merely the process. If the goal achieved by the process of pushing every commit to prod can be achieved via a different process, we shouldn't be so quick to criticise the other process. We should instead discuss the tradeoffs between the two, because in reality there is no best option, only tradeoffs.

Re: Trunk-Based Development

#166

Post by alkonaut » Sun, Jan 29, 2017, 10:58 PM UTC

dpark wrote: post #159

alkonaut wrote: post #150

Earlier quoted context omitted.

So the person that has an opportunity to misunderstand the specification (the developer) is given the task of writing the tests to OK them, even though the test is based on the same misunderstanding? That's terrifying. After 15 years in the same field I still don't have half the business knowledge as those writing the specs I implement. I would never ever want a nontrivial feature I wrote to hit a customer without ma…

Where do you work that software engineers write code and then hand off their code to non-software-engineer "experts" for testing? That sounds like a really broken process.

I implement a program used by structural engineers. So basically the user is a structural engineer and I'm a software engineer. They typically find nuances of program behavior that I never thought of because I'm not an expert in structural engineering.

I think the same would be true if I made a trading platform, an x-ray machine UI or whatever. When the expert uses it to do what they are experts in, they will invariably find issues (bugs, omissions, simple improvements) that weren't obvious to begin with.

One can argue that if the spec was 100% perfect then I can always test it myself and hopefully even do so with automated tests - but I have never seen a spec like that (perhaps more importantly - if you have in house "end user like testers" for expert software then it's likely more economical to have expert testing and iterate than spend the time on more detailed upfront specs)

Re: Trunk-Based Development

#167

Post by dpark » Sun, Jan 29, 2017, 11:14 PM UTC

alkonaut wrote: post #166

dpark wrote: post #159

Earlier quoted context omitted.

Where do you work that software engineers write code and then hand off their code to non-software-engineer "experts" for testing? That sounds like a really broken process.

I implement a program used by structural engineers. So basically the user is a structural engineer and I'm a software engineer. They typically find nuances of program behavior that I never thought of because I'm not an expert in structural engineering. I think the same would be true if I made a trading platform, an x-ray machine UI or whatever. When the expert uses it to do what they are experts in, they will invaria…

For a huge chunk of the industry, the consequences of broken software are limited to annoyed users and lost revenue. In cases like that, the benefits of shipping quickly often outweigh the value of "expert testing". Further, in many cases the software engineers have as much expertise as the customers, making the handoff for testing simply a way of abdicating responsibility for quality.

For your structural engineering example, I'm not sure you couldn't benefit from continuous, automated release. If the only risks are missing "bugs, omissions, simple improvements", you could fix those in the next release (which could be the next day). Delaying valuable features so that the customers can tell you that a tweak would be even better doesn't seem to be a net gain. The only reason to hold the release would be if you're catching dangerous bugs this way.

You could also build new features under a "flighting" system (pick your favorite name; there are several) where you don't expose new features to most customers until they are "baked" with your internal customers and/or customers who've opted into early features. This allows you to release constantly so your customers get bug fixes quickly and features as soon as they're ready without the complexity of separate branches and versions maintained in parallel.

Re: Trunk-Based Development

#168

Post by alkonaut » Sun, Jan 29, 2017, 11:21 PM UTC

viraptor wrote: post #154

alkonaut wrote: post #148

In subversion the only way I can possible develop with sanity is in an "unstable trunk". Branch off releases N weeks before shipping, and have only the requirement that trunk passes automated tests while release branches are manually tested. Obviously you don't release more than say once every month or two, but that's more than enough for most. I'm honestly not sure if the site is satire? No one develops in long live…

It's not satire. In your case the automated tests are just weak. You call the trunk unstable, because you need extra manual testing. What the page describes is an improvement on that. The CI system should have good enough tests to guarantee your trunk is never considered unstable. Then it doesn't really matter when you make a release - it's not a big deal. (Just choose a commit you like)

I think "pick a commit" only holds up for server based software.

For things like large desktop software with long term supported file formats, mission critical software where people might get killed if something malfunctions etc then it's pretty normal to have manual testing. If the cost of deployment of a release is high (N people training, downloading, installing) then you want few releases - and you can also motivate spending on manual testing to ensure you don't need a new release sooner than necessary.

The release branch also works as a beta/rc branch so that a version is tested on a subset of users before general availability, while things like large refactorings and major features can start being developed for vNext in the trunk/master.

I'm not sure but I'd guess this is how most software other than web and server software is developed.

Re: Trunk-Based Development

#169

Post by dpark » Sun, Jan 29, 2017, 11:30 PM UTC

jfim wrote: post #147

Animats wrote: post #94

"The core requirement of Continuous Integration that all team members commit to trunk at least once every 24 hours." Continuous integration is a means to an end, not an end in itself.

Indeed. There's a wide spectrum of software complexity and while committing to trunk every day might be reasonable for lower complexity projects, it's definitely not for higher complexity ones. Having such a requirement also means that people can't undertake major refactorings/rewrites of significant subsystems, leading to long term tech debt.

In my experience, the "refactor the world in a separate branch" strategy is always a giant mess. The initial work is perhaps simpler but the final merge is always horrible and the full impact of breaks isn't found until the merge is committed and you're stuck in "fix forward" mode because reverting is an unacceptable choice due to the cost of merging.

I'm not sure the overall cost to the team wouldn't be lower if these sorts of rewrites were done in place. (Also it's not really a refactoring if it's so big you can't do it in branch.)

Re: Trunk-Based Development

#170

Post by alkonaut » Sun, Jan 29, 2017, 11:37 PM UTC

dpark wrote: post #167

alkonaut wrote: post #166

Earlier quoted context omitted.

I implement a program used by structural engineers. So basically the user is a structural engineer and I'm a software engineer. They typically find nuances of program behavior that I never thought of because I'm not an expert in structural engineering. I think the same would be true if I made a trading platform, an x-ray machine UI or whatever. When the expert uses it to do what they are experts in, they will invaria…

For a huge chunk of the industry, the consequences of broken software are limited to annoyed users and lost revenue. In cases like that, the benefits of shipping quickly often outweigh the value of "expert testing". Further, in many cases the software engineers have as much expertise as the customers, making the handoff for testing simply a way of abdicating responsibility for quality. For your structural engineering…

We can't ship often for the same reason all of the big and complex software packages (IDE:s, spreadsheets etc) on your machine don't ship very often. Documentation needs to be produced and specific to a version. The application needs to be a consistent whole with UI changes, file format changes etc not happening too often.

I don't think it will ever be a good practice for large complex apps to change a tiny bit every day (Facebook might be challenging my theory, but their app is relatively simple they don't produce training docs, and most importantly they don't have to have the latest app compatible with all Facebook data from the beginning of time - instead they keep their data on their servers and modify it to the latest programs when necessary)

I agree you could have more feature gating, but large backwards compatible file formats are a complex business already with 10 releases over a decade - I can only imagine what it would be like supporting reference-rich documents with many more releases and the additional complexity of the sender and receiver having to agree on a feature set (unless you make the feature set/flight implicit from the data - but that's a new kind of headache). We already have tons of code in new versions dealing with loading malformed data in old formats because of bugs closed years ago! Every document we wrote we must also be able to read.

Lots of challenges in this area, but they are pretty fun to work with tbh.