Live data from Hacker News

Code reviews aren’t just for catching bugs

blog.fullstory.com

141–150 of 150 posts

Re: Code reviews aren’t just for catching bugs

#141
post #112

While I value many of the same things as the author, I've found code reviews to be far inferior in every respect to pairing (especially promiscuous pairing (google it)), and to have negative effects in several important ways: * They delay integration. * They tend to encourage focus on abstract code polishing without proportionality or relation to the value of the code. * They favor superficial improvements, while inc…

I agree that those things could be issues with code reviews, but I don't see how "promiscuous pairing" helps with most of them. * They delay integration. Pairing by definition slows down all work on code(you have 1 person working instead of two). Also, code reviews need not delay integration, even if you're following the article's suggestions. You can still merge all work in an integration branch, and pull it if it e…

Thanks for your comments!

> I don't see how "promiscuous pairing" helps with most of them.

Maybe you should try it! The "promiscuous" part transfers context and knowledge around more quickly, and gives more opportunities, earlier in the process, to respond to feedback from a wider variety of perspectives. Two people might come to the same conclusion that another, joining a couple of hours later and without following the same mental path, would find crazy. It's generally pretty easy to change direction after a couple of hours.

> Pairing by definition slows down all work on code(you have 1 person working instead of two).

"Work on code" is not all that's involved in shipping product. In my experience, pairing produces less/simpler/better code that more directly addresses intended business value and cuts off unproductive paths more quickly than other methods. I much much much prefer cranking out less of the right code than producing tons of "perfect" code that solves the wrong problem (slightly hyperbolic, but you get the point, and it totally happens all the time). It's worth mentioning that what I'm talking about is pairing done well, by people who have learned how to do it well. It's absolutely possible to do a shit job pairing, as with anything else.

>* They tend to encourage focus on abstract code polishing without proportionality or relation to the value of the code. >This is an issue with the priorities of the people doing the code review, and thus applies just as much to pairing.

This is true to a certain degree, but good pairing typically involves thousands of little tradeoff decisions and discussions about how to solve problems. A code review can't reproduce the richness of all of that communication, so it necessarily emphasizes code in the abstract over deep consideration of the tradeoffs involved in the solution to a problem.

> I have no idea how pairing is possibly different from code review in [the case of superficial improvements.]

When I'm sitting down with someone to solve a problem and we explore a path for a few minutes, then I can step back and say, "wait, based on what we've just done, I think we're thinking about this whole thing in the wrong way, what about this other thing," it's easy to turn around and explore a totally different approach. When somebody has sunk an entire day into solving a problem, polishing the code to impress their coworkers, and responding to code review comments, it's extremely difficult to say, "hey, I think we're thinking about this the wrong way, what about this other thing?" I've experienced this problem nearly every time I've participated in code reviews and rarely during pairing. (To that point, pairing is such a better dynamic for this type of exchange. A code review has an oppositional nature, while with pairing, you're literally on the same side of the problem, sitting together to find a solution. The dynamic of questioning the path/approach starts off in a much more natural/friendly/collaborative place.)

> so the most egregious problem with code reviews is the people that do them might have wrong priorities

I think it goes beyond that. As mentioned above, the structure of code reviews (vs. pairing) favors certain types of feedback over others, and makes the feedback that I believe to be most valuable very difficult to give, socially, and too late to be really valuable.

>A benefit of pairing that I can see over code reviews is when you get feedback - instant, real-time in pairing vs late in code reviews.

Yes! Exactly. And from what I've seen, this makes an enormous amount of difference.

I don't agree with you about the time differences, especially when considering the full cycle of delivery. It's a notoriously difficult thing to measure effectively and/or prove, so...it's one of those things you have to experience first hand (in an environment where it's done well...there are plenty where it's not) to really buy into perhaps.

It's also true that some people just want to work solo. I'm motivated by producing the best possible product and generating the best possible outcomes, and pairing is the best way I've encountered to do that, so...in general, I'd prefer not to work with those people when they're steadfastly opposed to trying collaborative work. At the same time, I've found that most people who give it a shot with a good pair who has some patience, empathy, and skill, end up loving it. Even folks who generally work alone.

Re: Code reviews aren’t just for catching bugs

#142
post #110

What often goes unmentioned in praise for code review processes is their insanely exorbitant costs -- measured in engineer hours but perhaps more costly is all of the blocking and impedance [1]. Most of the "problems" that code reviews claim to address can be solved by much more direct and optimal measures. Code reviews are damn expensive. This post concedes that code reviews are better for the more fluffy ends -- te…

> What often goes unmentioned in praise for code review processes is their insanely exorbitant costs -- measured in engineer hours but perhaps more costly is all of the blocking and impedance [1]. I don't get why you think code reviews are so expensive. An engineer should rarely be blocked by a code review. If waiting for a review - don't. Pick up another task! A great thing about code reviews is they are done async.…

As the comments all over this thread demonstrate, code review is done for many reasons. Where I've worked, the reasons have not been clearly stated, and as a result there is a lot of time wasted. I wouldn't say that there should not be code reviews, but they could be cut way down. A code review should not be a substitute for a design review. It often is, in my experience. A code review should not be done for anything that can be done by static code analysis: code style, finding dead code, etc.

Also, a lot depends on how code reviews are done. I've seen them used as a veto, with lots of back and forth to get the veto removed over subjective and trivial issues. Don't do that.

Finally, let's be honest. Some people's code doesn't need review. Time spent reviewing such code is wasted. But it is often socially difficult to say this, so everyone's code gets reviewed.

Other people can't be trusted to write acceptable code, and reviews are essential. That's a different problem. (Beyond the scope of this note. Left as an exercise to the reader. Other cliches may apply.)

Re: Code reviews aren’t just for catching bugs

#143
post #62

How do people handle reviews of highly specialized stuff? We have people who do stuff nobody else on the team understands or at least it would take them a long time of learning to do a real review. I look at a lot of stuff and check if it makes halfways sense. I can look at the coding style but I can't judge the overall design without spending many hours on it (which I don't have. Nobody else on the team has it eithe…

Where I work the same situation often comes up -- a developer may spend weeks doing research for a specialized function building prototypes, etc. Ensure the specialized developer is doing due-diligence in defining and verifying the module works as intended and have others analyze its interactions in the larger system to prevent cascading failure, conforms to application norms, __is documented__, etc. Even if most oth…

The thing is they can document as much as they want. Often the implementation is faulty and not what the documentation says it is. This happens a lot in multithreaded code.

Re: Code reviews aren’t just for catching bugs

#144

Earlier quoted context omitted.

> Another huge cost of code reviews is distraction. We've all seen the Paul Graham essay on maker's schedules vs. manager's schedules. We've all read the statistics on how much time is lost to interruptions. Code reviews are a massive interruption, done on a manager's schedule. Each code review is a distraction, and can take a significant time commitment, if it is to be a meaningful review. Wait why are we doing code…

What? And keep the other developer waiting?

Waiting to do what? Why can't they just go off and do something else?

Re: Code reviews aren’t just for catching bugs

#145
post #126

Earlier quoted context omitted.

Code reviews should be asynchronous. Everyone on your team should be able to be working on multiple small changes/commits/whatever in parallel. While one is out for review, they're working on the others. Different people keep different schedules: I do all my reviews first thing in the morning, to settle in, and then often do another round after lunch. Other people do them at the end of the day, or don't mind the inte…

Context switching also creates some overhead. YMMV as the time it takes to switch tasks IMHO depends on particular person, context scale and problem difficulty.

This has nothing to do with code review. If you're finished with a task, you're going to have to switch contexts; you can't just keep working on something that's done.

Re: Code reviews aren’t just for catching bugs

#146

Earlier quoted context omitted.

Not sure how to formalize but the team's I've worked on have generally negotiated mutual respect, openness, and teamwork by collaborating on the things of greater import -- the architecture, domain conceptualization, etc. And peer code reviews in some cases tend to work against these goals--because you tend to be down in the weeds of LOC, bike-shedding, arguing over the equivalents of tabbing and spacing or whether a…

Well perhaps another part of Google's code review culture is important, then: proposed changes must be style compliant, must compile, must have tests, and all the tests must pass before anyone will bother looking at them. If I got a code review (at Google) that was incorrectly indented, there would be a little red chip in the review UI that indicated such style violation, and I would just reply "Please fix" and not l…

What UI does Google use for code review?

Re: Code reviews aren’t just for catching bugs

#147

I'm a dev with one year's experience, and recently moved to a team that reviews every PR. The benefits to me personally are super clear. #1, more experienced engineers are giving me frequent feedback, and that's obviously worth a ton. #2, it's part of my job to read other developers' code. I get exposed to patterns and design choices that I may not have in my repertoire. Maybe the feedback I give the other direction…

This is a great attitude to have!

Re: Code reviews aren’t just for catching bugs

#148
post #98

Earlier quoted context omitted.

Large commits getting stuck in review forever, or until they have too many conflicts to be merged cleanly; small commits being over-reviewed for trivial issues because the reviewer wants to prove that they have actually looked at the code.

You can also refuse changes for being too big to review. Creating incentives for small changes seem like a good thing to me, so I don't see where you complaint is.

As I said, review culture can also create incentives against small commits, because they will often be over-analysed. I've seen this sentiment on the internet, so it can't just be my own anecdata.

https://twitter.com/girayozil/status/306836785739210752

Not to mention trivial commits, like clarifying comments, something which doesn't happen anymore in my current project - nobody can be bothered to go through the review process for that.

I'm not against code reviews, but I prefer casual post-commit reviews for uncontroversial changes.

Re: Code reviews aren’t just for catching bugs

#149
post #68
post #7

I would go further and say that you should never trust code reviews to catch bugs. They're great for all the reasons in the article and what "zhemao" said but they're horrible for catching any bugs beyond the most trivial ones. Humans simply aren't good at running code in their head like that. Code review is no substitute for good code coverage and automated testing, preferably pre-commit.

Honest question: what strategy would you recommend to deal with a (senior) developer who specializes in opening gigantic pull request with significant number of bugs? (I invest a lot of time to read through the code and I catch lot of stuff, but it's draining huge amounts of my energy). Declining anything with coverage below 100% is not a viable option unfortunately, I think, and preaching about best practices gives…

There could be a bunch of reasons there's bugs.

I view bugs as a potential issue in an individual's development process. It sounds like the biggest reason you already hit on -- gigantic pull requests. Gigantic requests usually mean PRs that aren't focused to one discrete work-piece, but represent work that kinda meanders to completion. Make sure they're focused, and ask for nonessential pieces to be delegated to another PR (style-only PRs being a good example).

Ultimately the only way to suppress large pull requests (or problematic practices in general) is to evangelize simple development workflows that others can help you with. Keep in mind that change for anyone, no matter intelligence or stubbornness, takes time - culture especially. You need to make it easier for them to adopt your methods instead of theirs when the time comes and they get fed up and are ready to change. Make sure you don't have an antagonistic relationship, either, or they'll do anything but what you want just to spite you :(

I've broken up some stuff to look for in code reviews, in code bases, and in developers. You probably only want to look at developers.

For the code itself:

- Do we have overdeveloped patterns that get in the way of expressive code? - Do these bugs fall into a general classification that can pinpoint a source (such as timing issues, authentication, database calls, or info validation?) - Is your code-base DRY? Repetitious functionality sprinkled around will mean bugs that never get fixed. Eliminate repetition. - Age of code-base? Newer code-bases should be more accepting of large PR's/more bugs, but should compensate with refactoring-only PRs, test blitzes, and accessible code coverage. Old code may have traditions that have outlived their usefulness, and owners that have grown too accustomed to the state of the code to understand the need for improvement. - No experts. Is this an inherited code-base with poorly documented patterns whose original authors have moved on?

For code reviews (speeding up review):

- First read-through of the code is only to familiarize myself with the code added. - Does this PR solve one problem or a bunch of problems? One feature/bug/style-change == one PR (per repository). Ask to break it up if it's big and doing more than one thing (unless those things are highly coupled). - Is there core functionality? Start with the most reused piece, and critique that first. - Is this code shared? Shared code should be held to a higher standard, especially if other people depend on it. Get their eyes on it too, it takes some work off you, builds ownership, and if this person is a problem, you'll want allies to back you up. - Are methods long? Longer methods tend to do more than one thing, and invites glomming onto existing methods instead of creating your own. Higher standards for shared code modification tends to lead to shorter methods, because reusing code needs to be a deliberate decision. Plus long methods can hide code reuse. - Is manual testing difficult? Long or difficult verification loops tend to allow for more bugs.

For the developer:

- What are their tools? What tools are you using that makes you more effective? Learning new tools is hard, but most bugs come from ineffective development practices - Are they senior in knowledge or senior in age/time? Not all senior devs deserve their title. - Do they stay on-task? Do they have to thrash between tasks? Unfocused development is a big source of bugs. - How familiar are they with the code base? Senior or not, code from new devs should require more scrutiny (but not to the point that it discourages collaboration).

Re: Code reviews aren’t just for catching bugs

#150
post #126

Earlier quoted context omitted.

Context switching also creates some overhead. YMMV as the time it takes to switch tasks IMHO depends on particular person, context scale and problem difficulty.

This has nothing to do with code review. If you're finished with a task, you're going to have to switch contexts; you can't just keep working on something that's done.

It was related to "Everyone on your team should be able to be working on multiple small changes/commits/whatever in parallel."
Post reply on HN