Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Not all bugs are worth fixing and that's okay

blog.bugsnag.com

Post reply on HN
Print view View original

31–40 of 63 posts

Re: Not all bugs are worth fixing and that's okay

#31

Post by phyzome »

The author of this article seems to think that occurrence frequency is the sole metric of whether a bug should be fixed.

Which of these two is more important?

  - 0.1% of my users lose their data irrecoverably
  - 30% of my users get an error page and have to refresh
The article never waded into this at all, which is disappointing. I don't feel like I learned anything.

Re: Not all bugs are worth fixing and that's okay

#32

Post by ataggart »

>at a certain point, it’s too expensive to keep fixing bugs because of the high-opportunity cost of building new features.

While I may agree with this in the abstract, in practice most folks don't really know whether they're at that point. It also doesn't consider cumulative effects over time.

Bugs don't just affect application stability or user experience. A system that does not behave as designed/documented/expected is a system that will be more difficult to reason about and more difficult to safely change. This incidental complexity directly increases the cost of building new features in ways difficult to measure. Further, new features implemented by hacking around unfixed flaws will themselves be more difficult to reason about and more difficult to change, exacerbating the problem.

The larger the system grows over time, the more people working on it over time, the faster this incidental complexity problem grows over time. At a certain point, it's too expensive to not fix the bugs because of the increasingly high cost of building new features. At that point, folks start clamouring for a rewrite, and the cycle begins anew.

Re: Not all bugs are worth fixing and that's okay

#33

Post by wilun »

ryandrake wrote: post #23
nowayjose2 wrote: post #6

I have a funny feeling that the nontechnical people on my current project would be nodding their heads along to the article, but the truth is that our applications have bugs that 100% of our customers are running into; they simply aren't immediately noticeable to a layperson. That doesn't mean they're not important. The business relies on complying with the rules of third-party organizations and the software is blata…

One of the toughest things I struggled with while transitioning from a larval junior developer to a senior tech lead to a project manager was the fact that (at least in the context of a for-profit business) not all bugs need to be fixed, even the ones you personally think are really really bad. The goal is to make money, not necessarily by producing the most perfect software. The quality bar needs to be high, but the…

That's fair, but one has to remember that some of the key points are to be balanced, and that, like you said, "the quality bar needs to be high".

And I'm more for prioritizing trying to not introduce bugs than to fix all the old ones. Which is challenging on, how could we call that?, "legacy" software. So that priority can and must be reversed temporarily when that "legacy" is too much.

So it's all very context dependent, and not having anybody (or too few) working on making things better when its needed is not going to deliver any kind of velocity in the long term (and probably the short term velocity is already way too low in those cases). Too bad for the mythical time to market...

So you have to be able to say no to bugfixes, but you certainly also have to be able to say no to the eternal rush of new half-backed features, when needed. A short-term ever obsession on "opportunity cost of not working on a feature" could yield quite paradoxical results if trying to build them on some kind of zombie legacy code (that is only ever edited with disgust and great difficulty, but never seriously refactored).

Not only this balance is hard to achieve, but your role as a senior tech lead and project manager is certainly to consider carefully the cleanup needs, and be an advocate for them when needed, including by pushing back against feature creep pressure. Because if you are not, most of the time nobody else will. As a tech lead, this mean among other things, that a black box approach of parts of the maintained software is out of the question (of course you can delegate, but even then its imperative to stay in the equation for that purpose, only with less details). Paradoxically, even if the quality is crap and the organization notices and tracks loads of bugs, most people will be happy at the moment the bugs are triaged and assigned and eventually "fixed" by more horrible garbage (that is, the impression that something is done), rather than doing the right thing that is to organize a cleanup of the software more in depth.

I've got the impression that it is rare to find projects where this balance is achieved correctly, but maybe it's only because of bad luck. Well in lots of cases, the famous ones (I'm thinking on the level of Linux, Firefox, Python, etc. not just your random niche software) are actually not that bad, and their competitors have a way shorter lifespan when not as balanced...

Re: Not all bugs are worth fixing and that's okay

#34

Post by Phrodo_00 »

ryandrake wrote: post #23
nowayjose2 wrote: post #6

I have a funny feeling that the nontechnical people on my current project would be nodding their heads along to the article, but the truth is that our applications have bugs that 100% of our customers are running into; they simply aren't immediately noticeable to a layperson. That doesn't mean they're not important. The business relies on complying with the rules of third-party organizations and the software is blata…

One of the toughest things I struggled with while transitioning from a larval junior developer to a senior tech lead to a project manager was the fact that (at least in the context of a for-profit business) not all bugs need to be fixed, even the ones you personally think are really really bad. The goal is to make money, not necessarily by producing the most perfect software. The quality bar needs to be high, but the…

I agree, but I think you still need to be deliberate about not fixing bugs. Sure, don't fix it, but create a ticket for it and use it to track how common it actually is.

You also should probably be fixing bugs that "If we're caught out by those third parties before the bugs are fixed, there's a good chance it could sink the whole company"

Re: Not all bugs are worth fixing and that's okay

#35

Post by coldtea »

nowayjose2 wrote: post #6

I have a funny feeling that the nontechnical people on my current project would be nodding their heads along to the article, but the truth is that our applications have bugs that 100% of our customers are running into; they simply aren't immediately noticeable to a layperson. That doesn't mean they're not important. The business relies on complying with the rules of third-party organizations and the software is blata…

>I have a funny feeling that the nontechnical people on my current project would be nodding their heads along to the article, but the truth is that our applications have bugs that 100% of our customers are running into; they simply aren't immediately noticeable to a layperson. That doesn't mean they're not important.

If they're not noticeable, and the customers keep buying, despite 100% of them having them, how are they "important"?

>The business relies on complying with the rules of third-party organizations and the software is blatantly violating those rules right now.

That's a business decision. It could even be illegal (but still not big of a deal depending on circumstances) but not a bug.

Re: Not all bugs are worth fixing and that's okay

#36

Post by loopj »

nowayjose2 wrote: post #6

I have a funny feeling that the nontechnical people on my current project would be nodding their heads along to the article, but the truth is that our applications have bugs that 100% of our customers are running into; they simply aren't immediately noticeable to a layperson. That doesn't mean they're not important. The business relies on complying with the rules of third-party organizations and the software is blata…

Heads up - I co-wrote this article with Kristine and this is based around the whole thesis of our company existing - I'm a software developer by trade with 15 years of experience. Having said that, dismissing the content based on the author I think comes across as an ad-hominem attack - after working with her for 4 years I count Kristine as an expert on this matter.

The overall point here isn't to use this as an excuse to not fix bugs, more that you should consider applying the same logic as devops/sre teams use for "uptime and availability" (5 nines, etc) to software stability to help you move faster as a company.

Re: Not all bugs are worth fixing and that's okay

#37

Post by wilun »

ataggart wrote: post #32

>at a certain point, it’s too expensive to keep fixing bugs because of the high-opportunity cost of building new features. While I may agree with this in the abstract, in practice most folks don't really know whether they're at that point. It also doesn't consider cumulative effects over time. Bugs don't just affect application stability or user experience. A system that does not behave as designed/documented/expecte…

If the only alternative is between a rewrite, and not-fixing the mess gradually, then I'll take the rewrite anytime and let the cycle continue.

The problem is: is your rewrite really going to be a full-rewrite, or some kind of hybrid monster (at the architectural level, of course, there is no problem in reusing little independent pieces, if any exist)? Because you can easily fall in all the traps of both sides, if the technical side is not mastered well enough by the project management...

Re: Not all bugs are worth fixing and that's okay

#38

Post by quanticle »

ryandrake wrote: post #23
nowayjose2 wrote: post #6

I have a funny feeling that the nontechnical people on my current project would be nodding their heads along to the article, but the truth is that our applications have bugs that 100% of our customers are running into; they simply aren't immediately noticeable to a layperson. That doesn't mean they're not important. The business relies on complying with the rules of third-party organizations and the software is blata…

One of the toughest things I struggled with while transitioning from a larval junior developer to a senior tech lead to a project manager was the fact that (at least in the context of a for-profit business) not all bugs need to be fixed, even the ones you personally think are really really bad. The goal is to make money, not necessarily by producing the most perfect software. The quality bar needs to be high, but the…

    One of the toughest things I struggled with while transitioning from a 
    larval junior developer to a senior tech lead to a project manager was the 
    fact that (at least in the context of a for-profit business) not all bugs 
    need to be fixed, even the ones you personally think are really really 
    bad. The goal is to make money, not necessarily by producing the most 
    perfect software.
On the other hand, you have to remember the that impact = risk x loss. And developers and managers are notoriously bad at evaluating the risk posed by a bug. It does no good for software to make $10 millon dollars a year for 5 years and then make a $100 millon dollar loss in the sixth, because of a catastrophic bug that no one prioritized fixing because, "It's been 5 years and no one's run into this bug yet."

Re: Not all bugs are worth fixing and that's okay

#39

Post by wilun »

izzydata wrote: post #21
debt wrote: post #10

Bugs shouldn't happen. Actually, in some critical systems, bugs can't happen. Bugs aren't magic; they happen for a reason. It could be a broken dependency(unsupported versions, fatal bug in a dependency, deprecation, configuration etc.), resource limitation(out of memory, security breach etc.), poor design which leads to poor implementation(logical errors, bad data abstractions). Abstractly waving your hands and sayi…

I dare you to write 100 lines of useful code without a bug in it.

Are you even trying?

A random search tells me that "The mean DD for the studied sample of projects is 7.47 post release defects per thousand lines of code (KLoC), the median is 4.3 with a standard deviation of 7.99." ( https://ieeexplore.ieee.org/document/6462687/ )

So clearly if you are careful and use state of the art practices, this is very doable.

Not only this is doable, but various individuals and teams in history have been able to reach way lower defect densities. Hey, for all practical purposes, TeX is bug free, for example.

If you are not able to write 100 lines of useful code without a bug in it (not in an infallible way, but at least sufficiently often enough), maybe you should simply study and practice to get that ability.

Re: Not all bugs are worth fixing and that's okay

#40

Post by quanticle »

nathan_long wrote: post #9

Good article. It reminds me of Sandi Metz's treatment of well-designed code as a business proposition: the goal is to save money, because a good design makes changes cheaper. It makes sense to consider the cost of having a given bug vs the cost of fixing it. Of course, such estimates will almost always be hand-wavey. I would also say that when in doubt, fix it. A bug is, by definition, the software not doing what it'…

    It makes sense to consider the cost of having a given bug vs the cost of 
    fixing it. Of course, such estimates will almost always be hand-wavey.
I agree with that approach in theory, but in practice it turns out that that it's a lot easier to estimate the costs of fixing the bug than it is to estimate the cost of having the bug. As a result, because of our biases, in any ambiguous case, our bias will be for keeping the bug, since the cost of having the bug is the impact of the bug multiplied by the probability of someone hitting it, and it's always easy to lowball those probabilities. "Oh, no one will notice that," or "Yeah, but that's a really obscure case." And then you find out that all it takes is one obscure case for your trading application to lose hundreds of millions of dollars a day. Or for hackers to breach your systems and make off with millions of credit card numbers. Or for malware to turn your IoT devices into a botnet.
Post reply on HN