Live data from Hacker News

Debugging: Indispensable rules for finding even the most elusive problems (2004)

dwheeler.com

131–140 of 238 posts

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#131
post #48

Rule 0: Don't panic Really, that's important. You need to think clearly, deadlines and angry customers are a distraction. That's also when having a good manager who can trust you is important, his job is to shield you from all that so that you can devote all of your attention to solving the problem.

This is very underrated. Also an extension to this is don’t be afraid to break things further to probe. I often see a lot of devs mid level included panicking and thus preventing them to even know where to start. I’ve come to believe that some people just have an inherent intuition and some just need to learn it.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#132

Some additional rules: - "It is your own fault". Always suspect your code changes before anything else. It can be a compiler bug or even a hardware error, but those are very rare. - "When you find a bug, go back hunt down its family and friends". Think where else the same kind of thing could have happened, and check those. - "Optimize for the user first, the maintenance programmer second, and last if at all for the c…

The first one is known in the Pragmatic Programmer as “select isn’t broken.” Summarized at https://blog.codinghorror.com/the-first-rule-of-programming-...

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#133
post #89

I have been bitten more than once thinking that my initial assumption was correct, diving deeper and deeper - only to realize I had to ascend and look outside of the rabbit hole to find the actual issue. > Assumption is the mother of all screwups.

I've once heard from an RTS game caster (IIRC it was Day9 about Starcraft) "Assuming... Is killing you".

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#135

Earlier quoted context omitted.

> why I maintain the discipline that all commits to the "real" branch, however you define that term, should all individually build and pass all (known-at-the-time) tests and generally be deployable in the sense that they would "work" to the best of your knowledge, even if you do not actually want to deploy that literal release You’re spot on. However it’s clearly a missing feature that Git/Mercurial can’t tag diffs a…

This is why we use squash like here https://docs.gitlab.com/ee/user/project/merge_requests/squas...

I explicitly don’t want squash. The commits are still worth keeping separate. There’s lots of distinct pieces of work. But sometimes you break something and fix it later. Or you add something new but support different environments/platforms later.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#136
post #48

Rule 0: Don't panic Really, that's important. You need to think clearly, deadlines and angry customers are a distraction. That's also when having a good manager who can trust you is important, his job is to shield you from all that so that you can devote all of your attention to solving the problem.

100% agree. I remember I had an on-call and our pagerduty started going off for a SEV-2 and naturally a lot of managers from teams that are affected are in there sweating bullets because their products/features/metrics are impacted. It can get pretty frustrating having so many people try to be cooks in the kitchen. We had a great manager who literally just moved us to a different call/meeting and he told us "ignore everything those people are saying; just stay focused and I'll handle them." Everyone's respect for our manager really went up from there.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#137
post #57
post #33

For #4 (divide and conquer), I've found `git bisect` helps a lot. If you have a known good commit and one of dozens or hundreds of commits after that is bad, this can help you identify the bad commit / code in a few steps. Here's a walk through on using it: https://nickjanetakis.com/blog/using-git-bisect-to-help-find... I jumped into a pretty big unknown code base in a live consulting call and we found the problem pr…

"git bisect" is why I maintain the discipline that all commits to the "real" branch, however you define that term, should all individually build and pass all (known-at-the-time) tests and generally be deployable in the sense that they would "work" to the best of your knowledge, even if you do not actually want to deploy that literal release. I use this as my #1 principle, above "I should be able to see every keystrok…

Sometimes you'll find a repo where that isn't true. Fortunately, git bisect has a way to deal with failed builds, etc: three-value logic. The test program that git bisect runs can return an exit value that means that the failure didn't happen, a different value that means that it did, or a third that means that it neither failed nor succeeded. I wrote up an example here:

https://speechcode.com/blog/git-bisect

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#138
post #48

Rule 0: Don't panic Really, that's important. You need to think clearly, deadlines and angry customers are a distraction. That's also when having a good manager who can trust you is important, his job is to shield you from all that so that you can devote all of your attention to solving the problem.

This is very underrated. Also an extension to this is don’t be afraid to break things further to probe. I often see a lot of devs mid level included panicking and thus preventing them to even know where to start. I’ve come to believe that some people just have an inherent intuition and some just need to learn it.

Yes its sometimes instinct takes over when your on the spot in a pinch but there are institutional things you can do to be prepared in advance that expand your set of options in the moment much like a pre-prepared firedrill playbook you can pull from also there are training courses like Kepner-Tregoe but you are right there are just some people who do better than others when _it's hitting the fan.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#139
post #56

Earlier quoted context omitted.

Slow is smooth and smooth is fast. If you don't have time to do it right, what makes you think there is time to do it twice?

"We have to do something !"

And this is something, so I’m doing it.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#140
post #33

For #4 (divide and conquer), I've found `git bisect` helps a lot. If you have a known good commit and one of dozens or hundreds of commits after that is bad, this can help you identify the bad commit / code in a few steps. Here's a walk through on using it: https://nickjanetakis.com/blog/using-git-bisect-to-help-find... I jumped into a pretty big unknown code base in a live consulting call and we found the problem pr…

Binary search rules. Being systematic about dividing the problem in half, determining which half the issue is in, and then repeating applies to non software problems quite well. I use the strategy all the time while troubleshooting issue with cars, etc.
Post reply on HN