Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debugging: Indispensable rules for finding even the most elusive problems (2004)

dwheeler.com

Post reply on HN
Print view View original

101–110 of 238 posts

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#101

Post by goshx »

> Quit thinking and look (get data first, don't just do complicated repairs based on guessing)

From my experience, this is the single most important part of the process. Once you keep in mind that nothing paranormal ever happens in systems and everything has an explanation, it is your job to find the reason for things, not guess them.

I tell my team: just put your brain aside and start following the flow of events checking the data and eventually you will find where things mismatch.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#102

Post by Hackbraten »

knlb wrote: post #9

I wrote a fairly similar take on this a few years ago (without having read the original book mentioned here) -- https://explog.in/notes/debugging.html Julia Evans also has a very nice zine on debugging: https://wizardzines.com/zines/debugging-guide/

I love Julia Evans’ zine! Bought several copies when it came out, gave some to coworkers and donated one to our office library.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#103

Post by pbalau »

goshx wrote: post #101

> Quit thinking and look (get data first, don't just do complicated repairs based on guessing) From my experience, this is the single most important part of the process. Once you keep in mind that nothing paranormal ever happens in systems and everything has an explanation, it is your job to find the reason for things, not guess them. I tell my team: just put your brain aside and start following the flow of events ch…

Acquire a rubber duck. Teach the duck how the system works.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#105

Post by SoftTalker »

jerf wrote: post #57
nickjj wrote: post #33

For #4 (divide and conquer), I've found `git bisect` helps a lot. If you have a known good commit and one of dozens or hundreds of commits after that is bad, this can help you identify the bad commit / code in a few steps. Here's a walk through on using it: https://nickjanetakis.com/blog/using-git-bisect-to-help-find... I jumped into a pretty big unknown code base in a live consulting call and we found the problem pr…

"git bisect" is why I maintain the discipline that all commits to the "real" branch, however you define that term, should all individually build and pass all (known-at-the-time) tests and generally be deployable in the sense that they would "work" to the best of your knowledge, even if you do not actually want to deploy that literal release. I use this as my #1 principle, above "I should be able to see every keystrok…

I do bisecting almost as a last resort. I've used it when all else fails only a few times. Especially as I've never worked on code where it was very easy to just build and deploy a working debug system from a random past commit.

Edit to add: I will study old diffs when there is a bug, particularly for bugs that seem correlated with a new release. Asking "what has changed since this used to work?" often leads to an obvious cause or at least helps narrow where to look. Also asking the person who made those changes for help looking at the bug can be useful, as the code may be more fresh in their mind than in yours.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#106

Post by epolanski »

nickjj wrote: post #33

For #4 (divide and conquer), I've found `git bisect` helps a lot. If you have a known good commit and one of dozens or hundreds of commits after that is bad, this can help you identify the bad commit / code in a few steps. Here's a walk through on using it: https://nickjanetakis.com/blog/using-git-bisect-to-help-find... I jumped into a pretty big unknown code base in a live consulting call and we found the problem pr…

Bisection is also useful when debugging css.

When you don't know what is breaking that specific scroll or layout somewhere in the page, you can just remove half the DOM in the dev tools and check if the problem is still there.

Rinse and repeat, it's a basic binary search.

I am often surprised that leetcode black belts are absolutely unable to apply what they learn in the real world, neither in code nor debugging which always reminds me of what a useless metric to hire engineers it is.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#107

Post by lanstin »

Tepix wrote: post #40

My first rule for debugging debutants: Don't be too embarassed to scatter debug logmessages in the code. It helps. My second rule: Don't forget to remove them when you're done.

My rule for a long time has been anytime I add a print or log, except for the first time I am writing some new cide with tricky logic, which I try not to do, never delete it. Lower it to the lowest possible debug or trace level but if it was useful once it will be useful again, even if only to document the flow thru the code on full debug.

The nicest log package I had would always count the number of times a log msg was hit even if the debug level meant nothing happened. The C preprocessor made this easy, haven't been able to get a short way to do this counting efficiently in other languages.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#108

Post by Ragnarork »

seanwilson wrote: post #91
Ragnarork wrote: post #49

Earlier quoted context omitted.

Writing tests isn't free but writing non-regression tests for bugs that were actually fixed is one of the best test cases to consider writing right away, before the bug is fixed. You'll be reproducing the bug anyway (so already consider how to reproduce). You'll also have the most information about it to make sure the test is well written anyway, after building a mental model around the bug. Writing tests isn't free,…

> Writing tests isn't free, I agree, but in this case a good chunk of the cost of writing them will have already been paid in a way. Some examples that come to mind are bugs to do with UI interactions, visuals/styling, external online APIs/services, gaming/simulation stuff, and asynchronous/thread code, where it can be a substantial effort to write tests for, vs fixing the bug that might just be a typo. This is reall…

Definitely agree with you on the fact that there are tests which are complicated to write and will take effort.

But I think all other things considered my impression still holds, and that I should maybe rather say they're easier to write in a way, though not necessarily easy.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#109

Post by SoftTalker »

BWStearns wrote: post #84

> Check the plug I just spent a whole day trying to figure out what was going on with a radio. Turns out I had tx/rx swapped. When I went to check tx/rx alignment I misread the documentation in the same way as the first. So, I would even add "try switching things anyways" to the list. If you have solid (but wrong) reasoning for why you did something then you won't see the error later even if it's right in front of yo…

Yes the human brain can really be blind when its a priori assumptions turn out to be wrong.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#110

Post by throwawayfks »

ianmcgowan wrote: post #87
GuB-42 wrote: post #48

Rule 0: Don't panic Really, that's important. You need to think clearly, deadlines and angry customers are a distraction. That's also when having a good manager who can trust you is important, his job is to shield you from all that so that you can devote all of your attention to solving the problem.

There's a story in the book - on nuclear submarines there's a brass bar in front of all the dials and knobs, and the engineers are trained to "grab the bar" when something goes wrong rather than jumping right to twiddling knobs to see what happens.

I read this book and took this advice to heart. I don't have a brass bar in the office, but when I'm about to push a button that could cause destructive changes, especially in prod, my hands reflexively fly up into the air while I double-check everything.
Post reply on HN