Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Debugging: Indispensable rules for finding even the most elusive problems (2004)

dwheeler.com

Post reply on HN
Print view View original

111–120 of 238 posts

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#111

Post by throwawayfks »

goshx wrote: post #101

> Quit thinking and look (get data first, don't just do complicated repairs based on guessing) From my experience, this is the single most important part of the process. Once you keep in mind that nothing paranormal ever happens in systems and everything has an explanation, it is your job to find the reason for things, not guess them. I tell my team: just put your brain aside and start following the flow of events ch…

I worked at a place once where the process was "Quit thinking, and have a meeting where everyone speculates about what it might be." "Everyone" included all the nontechnical staff to whom the computer might as well be magic, and all the engineers who were sitting there guessing and as a consequence not at a keyboard looking.

I don't miss working there.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#112

Post by toolslive »

ianmcgowan wrote: post #87
GuB-42 wrote: post #48

Rule 0: Don't panic Really, that's important. You need to think clearly, deadlines and angry customers are a distraction. That's also when having a good manager who can trust you is important, his job is to shield you from all that so that you can devote all of your attention to solving the problem.

There's a story in the book - on nuclear submarines there's a brass bar in front of all the dials and knobs, and the engineers are trained to "grab the bar" when something goes wrong rather than jumping right to twiddling knobs to see what happens.

"a good chess player sits on his hands" (NN). It's good advice as it prevents you from playing an impulsive move.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#113

Post by __MatrixMan__ »

> Check that it's really fixed, check that it's really your fix that fixed it, know that it never just goes away by itself

I wish this were true, and maybe it was in 2004, but when you've got noise coming in from the cloud provider and noise coming in from all of your vendors I think it's actually quite likely that you'll see a failure once and never again.

I know I've fixed things for people without without asking if they ever noticed it was broken, and I'm sure people are doing that to me also.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#114

Post by Izkata »

forrestthewoods wrote: post #93
jerf wrote: post #57

Earlier quoted context omitted.

"git bisect" is why I maintain the discipline that all commits to the "real" branch, however you define that term, should all individually build and pass all (known-at-the-time) tests and generally be deployable in the sense that they would "work" to the best of your knowledge, even if you do not actually want to deploy that literal release. I use this as my #1 principle, above "I should be able to see every keystrok…

> why I maintain the discipline that all commits to the "real" branch, however you define that term, should all individually build and pass all (known-at-the-time) tests and generally be deployable in the sense that they would "work" to the best of your knowledge, even if you do not actually want to deploy that literal release You’re spot on. However it’s clearly a missing feature that Git/Mercurial can’t tag diffs a…

If there's a way to identify those incomplete commits, git bisect does support "skip" - a commit that's neither good nor bad, just ignored.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#115

Post by ajross »

nottorp wrote: post #99
ajross wrote: post #83

Earlier quoted context omitted.

Not to complain about bisect, which is great. But IMHO it's really important to distinguish the philosophy and mindspace aspect to this book (the "rules") from the practical advice ("tools"). Someone who thinks about a problem via "which tool do I want" (c.f. "git bisect helps a lot"[1]) is going to be at a huge disadvantage to someone else coming at the same decisions via "didn't this used to work?"[2] The world is…

Just be careful to not contradict #3 “Quit thinking and look”.

Touché

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#116

Post by michalsustr »

forrestthewoods wrote: post #93
jerf wrote: post #57

Earlier quoted context omitted.

"git bisect" is why I maintain the discipline that all commits to the "real" branch, however you define that term, should all individually build and pass all (known-at-the-time) tests and generally be deployable in the sense that they would "work" to the best of your knowledge, even if you do not actually want to deploy that literal release. I use this as my #1 principle, above "I should be able to see every keystrok…

> why I maintain the discipline that all commits to the "real" branch, however you define that term, should all individually build and pass all (known-at-the-time) tests and generally be deployable in the sense that they would "work" to the best of your knowledge, even if you do not actually want to deploy that literal release You’re spot on. However it’s clearly a missing feature that Git/Mercurial can’t tag diffs a…

This is why we use squash like here https://docs.gitlab.com/ee/user/project/merge_requests/squas...

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#117

Post by stronglikedan »

toolslive wrote: post #112
ianmcgowan wrote: post #87

Earlier quoted context omitted.

There's a story in the book - on nuclear submarines there's a brass bar in front of all the dials and knobs, and the engineers are trained to "grab the bar" when something goes wrong rather than jumping right to twiddling knobs to see what happens.

"a good chess player sits on his hands" (NN). It's good advice as it prevents you from playing an impulsive move.

I have to sit on my hands at the dentist to prevent impulse moves.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#118

Post by tetha »

throwawayfks wrote: post #110
ianmcgowan wrote: post #87

Earlier quoted context omitted.

There's a story in the book - on nuclear submarines there's a brass bar in front of all the dials and knobs, and the engineers are trained to "grab the bar" when something goes wrong rather than jumping right to twiddling knobs to see what happens.

I read this book and took this advice to heart. I don't have a brass bar in the office, but when I'm about to push a button that could cause destructive changes, especially in prod, my hands reflexively fly up into the air while I double-check everything.

A weird, yet effective recommendation from someone at my last job: If it's a destructive or dangerous action in prod, touch both your elbows first. This forces ou to take the hands away from the keyboard, stop any possible auto-pilot and look what you're doing.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#119

Post by dehrmann »

wormlord wrote: post #77
heikkilevanto wrote: post #36

Some additional rules: - "It is your own fault". Always suspect your code changes before anything else. It can be a compiler bug or even a hardware error, but those are very rare. - "When you find a bug, go back hunt down its family and friends". Think where else the same kind of thing could have happened, and check those. - "Optimize for the user first, the maintenance programmer second, and last if at all for the c…

I always have the mindset of "its my fault". My Linux workstation constantly crashing because of the i9-13900k in it was honestly humiliating. Was very relieved when I realized it was the CPU and not some impossible to find code error.

Linux is like an abusive relationship in that way--it's always your fault.

Re: Debugging: Indispensable rules for finding even the most elusive problems (2004)

#120

Post by sitkack »

Cerium wrote: post #56
GuB-42 wrote: post #48

Rule 0: Don't panic Really, that's important. You need to think clearly, deadlines and angry customers are a distraction. That's also when having a good manager who can trust you is important, his job is to shield you from all that so that you can devote all of your attention to solving the problem.

Slow is smooth and smooth is fast. If you don't have time to do it right, what makes you think there is time to do it twice?

"We have to do something!"
Post reply on HN