Live data from Hacker News

Why HN was down

news.ycombinator.com

161–170 of 303 posts

Re: Why HN was down

#161
post #73

Amazing that such a large percentage of debugging involves determining exactly what you are debugging. The definition of the problem, many times, is the solution. Might be a good time to mention Rubber Duck Debuggging. http://en.wikipedia.org/wiki/Rubber_duck_debugging

People at work are amazed when I successfully debug an issue over the phone. In reality, it amounts to 50% experience plus another 50% of Sherlock Holmes: "When you have eliminated the impossible, whatever remains, however improbable, must be the truth". Once you've identified what you're dealing with via a few strategic questions, it becomes simple quite rapidly.

Debugging is often best accomplished as a binary tree search aimed by familiarity/experience. Once you can put bounds on the search, it becomes possible to get the answer in just a few questions.

Totally agree.

Re: Why HN was down

#162

Earlier quoted context omitted.

There is a line from Futurama that perfectly applies ton a lot of debugging. Farnsworth: My God, is it really possible? Fry: It must be possible, it's happening. Fry: By the way, what's happening?

Is this the forward time machine episode?? I love futurama more than any man could love any tv show.

It is -- it's when they are observing the second big bang.

Re: Why HN was down

#163

Amazing that such a large percentage of debugging involves determining exactly what you are debugging. The definition of the problem, many times, is the solution. Might be a good time to mention Rubber Duck Debuggging. http://en.wikipedia.org/wiki/Rubber_duck_debugging

A few times a month, I'll look up at one of my colleagues and say, "hey, got a sec? I need to talk to the duck," and they know this means I'm going to talk to their head but they can basically keep doing what they're doing and nod occasionally. This serves several purposes: (1) It's less insane-sounding than actually talking to an inanimate object in an open work environment. (2) It actually feels better and forces m…

This is so true, probably about 90% of the time my colleagues call me over about a problem they are facing, explain in detail what the problem is and then eureka! Most of the time it would actually take me much longer to figure out the exact issue since I don't know the ins/outs and subtleties of the code but it's exactly as you say.

Of course, it makes me look really good cos I just "helped" them solve their issue :)

Re: Why HN was down

#164

Earlier quoted context omitted.

You could assert a limit on depth, perhaps. Then the cycle would still exist but after X number of comments, the rendering ends.

This is a reasonable solution. While it will (almost) never provide the correct result (it might print out a cycle of comments until X is reached, or it might cut off a very long but legitimate comment thread), it would provide a reasonable guarantee on this sort of problem not generating infinite pages.

At the risk of being accused of flame-baiting, I'd say it's the engineering solution rather than the mathematical one.. ;-)

For some reason I tend to be a fan of the "stick it in a secure box" rather than "get it right in the first place" approach..

Re: Why HN was down

#165
post #108
post #3

So what do you do to avoid this in the future? Do you stop doing surgery in the repl, or do you do the surgery with functions that check for cycles from now on?

This reminds me of the countless conversation I had with people after a crisis. What can we do to prevent this from happening again? What process can we put in place? What restriction needed to be tightened up? And that's how processes are born.

> And that's how processes are born.

Not necessarily. Processes are implemented by people, so they can break at any time.

The correct solution is more code, or less bad code.

Re: Why HN was down

#166
post #73

Amazing that such a large percentage of debugging involves determining exactly what you are debugging. The definition of the problem, many times, is the solution. Might be a good time to mention Rubber Duck Debuggging. http://en.wikipedia.org/wiki/Rubber_duck_debugging

People at work are amazed when I successfully debug an issue over the phone. In reality, it amounts to 50% experience plus another 50% of Sherlock Holmes: "When you have eliminated the impossible, whatever remains, however improbable, must be the truth". Once you've identified what you're dealing with via a few strategic questions, it becomes simple quite rapidly.

One of my favourite debugging tips has always been "give everyone full access to the folder/service" and see if the problem is "fixed". If so revert and now apply the correct permissions. I've seen this come up so many times although my superiors always complained "it's not the right way to do it", whilst I agree "every full access" is bad, this was for debugging purposes only!

Re: Why HN was down

#167
post #83

Earlier quoted context omitted.

There must be a rare personality type that never experiences this kind of overconfidence. Perhaps a less glamorous cousin to the Buddhist beginner's mind?

I find that if I am doing something "dangerous" (example might be using power tools) I have to say to my self "be careful this is dangerous" to avoid being on autopilot and making casual errors. Maybe a better example is the way you train yourself after you've picked up a box the wrong way and pull something to try to remember each and every time to watch your specific movements.

But at some point, you will become complacent, and it will taken a mistake to remind yourself again.

We've all done it. I shut down an NT4 production server because I was connected via remote desktop and clicked shutdown rather than log off. This was back in the day when there was no pop-up asking for reason you want to shut down and confirmation.

Luckily it was just our internal intranet server!

Re: Why HN was down

#168
post #104

Great postmortem and good lessons to learn here: * Don't manually modify database without a well-tested procedure and another pair of eyes * Don't leave persistent problems (e.g. memory problems) uninvestigated so that you miss new problems with similar symptoms * Don't push new code to production while operational problem is ongoing (unless it addresses the operational problem) I'm pretty sure I've repeated this exa…

* while you are displaying a tree keep track of the items you already displayed so you can detect a cycle

I think the assumption there was that it was safe, since the code disallowed this from happening, naturally.

Re: Why HN was down

#169
post #108

Earlier quoted context omitted.

This reminds me of the countless conversation I had with people after a crisis. What can we do to prevent this from happening again? What process can we put in place? What restriction needed to be tightened up? And that's how processes are born.

> And that's how processes are born. Not necessarily. Processes are implemented by people, so they can break at any time. The correct solution is more code, or less bad code.

And how do you get less bad code? Magic dust or process? My money would be on the latter, as in http://www.fastcompany.com/28121/they-write-right-stuff.

Re: Why HN was down

#170

Amazing that such a large percentage of debugging involves determining exactly what you are debugging. The definition of the problem, many times, is the solution. Might be a good time to mention Rubber Duck Debuggging. http://en.wikipedia.org/wiki/Rubber_duck_debugging

A few times a month, I'll look up at one of my colleagues and say, "hey, got a sec? I need to talk to the duck," and they know this means I'm going to talk to their head but they can basically keep doing what they're doing and nod occasionally. This serves several purposes: (1) It's less insane-sounding than actually talking to an inanimate object in an open work environment. (2) It actually feels better and forces m…

Working from home I tend to just write out my thoughts on a piece of paper. It works perfectly.
Post reply on HN