Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

War story: the hardest bug I ever debugged

clientserver.dev

Post reply on HN
Print view View original

61–70 of 194 posts

Re: War story: the hardest bug I ever debugged

#61

Post by atq2119 »

friendzis wrote: post #56
atq2119 wrote: post #48

Earlier quoted context omitted.

I agree with your assessment of how stupid this is, but I'm not surprised. To be clear, there are good reasons for this different mode. The fuck-up is not testing it properly. These kinds of modes can be tested properly in various ways, e.g. by having an override switch that forces the chosen mode to be used all the time instead of using the default heuristics for switching between modes. And then you run your test s…

Isn't stochastic testing becoming more and more of a standard practice? Even if you have the hardware and time to run a full testsuite, you still want to add some randomness just to catch accidental dependencies between tests.

Maybe? I'd love to hear if there are some good tools for it that can be integrated into typical setups with Git repositories, Jenkins or GitHub Actions, etc.

Re: War story: the hardest bug I ever debugged

#62

Post by latexr »

It’s amusing how so many of the comments here are like “You think two days is hard? Well, I debugged a problem which was passed down to me by my father, and his father before him”. It reminds me of the Four Yorkshiremen sketch.

https://youtube.com/watch?v=sGTDhaV0bcw

The author’s “error”, of course, was calling it “the hardest bug I ever debugged”. It drives clicks, but comparisons too.

Re: War story: the hardest bug I ever debugged

#63

Post by Rygian »

> What can I even do from here as the newsletter author? Normally I like finding a teachable lesson. But it was 2 days of grueling debugging and somehow there aren’t any teachable lessons there.

A lesson to learn seems obvious to me: the V8 team did not communicate upfront sufficiently on the "oops our Math.abs() may return negative numbers, we fixed that in version X, be warned".

Which the V8 should be able to do in a "advisory for Google developers that work on high-performance client-side view rendering stuff" sort of weekly newsletter.

Re: War story: the hardest bug I ever debugged

#64

Post by onli »

toast0 wrote: post #7
kccqzy wrote: post #4

Earlier quoted context omitted.

I suppose the Google Doc team initially thought this would surely be a bug in their own code, not in Chrome or in V8, so it wouldn't help to bisect their own code. Nobody really begins to debug by blaming the compiler.

I mean... > It didn’t correspond to a Google Docs release. The stack trace added very little information. There wasn’t an associated spike in user complaints, so we weren’t even sure it was really happening — but if it was happening it would be really bad. It was Chrome-only starting at a specific release. That sounds like a Chrome bug. Or, at least, a bug triggered by a change in Chrome. Bisecting your code when the…

Not if it your job to solve the situation. It's not like it would be an easier task to dig into chrome and find the issue there.

Re: War story: the hardest bug I ever debugged

#66

Post by zdc1 »

My worst bug had me using statistics to try and correlate occurrence rates with traffic/time of day, API requests, app versions, Node.js versions, resource allocations, etc. And when that failed I was capturing Prod traffic for examination in Wireshark...

Turned out that Node.js didn't gracefully close TCP connections. It just silently dropped the connection and sent a RST packet if the other side tried to reuse it. Fun times.

Re: War story: the hardest bug I ever debugged

#67

Post by franktankbank »

danielodievich wrote: post #17

When I was 12 I was just learning stuff and wrote something in C, which crashed at unpredictable intervals and I could not explain it. I took it to my 14 year old uncle who was better than me at coding for help. Now mind you this is ~ 40 years ago but I seem to remember that Borland Turbo C (I still love that IDE blue color) had debugging with breakpoints (mind blowing!) which eventually led to "duh you didn't dispos…

I like that your uncle was only 2 years older than you.

Re: War story: the hardest bug I ever debugged

#68

Post by richardw »

Funkiest for me was a random crash in a C# app. No pattern whatsoever. No function or user role or part of the software or time of day. I had to learn crash dump analysis and bought my first Kindle books (on desktop, no kindle because I needed it asap), one of which had a trick to make a memory issue crash closer to the source, rather than leave it around to be stumbled over hours later. Which was the source of the randomness. Click button, crash. Move mouse, crash.

This had worked perfectly for many years but windows was upgraded underneath it, and some smartass had used clever tricks for a hover menu that didn’t work in a future (safer) version of the OS. A rarely triggered hover menu.

Thank you, authors of advanced windows debugging and advanced .net debugging.

Re: War story: the hardest bug I ever debugged

#69

Post by Taniwha »

In interviews I've never forced anyone to code, what I do is try to get them to tell me these sorts of war stories - I want to hear how you fixed it, why it was cooly bizarre, and I'm hoping for some enthusiasm when you talk about it.

I couldn't always get people to talk this way, but people who did usually worked out well

Re: War story: the hardest bug I ever debugged

#70

Post by sofixa »

nopurpose wrote: post #33

> It didn’t correspond to a Google Docs release. The stack trace added very little information. There wasn’t an associated spike in user complaints Where mere mortals can complain about Google product?

Apparently paying for Google One gives access to Google support. If that's worth anything I have no idea, but it exists
Post reply on HN