Every time we lift a pallet from the shipping room, the server times out (2006)

Yes, as pointed out on many debugging stories: problems are easy to find if you look in the right place (hint: knowing where to look is the hard part). Unfortunately for the people living them, there are wireless repeaters and pallets of shampoo that make looking in the right place harder. You might as well have said that Waldo is easy to find if you remove all of the other people on the page.

As for it being "a basic L2 loop that should have taken 10 minutes to track down." If it were a "normal" L2 loop, and he could have taken 10 minutes tracking it down; sure. But when it only happens for ~2 minutes at a time, getting 10 minutes of actual tracking in is hard.

Sure, having managed switches would have made finding it easier. But most debugging stories have something that could have made them easier if done beforehand.

About 10 years ago I was working for one of the Big 4 banks here in Australia, doing 3rd level support. An issue came up with, if memory serves, Siebel. Desktop issue, don't remember the details, but it was pretty serious - a team couldn't do the thing that they do.

I was on this thing for weeks. Hacking away in whatever that tool from sysinternals was called (Procmon?), monitoring calls at the process level, running multiple tests on multiple machines, the whole lot. It's the most complex troubleshooting I've ever done.

I found nothing.

The guys in the team were in an office a few miles away from me so one day I said, hey, I'm going to come down. I need to see this with my own eyes rather than over a remote connection.

I went down there. We started up the desktop. They launched the software. They clicked the button to do The Thing That Wasn't Working.

And it worked. It just did the thing it was meant to do.

And the problem never came back.

Around 2000: a colleague told me about his slow PC, rattling noises, random crashes. i did what any responsible workmate would do: told him to call IT support, I'm a web-designer (well actually I was a frontend, backend developer, database guy and project manager but devision of labor in webdevelopment didn't happen yet so my job title was webdesigner, anyway...)!

5 weeks of various visits by the IT support later, including full replacement of all PC hardware and software - still random crashes.

so one day my direct boss tells me to take a look as the corporate IT support "couldn't do it". i just went over there and encounterd a desktop PC covered in refrigorater-sticky-note-magnets.....

turns out he removed them every time "so that the IT guys have better access to the computer"...

I used to work as a fire alarm and security system engineer, and my boss was a dodgy maverick, picking up contracts here and there. He somehow had me responsible for a shopping centre that was about 150 miles away ,and it housed the cctv for the entire town centre (cctv is big in the UK). I was on a salary, so didn't get overtime. One Saturday morning I got a call saying all the cctv system was down, and that I needed to got sort it out ASAP!

Drove for 4 hours, and got to the control room, where I was confronted with a totally dead system - all the monitors were on, but all black screens. I asked the guy what happened, mostly to stall for a bit as I'd never been on site before and there were no schematics available, and I was expecting to be there for days.

He told me that he'd gone to put the kettle on, come back and when he came back, it was all dead. I asked where the kettle was, and he looked bemused,but showed me. Instead of it being in a rest room, it was located on a shelf behind the equipment rack. I took a closer look, and saw that it was plugged into the power strip that was built into the rack, and the cable was twisted round other power cables, one of which was... The demultiplexer that carried all the cctv camera signals from round the town, and which had been pulled out (iec mains lead, easily done). Pushed it back in by 5mm, went round the front to see all the monitors on... Said "thanks for that" to the guy, got the paperwork signed and left. Spent a total of 15 minutes on site, and over 8 driving as the traffic was terrible on the way home.

A newly hired employee had the strangest problem with his ethernet when he started.

Whenever he plugged in ethernet it would only give him 10mbps. But if he unplugged and replugged it in. It'd switch over to 1000.

What was more strange. On off. Router or hardware on off. Reboot. Nothing would switch it over to higher speed. We tested it a dozen times. But if you plug it in. Unplug then plug in a 2nd time. Viola 1gbps connection made. Always 10mbps the first time. Tried 4 wires and 3 different switches on multiple computers. Problem is his computer. Even reformatted computer and tried different hardware ethernet ports. Never fully resolved why it always need to be replugged in twice though for faster connection.