Boss: We have thousands of bad orders that must be fixed now! Me: No we don't. We have 121 bad orders. Boss: There are thousands of them! Me: No there aren't. There are exactly 121 of them. I'm sure. Boss: I'm not going to argue with you! Me: Good. Because you'd lose. I fixed 121 orders that night. The next day my login & password wouldn't work.
I might be dense, but I dont get it.
Ask HN: What's the worst you've ever screwed up at work?
111–120 of 322 posts
Re: Ask HN: What's the worst you've ever screwed up at work?
#112Boss: We have thousands of bad orders that must be fixed now! Me: No we don't. We have 121 bad orders. Boss: There are thousands of them! Me: No there aren't. There are exactly 121 of them. I'm sure. Boss: I'm not going to argue with you! Me: Good. Because you'd lose. I fixed 121 orders that night. The next day my login & password wouldn't work.
I might be dense, but I dont get it.
Re: Ask HN: What's the worst you've ever screwed up at work?
#113Re: Ask HN: What's the worst you've ever screwed up at work?
#114After the test was complete, I forgot to turn off the Adwords. (Such a silly mistake...) Nobody notices until our bill arrives from Google, and it's substantially higher than normal. When my coworker came to ask me about it, "are these your campaigns?!?" I just sank in my chair.
I think it cost the company $30k. I suppose it's not that much money in the grand scheme of things, but I felt very bad.
Re: Ask HN: What's the worst you've ever screwed up at work?
#115Boss: We have thousands of bad orders that must be fixed now! Me: No we don't. We have 121 bad orders. Boss: There are thousands of them! Me: No there aren't. There are exactly 121 of them. I'm sure. Boss: I'm not going to argue with you! Me: Good. Because you'd lose. I fixed 121 orders that night. The next day my login & password wouldn't work.
I might be dense, but I dont get it.
In theory you are supposed to be able to tell your boss he is wrong (and a good boss will appreciate this) in practice it depends on the boss.
Re: Ask HN: What's the worst you've ever screwed up at work?
#116Earlier quoted context omitted.
You didn't screw up here. The entire infrastructure, org chart, and policies that allowed you to accidentally modify a production database containing critical medical information screwed up. Blaming yourself here is like blaming yourself for being hurt after being told to drive a car with no seatbelt or brakes.
Sure there's plenty of blame to spread around, but I still would have felt terrible if someone had been hurt or killed. What system would you put in place to prevent this? The issue was that I connected to prod when I thought I was connecting to a test DB. We each had different credentials for prod vs everything else, but the SQL client remembered my username and password. Anyone with prod access could have made the…
Re: Ask HN: What's the worst you've ever screwed up at work?
#117chmod -R 0777 /* instead of chmod -R 0777
Re: Ask HN: What's the worst you've ever screwed up at work?
#118A local Subway franchise was the very first company that hired me. I was extremely young, shy, and intensely socially awkward, yet excited to join the workforce (as I had my eyes set on a Pentium processor). When I worked at Subway, the bread dough came frozen, but you would put loaves in a proofer, proof it for a certain amount of time, and then bake it. My first shift, however, got busy and I left several trays in…
Re: Ask HN: What's the worst you've ever screwed up at work?
#119Re: Ask HN: What's the worst you've ever screwed up at work?
#120The billing specs kept changing, as did the specs for the show itself. New price points, more plans, change the show interface, add another option here, etc. The plan had been to do a free preview show the day before to work out the kinks. That didn't happen.
The time leading up to show start was pretty tense, lots of updates, even a few last minute changes! Then the show actually started, brief relief. The chat system built in started deleting messages, one of those last minute feature changes had screwed up automatic old-message deletion. We had a fix though, update the JS, and bounce everyone out of the show and back in so the JS updates. Fixed!
Then the CEO pointed out that the quality just kept getting worse. Turns out that while the video player had both a numeric value and a string description for the different quality levels, it assumed they were in ascending order. So once it confirmed it could stream well at a given level, it automatically tried the next, which worked! Poor quality for everyone. Fixed, and another bounce.
Then it was over, time to go home. Back in the next day to finish off the billing code. I decided to approach it like a time card system. Traverse the logs in order, recording punch in time, when someone punches out, look up their punch-in times and set that user's time spent to the difference. Remove punch-in and out from the current record so they're not used again.
Now two facts from above added up to a pretty serious bug. 1) I _set_ the time spent to the difference between the two times. Not added, set. 2) We bounced everyone from the show twice to update their JS, and video player. So everyone had multiple join/parts.
I under-billed customers by tens of thousands of dollars.
Things I learned:
- Don't just argue that you need a trial run, make sure management understands the benefits. Why, not What.
- Duplicate billing code. After that a co-worker and I wrote two separate billing parsers for things, 1 designed to be different, not efficient.
- Give yourself ways to fix problems after they crop up. The bounce killed my billing code, but not doing it would have damaged the actual product (which later became a regular feature). Wish that thing had been my idea.