Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What's the worst you've ever screwed up at work?

news.ycombinator.com

Post reply on HN
Print view View original

181–190 of 322 posts

Re: Ask HN: What's the worst you've ever screwed up at work?

#181

Post by azth »

sillysaurus2 wrote: post #69
yan wrote: post #21

Not the worst at all, but probably one I found most amusing. One of my jobs included some sys admin tasks (this wasn't the position, but we all did dev ops), among my other responsibilities. I spent half a day going through everything with the person responsible for most of the admin tasks at the time. She was an extremely dilligent and competent admin, did absolutely everything through configuration management and k…

She found out about it pretty quickly due to having syslog be a constant presence in one of her gnu screen windows I'm amazed that this is possible. How would I set something like that up? A realtime log of only the most significant events of a remote system? In fact, I'd like to take this opportunity of ignorance-admitting to ask the community for general linux/bsd sysadmin resources. What books should I read, or wh…

> I've been thinking of going through any MIT OCW on the subject

Are there any MIT OCW courses on this? I haven't come across any.

Re: Ask HN: What's the worst you've ever screwed up at work?

#182

Post by jjindev »

Once, as a relative UNIX newbie, I "cleaned up" a Sun box, until I had moved things I needed for boot off the boot partition. I got it all back, manually mounting partitions, and etc. but I was certainly in a cold sweat for about 15 minutes.

Perhaps the only lesson is "slow down."

Re: Ask HN: What's the worst you've ever screwed up at work?

#183

Post by kgermino »

BU_student wrote: post #159
patio11 wrote: post #93

I've only cried literal tears once in the last ten years, over business. Due to inattention while coding during an apartment move, I pushed a change to Appointment Reminder which was poorly considered. It didn't cause any immediate problems and passed my test suites, but the upshot is it was a time bomb that would inevitably bring down the site's queue worker processes and keep them down. Lesson #1: Don't code when y…

> Fortuitously, AR didn't have all that many customers at the time, so only 15 or so people were affected. Less than fortuitously, those 15 folks had 10 to 100 messages queued, each. Excuse me for caviling at your post, but "fortuitously" is a synonym for "accidentally", not "fortunately".

Well, a quick definition says "happening by accident or chance." Well I suspect he didn't know the proper definition (I certainly didn't) it's not that bad a choice.

Thanks for the comment though - I personally always appreciate a gracious correction.

Re: Ask HN: What's the worst you've ever screwed up at work?

#185

Post by pkinsky »

d0m wrote: post #129
patio11 wrote: post #93

I've only cried literal tears once in the last ten years, over business. Due to inattention while coding during an apartment move, I pushed a change to Appointment Reminder which was poorly considered. It didn't cause any immediate problems and passed my test suites, but the upshot is it was a time bomb that would inevitably bring down the site's queue worker processes and keep them down. Lesson #1: Don't code when y…

Can't agree more with Patrick, mine is formulated a bit differently: Lesson #1: Don't push code on Friday afternoon. Lesson #2: Beer, Code and Commit is totally fine. Just don't push! Wait until next day to review and push/deploy it..

I work for a Forex (have USD? Want JPY?) service provider that runs a 24-5 service (At any time of day during a weekday an exchange will be open somewhere, on some continent. They all close on weekends)

We exclusively push code on Fridays.

Re: Ask HN: What's the worst you've ever screwed up at work?

#186

Post by Phlarp »

asdasf wrote: post #166
sentenza wrote: post #80

Earlier quoted context omitted.

What position are you starting from? My old workplace was a university group where we (admins) were recruted from the available pool of PhD students. So I'm used to guiding people from "no knowledge" to "enough knowledge to be dangerous". The first step was to force the prospective admins to run a specific system on their "productive machine" and keep it in such conditions that _everything_ works. This way, a complet…

>If they were using Arch or Gentoo at home, If they are using gentoo, you should be finding someone else. Gentoo users are typically the most dangerous combination of profoundly ignorant, yet absurdly overconfident in their abilities. Seeing a bunch of autotools and gcc output scroll by does not teach you anything. But the mistaken reputation as an "advanced" distro makes people think that by using gentoo, they are t…

Because large groups of people can always be prejudged by which technologies they deploy!

Re: Ask HN: What's the worst you've ever screwed up at work?

#188

Post by steveklabnik »

jwdunne wrote: post #163
d0m wrote: post #129

Earlier quoted context omitted.

Can't agree more with Patrick, mine is formulated a bit differently: Lesson #1: Don't push code on Friday afternoon. Lesson #2: Beer, Code and Commit is totally fine. Just don't push! Wait until next day to review and push/deploy it..

Lesson 1 rings a mad bell. I distinctively remember a text from a colleague saying "yeah, I'm never launching anything on a Friday". As a general rule, the best time to launch is first thing on a Tuesday. Why Tuesday morning? We are also well aware of "Monday morning madness" too - pre-planning stress for Monday morning is akin to masochism.

I wonder if this is the rationale for Patch Tuesday.
Post reply on HN