Ask HN: What's the worst you've ever screwed up at work? • Hacker News Board

Re: Ask HN: What's the worst you've ever screwed up at work?

#281

Post by angersock » Thu, Jan 30, 2014, 5:45 AM UTC

Phlarp wrote: post #186

asdasf wrote: post #166

Earlier quoted context omitted.

>If they were using Arch or Gentoo at home, If they are using gentoo, you should be finding someone else. Gentoo users are typically the most dangerous combination of profoundly ignorant, yet absurdly overconfident in their abilities. Seeing a bunch of autotools and gcc output scroll by does not teach you anything. But the mistaken reputation as an "advanced" distro makes people think that by using gentoo, they are t…

Because large groups of people can always be prejudged by which technologies they deploy!

Are Gentoo users really that large a group of people?

In numbers, I mean, not form factor.

Re: Ask HN: What's the worst you've ever screwed up at work?

#282

Post by rajacombinator » Thu, Jan 30, 2014, 5:46 AM UTC

drdeadringer wrote: post #127

I dropped two units of equipment, ~$1.5Mil a piece. Each unit was dropped in separate incidents. No damage at all, but management didn't care. I blamed myself despite mitigating factors such as impossible schedules, vicious multi-tasking "to compensate", and less-than-ideal support equipment. At the time, I didn't handle it very well but I ended up living through it -- first job//assignment ever in the worst environm…

what kind of equipment costs 1.5Mil and can be carried/dropped by a single person?

Re: Ask HN: What's the worst you've ever screwed up at work?

#283

Post by NamTaf » Thu, Jan 30, 2014, 6:47 AM UTC

benburleson wrote: post #162

bengarvey wrote: post #48

I poured gasoline into the tractor's radiator instead of the gas tank. Thankfully, someone stopped me before I turned it on.

Is this really that bad? IANAMechanic, but if the coolant system is sealed, shouldn't the gas just work similar to antifreeze?

Diesel doesn't require a spark to combust, just head and pressure (and oxygen).

Re: Ask HN: What's the worst you've ever screwed up at work?

#284

Post by michaelmior » Thu, Jan 30, 2014, 7:13 AM UTC

kordless wrote: post #95

sillysaurus2 wrote: post #69

Earlier quoted context omitted.

She found out about it pretty quickly due to having syslog be a constant presence in one of her gnu screen windows I'm amazed that this is possible. How would I set something like that up? A realtime log of only the most significant events of a remote system? In fact, I'd like to take this opportunity of ignorance-admitting to ask the community for general linux/bsd sysadmin resources. What books should I read, or wh…

Splunk: http://splunk.com/ . Free download, free to use up to X many GB.

Also Splunk Storm https://www.splunkstorm.com/

Re: Ask HN: What's the worst you've ever screwed up at work?

#285

Post by scott_karana » Thu, Jan 30, 2014, 8:36 AM UTC

ambiate wrote: post #29

zimpenfish wrote: post #2

Many years ago, when I was but a fresh faced idiot, the partition that contained the mSQL database which had All The Data filled up. I moved it into /tmp because there was plenty of space. On a Solaris box. Hilarity ensued when we next rebooted it.

For those who don't know, solaris uses tmpfs for /tmp. It is a virtual memory/swap based file system. Anything in /tmp is actually temporary if the machine reboots/powers off.

I like setting this up on Linux machines too. There are tons of ephemeral files that get written there, depending on your usage case, and I'd rather not waste the IO for writing pids to lockfiles. Disk is cheap, but RAM is fast and cheap. :)

Re: Ask HN: What's the worst you've ever screwed up at work?

#286

Post by scott_karana » Thu, Jan 30, 2014, 8:44 AM UTC

brey wrote: post #236

hcarvalhoalves wrote: post #130

Happened to a colleague: it was the end of the day, and we were packing up to leave. He used Ubuntu on his notebook, so he typed "shutdown -h now" on his shell prior to closing the lid. Seconds later he's groaning, having noticed it was a SSH session to the production server... It wouldn't be a big deal, wasn't for the fact it was an EC2 instance, and back then halting the instance was equivalent to deleting it perma…

molly-guard is an good way to prevent this - forces you to type in the hostname of the machine you are trying to halt / restart etc. http://manpages.ubuntu.com/manpages/trusty/man8/molly-guard....

Excellent if you have complete and thorough control of every server you touch, but if you don't, it could be dangerous to rely on. Murphy's Law means it'll be that one dang machine that gets shut down...

Personally, I'd think that training this is a lot more portable.

  # hostname
  
  # shutdown -h now

Re: Ask HN: What's the worst you've ever screwed up at work?

#287

Post by mattwritescode » Thu, Jan 30, 2014, 9:37 AM UTC

Deleted a database table and not the temporary table I was working on.

Re: Ask HN: What's the worst you've ever screwed up at work?

#288

Post by blueskin_ » Thu, Jan 30, 2014, 10:08 AM UTC

Meant to reboot my desktop...

  [root@importantServer]# reboot

"Hmm... This is taking a while..."

Re: Ask HN: What's the worst you've ever screwed up at work?

#289

Post by blueskin_ » Thu, Jan 30, 2014, 10:13 AM UTC

asdasf wrote: post #166

sentenza wrote: post #80

Earlier quoted context omitted.

What position are you starting from? My old workplace was a university group where we (admins) were recruted from the available pool of PhD students. So I'm used to guiding people from "no knowledge" to "enough knowledge to be dangerous". The first step was to force the prospective admins to run a specific system on their "productive machine" and keep it in such conditions that _everything_ works. This way, a complet…

>If they were using Arch or Gentoo at home, If they are using gentoo, you should be finding someone else. Gentoo users are typically the most dangerous combination of profoundly ignorant, yet absurdly overconfident in their abilities. Seeing a bunch of autotools and gcc output scroll by does not teach you anything. But the mistaken reputation as an "advanced" distro makes people think that by using gentoo, they are t…

Obligatory link to http://funroll-loops.info/

Re: Ask HN: What's the worst you've ever screwed up at work?

#290

Post by cpach » Thu, Jan 30, 2014, 11:13 AM UTC

bedhead wrote: post #270

lsc wrote: post #260

Earlier quoted context omitted.

Yeah. This was... my biggest takeaway from running my own company. I struck out on my own because I thought the business people who ran the companies I worked for were unethical idiots, and being not-an-idiot, I could do better. Turns out? Nope. I also find that the arguments I had with my bosses come around again, from my employees. I still don't really understand how business works - but I've learned enough to unde…

All true, but that still doesn't answer what possesses someone to pack their cell phone in a box .

Seriously guys, take it easy.

My question is: Who never makes mistakes? I certainly do not belong in that set.

And I like Patrick's candid business anecdotes. Snarky comments like this might disencourage people from writing useful advice here. Which would be very unfortunate.