Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The absolute worst scenario happened

reddit.com

Post reply on HN
Print view View original

381–390 of 566 posts

Re: The absolute worst scenario happened

#381

Post by jart »

danjac wrote: post #151
scandox wrote: post #35

How people handle this level of disaster is always very revealing. In every case I've ever experienced someone kept their head and a solution was found. Usually there was reputational damage but surprisingly most times the actual business bounced back. But it was always the actions of someone in particular in those first hours that made the difference. Thinking you're fucked is unhelpful, because you start talking ab…

Looking through the comments, it appears concerns were raised for years and management refused to allow the necessary changes to save money. While you can and should be the cool head in the room, it's a route to burnout if you end up being the hero who has to keep saving a dysfunctional organization and management from the consequences of its own failures. By all means do your best fix the immediate problem but get t…

Reading comments https://www.reddit.com/r/sysadmin/comments/ma4mwl/the_absolu... it sounds like he ran into a kernel panic, maybe because cosmic radiation messed with their full disk encryption because they went cheap on the ecc ram, and then he got discouraged, he probably asked management for the resources to rewrite the whole system from scratch, they said no, and then he sat on his hands collecting paychecks for two years until the thing he knew would happen finally happened. How else can you explain an MSP not having a copy of their zone file? That's their core business. It's also no secret in the MSP world how frequently these DNS servers get hacked. I read about it all the time. Especially on Reddit where you can just sense the pervasive fear. So you'd think that'd give a sysadmin a moment's pause to think, OK, have I scp'd my zone file in case that happens to me?

Re: The absolute worst scenario happened

#382

Post by dctoedt »

sufehmi wrote: post #349
ycombinete wrote: post #322

Earlier quoted context omitted.

Funnily enough it made me sick to my stomach. I manage a small company and the idea of being extorted in such a blasé and psychopathic fashion terrifies me to even think about.

"extorted" This is pure slander The company can avoid this if they only care to maintain the documentation / wiki left behind by said person When they failed to do even just basic proper operations, it's their own fault. I'm a business owner and a consultant, and I too charge more for these kind of asshole clients, because of the extra stress. But nowadays I just try to avoid them, it's just too damaging for your own…

> This is pure slander

It's libel (written) not slander (oral); "defamation" is the generic noun.

Re: The absolute worst scenario happened

#383

Post by fctorial »

jiggawatts wrote: post #133
mysterydip wrote: post #113

Earlier quoted context omitted.

honestly, "password" would be my first guess.

Oh come now, there are password complexity policies in place in any modern software! Try: password123.

Need capital letters and symbols:

Password@123

Re: The absolute worst scenario happened

#384

Post by 4e530344963049 »

notyourday wrote: post #147
4e530344963049 wrote: post #90

Earlier quoted context omitted.

A "firefighting" consultancy might be fun, no? And you would give people a lot of time off!

> A "firefighting" consultancy might be fun, no? It is... interesting. I have been doing this over last 4 years. The biggest problem, as I pointed out before, is the disconnect between what companies claim they would do to fix the problems vs. what they would actually do. So ~90% of selling is repeatedly explaining the same things to different people whose jobs are in the eminent danger of being restructured/depriori…

Do you mind if I email you with some questions? Thank you.

Re: The absolute worst scenario happened

#385

Post by sideshowb »

gambiting wrote: post #357
cogman10 wrote: post #355

Earlier quoted context omitted.

I mean... what could the other side of the story actually be? "We tried to hire back an employee to build something that we didn't train or try and figure out ourselves and a deadline forced us to pay a large lump sum to that employee". No matter how you slice it, that's really shitty planning on the businesses side. They should have spent the time and money learning how to maintain their shit. By not doing it, they…

I imagine if it came down to it they could try to pin it on him and say he didn't provide documentation he said he did, and that's why this entire 10M project now can't be delivered. Luckily for him they decided it's easier to just pay the fee.

Op says he did provide documentation but they lost it (and hadn't been backing up) after he left

Re: The absolute worst scenario happened

#386

Post by rangerelf »

ycombinete wrote: post #322
tofuahdude wrote: post #309

Earlier quoted context omitted.

I love this with all my heart. You did exactly the right thing at every step.

Funnily enough it made me sick to my stomach. I manage a small company and the idea of being extorted in such a blasé and psychopathic fashion terrifies me to even think about.

> ...I manage a small company...

And you identify with this band of miscreants? That's embarrassing.

Re: The absolute worst scenario happened

#387

Post by whatever1 »

ycombinete wrote: post #322
tofuahdude wrote: post #309

Earlier quoted context omitted.

I love this with all my heart. You did exactly the right thing at every step.

Funnily enough it made me sick to my stomach. I manage a small company and the idea of being extorted in such a blasé and psychopathic fashion terrifies me to even think about.

Thinking about business continuity is literally your job. Not your engineers’ job.

Re: The absolute worst scenario happened

#388

Post by Mauricebranagh »

ACS_Solver wrote: post #105
alistairSH wrote: post #89

Tangent Alert! I have a 3 months resignation period, so I can't even leave... Maybe it's my (US) American experience, but being unable to quit a job strikes me as a terrible policy. This guy(gal?) is really required to submit 3 months notice or face some sort of repercussions (financial, I assume)? Does this 3 month period apply in reverse (employer can't fire employees without a full 3 month warning)? Not that the A…

In my EU-centric worldview, the US system of at-will employment, and of two-week notices as the standard courtesy, is crazy. Three months is the typical notice period for tech workers and other highly-skilled jobs. One month is the legal minimum in Sweden, and certainly several other countries. Of course it goes both ways, in fact it's even more difficult for the employer to get rid of an employee. An employee doesn'…

I agree (I am from the UK) when first saw US professional workers complaining about > two weeks my first thought was wow you must have a really low status job.

Re: The absolute worst scenario happened

#389

Post by Mauricebranagh »

Eremotherium wrote: post #115
alistairSH wrote: post #89

Tangent Alert! I have a 3 months resignation period, so I can't even leave... Maybe it's my (US) American experience, but being unable to quit a job strikes me as a terrible policy. This guy(gal?) is really required to submit 3 months notice or face some sort of repercussions (financial, I assume)? Does this 3 month period apply in reverse (employer can't fire employees without a full 3 month warning)? Not that the A…

At least in Germany this goes both ways. The period can also be longer depending on your seniority. This is to give the company time to find a proper replacement for you and allow for a smoother handoff. Of course if both parties agree the grace period can be skipped or shortened. You usually don't get stuck with duds because there's a probationary period of (max.) 6 months in the beginning in which these protections…

Would not these count as gross misconduct?

Re: The absolute worst scenario happened

#390

Post by gambiting »

sideshowb wrote: post #385
gambiting wrote: post #357

Earlier quoted context omitted.

I imagine if it came down to it they could try to pin it on him and say he didn't provide documentation he said he did, and that's why this entire 10M project now can't be delivered. Luckily for him they decided it's easier to just pay the fee.

Op says he did provide documentation but they lost it (and hadn't been backing up) after he left

Of course, but that wouldn't stop them from saying otherwise, suing and binding him in prelonged and expensive legal battle, would it?
Post reply on HN