Live data from Hacker News

The absolute worst scenario happened

reddit.com

161–170 of 566 posts

Re: The absolute worst scenario happened

#161

50% of companies who lose all of their data go out of business within 6 months. They clearly failed to test their backups with regularly-tested restores. And, a separate alternative system should've had multiple revisions of critical pieces of information like customers, assets, inventory, bank accounts, debts, and employees. A data recovery shop could attempt to forensically-reconstruct SMS numbers. Look for logs an…

They didn't fail to test their backups. They tested them, they didn't work, and then they said "this is fine".

Re: The absolute worst scenario happened

#162

Tangent Alert! I have a 3 months resignation period, so I can't even leave... Maybe it's my (US) American experience, but being unable to quit a job strikes me as a terrible policy. This guy(gal?) is really required to submit 3 months notice or face some sort of repercussions (financial, I assume)? Does this 3 month period apply in reverse (employer can't fire employees without a full 3 month warning)? Not that the A…

In my EU-centric worldview, the US system of at-will employment, and of two-week notices as the standard courtesy, is crazy. Three months is the typical notice period for tech workers and other highly-skilled jobs. One month is the legal minimum in Sweden, and certainly several other countries. Of course it goes both ways, in fact it's even more difficult for the employer to get rid of an employee. An employee doesn'…

In reality, you can walk out of any job in EU as well. Unless you have some specific clauses in your contract, the worst they can do is simply not pay you anything. There may come a lawsuit after, but I doubt it. You can also get a doctors note and go on sick leave if you feel some serious stress on the job.

Re: The absolute worst scenario happened

#163

Earlier quoted context omitted.

At least in Germany this goes both ways. The period can also be longer depending on your seniority. This is to give the company time to find a proper replacement for you and allow for a smoother handoff. Of course if both parties agree the grace period can be skipped or shortened. You usually don't get stuck with duds because there's a probationary period of (max.) 6 months in the beginning in which these protections…

There are certain things that you can outright be fired for - such as stealing, I believe. I think that showing up drunk for work might also be one of them?

There are more, and they are immediate. And nearly always lead to a lawsuit, because it is only for grave errors in judgement. i.e. theft, violence, endangering co-workers, fraud, misrepresentations to the board and such. So when this is triggered it is never a clean break. Lawsuits can come from both sides as the reason for the firing is grave and the argument doesn't end when a person was kicked out.

Re: The absolute worst scenario happened

#164

Earlier quoted context omitted.

I bet if the hospital is to pay Bob/Charlie $1M for 6 months, post tax to fix it they would gladly do it. Note: this does not mean "We will pay you up to a million dollars, Net 90, after an invoice is approved and signed off by fifteen people". This means "We will wire a deposit of $250,000 today. The rest will be wired every month in 24 hours after an invoice is issued" The reality is company execs like to talk abou…

For $1M they can get the new version of the same system installed and likely get a step by step walkthrough of the whole thing by any number of new engineers they want. Also it's a public hospital - for a fraction of $1M they could get people who's only job is to replicate that system running around with pen and paper - they wouldn't have $1M to just drop on a support issue the same day. In context of this case - if…

>For $1M they can get the new version of the same system installed and likely get a step by step walkthrough of the whole thing by any number of new engineers they want.

A new system isn't going to do them much good when they don't even know who their customers are. I doubt any customer is going to hang around for a year while you rebuild your system from scratch, ignoring the fact that if they're an MSP the customers may not even have their own copy of the data.

All of that is ignoring the likely contractual obligations they're likely under to maintain the data on their systems.

>Also it's a public hospital - for a fraction of $1M they could get people who's only job is to replicate that system running around with pen and paper - they wouldn't have $1M to just drop on a support issue the same day.

Hospital? He said they're an MSP on reddit:

>Yeah we're an awful MSP. I've been trying to find a new job for quite some time.

https://www.reddit.com/r/sysadmin/comments/ma4mwl/the_absolu...

Re: The absolute worst scenario happened

#165
post #136

Earlier quoted context omitted.

wow, it was so much better before. What the heck? Who made the decision to switch?

They want people using their app, for some reason I don't understand. Maybe to make ad-blocking harder by getting people off computers where it's easy (most mobile browsers don't support plugins, so it can't be about them).

Ironically, every time I click their “open in app” link it’s drips me straight to the App Store despite me having the app installed.

Re: The absolute worst scenario happened

#166
post #114

This reminds me of when the Danish mega-corp Maersk was attacked by ransomware and had all of their computers locked and decrypted with no access. All computers except one, which due to a power failure had been offline the whole time. This computer had the critical DNS information needed to restore the network. Only problem: it was in Africa, in a country that required Visa which would take weeks to apply for and rec…

Sounds very interesting, but the story seems to be behind a paywall. I found some other articles[0][1] related to the incident, however none of them seem to mention the flying of a hard drive across continents. 0: https://www.zdnet.com/article/ransomware-the-key-lesson-maer... 1: https://portswigger.net/daily-swig/when-the-screens-went-bla...

Or if you want the story as a podcast I can wholeheartedly recommend this episode (and the podcast in its entirety) of Darknet Diaries:

https://darknetdiaries.com/episode/54/

Re: The absolute worst scenario happened

#167
post #132
post #88

I actually don't see any issue that isn't simply a matter of money. Everything they list that's an issue seems to revolve around the fact that the employees with domain knowledge for the system have left the company. Reach out to them and hire them on as contractors. If they left under bad terms because the business was a bunch of dicks, expect to pay 10x market rate. If this is truly "fix this or the business is out…

Not really related to the subject of the article, but your comment reminds me of employers in Romania complaining they can't find people to hire, while they offer small salaries and people just leave the country for better paying jobs abroad. The solution to their problem is really simple: make better offers.

The exact same thing is happening in Australia in regards to farmers and fruit picking labour. With international borders shut they are unable to exploit foreign workers and complain that young adults are refusing to relocate for $5 an hour hard labour jobs.

Re: The absolute worst scenario happened

#168

Tangent Alert! I have a 3 months resignation period, so I can't even leave... Maybe it's my (US) American experience, but being unable to quit a job strikes me as a terrible policy. This guy(gal?) is really required to submit 3 months notice or face some sort of repercussions (financial, I assume)? Does this 3 month period apply in reverse (employer can't fire employees without a full 3 month warning)? Not that the A…

I guess US fire-at-will system should make contracting even more attractive option for good engineers: you do not get any social protection from being fired at any time unlike say Germany. One solid benefit less to justify pay difference.

Is it the case?

Re: The absolute worst scenario happened

#170

has anyone figured out who the company is by now? Surely there must be some reports of the outage by now?

> Surely there must be some reports of the outage by now?

Not everyone is AWS. There's a lot of obscure software providers nobody cares when they're down.

Post reply on HN