The absolute worst scenario happened • Hacker News Board

Re: The absolute worst scenario happened

#241

Post by myself248 » Mon, Mar 22, 2021, 2:44 PM UTC

grumple wrote: post #145

arethuza wrote: post #41

Earlier quoted context omitted.

I contracted at one place for a bit where they shut everything down in each of their data centres once a year and power everything back up. When I was there this didn't go too well and they couldn't get one their data centres online again - failover to their other centres did work though. This was ~20 years ago in the finance sector.

That sounds like their testing worked out for them. Better than a random failure.

Yup. Have the problem when all the right people are awake and on-site to handle it.

I was in a building when someone inadvertently powered off the wrong equipment, which had been running for several years, and several of the power supplies failed to come back up. It was 1+1 redundant though, so we could quickly shuffle packs around to bring it back up without redundancy. Then, jogging through the building and asking if anyone had spares, we found a field tech in the lunchroom who had a pile of stuff in his van. Whole thing was back to 100% in less than an hour, and we let the beancounters sort out the field spares being used for office equipment.

If that same failure had happened during the overnight maintenance window (when volatile work was supposed to be performed), there certainly wouldn't have been the same resources around.

Re: The absolute worst scenario happened

#242

Post by throwaway14337 » Mon, Mar 22, 2021, 2:45 PM UTC

alistairSH wrote: post #89

Tangent Alert! I have a 3 months resignation period, so I can't even leave... Maybe it's my (US) American experience, but being unable to quit a job strikes me as a terrible policy. This guy(gal?) is really required to submit 3 months notice or face some sort of repercussions (financial, I assume)? Does this 3 month period apply in reverse (employer can't fire employees without a full 3 month warning)? Not that the A…

In a previous job it slowly dawned on me that the job was making me quite unhappy. This was about six months in. Only six more months until the "vesting cliff"! I managed to power through and get my stock options for the year.

So, while we don't have 3 months notice in the US, we certainly have similar ways to get stuck.

Re: The absolute worst scenario happened

#243

Post by devit » Mon, Mar 22, 2021, 2:45 PM UTC

Definitely not the "absolute worst", they merely lost all DNS records.

Re: The absolute worst scenario happened

#244

Post by skykooler » Mon, Mar 22, 2021, 2:45 PM UTC

kolbe wrote: post #186

NewLogic wrote: post #167

Earlier quoted context omitted.

The exact same thing is happening in Australia in regards to farmers and fruit picking labour. With international borders shut they are unable to exploit foreign workers and complain that young adults are refusing to relocate for $5 an hour hard labour jobs.

Farming is low margin. Farmers don’t get to set the price of their products, and they aren’t handed billions in capital to burn through or 1000x multiples on their profit in the public markets. When most businesses say they can’t hire someone and the reason is because they don’t pay enough, the underlying cause of their low pay is usually because that’s what the company can afford. Sure, farmers could pay $10/hr, but…

Sure, but they'd lose less money than if they didn't hire anyone and couldn't sell their crops at all.

Re: The absolute worst scenario happened

#245

Post by smnrchrds » Mon, Mar 22, 2021, 2:46 PM UTC

CydeWeys wrote: post #231

kolbe wrote: post #186

Earlier quoted context omitted.

Farming is low margin. Farmers don’t get to set the price of their products, and they aren’t handed billions in capital to burn through or 1000x multiples on their profit in the public markets. When most businesses say they can’t hire someone and the reason is because they don’t pay enough, the underlying cause of their low pay is usually because that’s what the company can afford. Sure, farmers could pay $10/hr, but…

People have to eat food. Food in aggregate has one of the most inelastic demand curves imaginable. If the price rises because the production costs increase, people will still buy food. They might buy different food, and you'll see a decrease in acreage given over to labor-intensive crops, but that's just the free market at work.

The thing is, if farmers in Australia increase wages and thus their product becomes more expensive, the product on supermarket shelves will not become more expensive. Rather, supermarkets will import food from another country. Food is, with few exceptions, a commodity. The producer does not set the price of the commodity, the market does.

I am telling this as an engineer working in the Canada's oil and gas sector. Since 2014, salaries in our sector have decreased and firms are struggling. "Just raise the price of oil" is not an actionable advice. The price of oil is not ours to set. It's the market's

Re: The absolute worst scenario happened

#246

Post by iJohnDoe » Mon, Mar 22, 2021, 2:47 PM UTC

I’m guessing the IT department reported to the CFO. Bean counters have zero IT knowledge but like to think they do. They also by default like to say no to any expenditures. Combine the two and you have a disaster waiting to happen.

IT reporting to finance/CFO is always the worst decision.

Re: The absolute worst scenario happened

#247

Post by geoduck14 » Mon, Mar 22, 2021, 2:47 PM UTC

tw04 wrote: post #88

I actually don't see any issue that isn't simply a matter of money. Everything they list that's an issue seems to revolve around the fact that the employees with domain knowledge for the system have left the company. Reach out to them and hire them on as contractors. If they left under bad terms because the business was a bunch of dicks, expect to pay 10x market rate. If this is truly "fix this or the business is out…

Let's not forget about the Mythical Man Month. Sure, specific people amd money can fix the problem, but we can't just assume that throwing people and money can fix this.

I would anticipate a mix of: the people who know where the right bodies are buried, the right incentive (yes, that includes money), but also people to look over the entire program, a designer to make sure it is designed correctly, an organized person to make sure it is properly documented, and someone to make sure it is adhered to in the future.

Re: The absolute worst scenario happened

#248

Post by smnrchrds » Mon, Mar 22, 2021, 2:53 PM UTC

intrasight wrote: post #139

drclau wrote: post #132

Earlier quoted context omitted.

Not really related to the subject of the article, but your comment reminds me of employers in Romania complaining they can't find people to hire, while they offer small salaries and people just leave the country for better paying jobs abroad. The solution to their problem is really simple: make better offers.

That is, of course, the solution everywhere that companies complain about not being able to hire skilled employees. Fund it and they will come.

This kinda sounds like saying "why are poor countries poor? Can't they just pay everyone more and then they won't be poor?" I get what you are saying but I don't think this will work if the problem is country-wide. You can't just magically increase everyone's wages in a country.

Re: The absolute worst scenario happened

#249

Post by ksec » Mon, Mar 22, 2021, 2:53 PM UTC

smnrchrds wrote: post #245

CydeWeys wrote: post #231

Earlier quoted context omitted.

People have to eat food. Food in aggregate has one of the most inelastic demand curves imaginable. If the price rises because the production costs increase, people will still buy food. They might buy different food, and you'll see a decrease in acreage given over to labor-intensive crops, but that's just the free market at work.

The thing is, if farmers in Australia increase wages and thus their product becomes more expensive, the product on supermarket shelves will not become more expensive. Rather, supermarkets will import food from another country. Food is, with few exceptions, a commodity. The producer does not set the price of the commodity, the market does. I am telling this as an engineer working in the Canada's oil and gas sector. Si…

It just shows most people working in Tech Industry, and specifically software ( or web ) development ( as oppose to Tech / software within Oil Industry or any other industry ) have absolutely zero idea on market or commodity trading.

Re: The absolute worst scenario happened

#250

Post by intrasight » Mon, Mar 22, 2021, 2:57 PM UTC

smnrchrds wrote: post #248

intrasight wrote: post #139

Earlier quoted context omitted.

That is, of course, the solution everywhere that companies complain about not being able to hire skilled employees. Fund it and they will come.

This kinda sounds like saying "why are poor countries poor? Can't they just pay everyone more and then they won't be poor?" I get what you are saying but I don't think this will work if the problem is country-wide. You can't just magically increase everyone's wages in a country.

1. The problem isn't country wide. 2. Don't have to increase everyone's - just those with a "shortage". Companies want a market economy - except when it comes to salaries.