Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

British Airways: All flights cancelled amid IT crash

bbc.com

Post reply on HN
Print view View original

141–150 of 255 posts

Re: British Airways: All flights cancelled amid IT crash

#141

Post by ivan_gammel »

tudorconstantin wrote: post #132
raesene9 wrote: post #96

Earlier quoted context omitted.

The key is incentives. An IT outsourcer's goal is generally to fulfill their contractual obligations (to the letter not the spirit) for as little money as possible. So if something like security or reliability isn't clearly stated in contracts (and it's extremely difficult to do that well) it is disregarded. An outsourcing company is not going to spend money it doesn't have to, simple as that. SAAS like gsuite is an…

I've worked in outsourcing for 9 years in Romanian companies as a software developer. I had the impression that the goal was to have the customer satisfied, not to fulfill a contract (as long as the customer was also reasonable, of course) and for pragmatic reasons: a happy customer pays you longer. My longest project was for 4 years, with the same customer. And my former employer keeps working on that project, 2 yea…

To make customer happy you don't have to build reliable software. Corporations are not machines, there's a lot of people with their own interests, who make important decisions. You have to make these people happy, first of all. They can be bribed, convinced or fooled to achieve the desired outcome for your business. And if something bad happens, you have to dodge the bullet together, because you are in the same boat. So you blame other contractors, hackers or the guy who left the customer company some time ago, and do a lot of other things to convince the customer management, that it's not your fault, you actually saved them from bigger losses somehow and you can help to build better software. And they won't be happier, but they will think that you are good guy trying to solve their problems. And that's the only thing you need to sell another contract.

Re: British Airways: All flights cancelled amid IT crash

#142

Post by gaius »

funkyy wrote: post #124
davidf18 wrote: post #62

By using IT outsourcing in India (Tata?), Poland, BA and its parent is revealing some important information: They are cutting corners to the point of risking airline operations. The question is, where else are they unnecessarily cutting corners? Are they cutting unnecessary corners on airplane maintenance? Other quality airlines have outsourced IT or part of their IT operations, but they are careful about choosing th…

I think you should not put IT specialists from India and Poland in the one bag. Polish engineers are sometimes a bit late when it comes to the new technologies, but are very good compared to ones from, for example, Ireland.

Polish engineers are sometimes a bit late when it comes to the new technologies, but are very good compared to ones from, for example, Ireland.

It doesn't matter. They are no smarter than BA's original British engineers, but with decades less experience on those systems. Even if they were smarter - and in general the talent prefers to work for real companies, not bodyshops, so that's unlikely - they can't compensate for the lack of real experience.

Only a person who believes experience doesn't matter would sign an outsourcing deal.

Re: British Airways: All flights cancelled amid IT crash

#143

Post by raesene9 »

fauigerzigerk wrote: post #106
raesene9 wrote: post #96

Earlier quoted context omitted.

The key is incentives. An IT outsourcer's goal is generally to fulfill their contractual obligations (to the letter not the spirit) for as little money as possible. So if something like security or reliability isn't clearly stated in contracts (and it's extremely difficult to do that well) it is disregarded. An outsourcing company is not going to spend money it doesn't have to, simple as that. SAAS like gsuite is an…

I don't see why an IT outsourcing company would not have the greatest incentive to take security and reliability extremely seriously. Is there anything worse for an IT service provider than being blamed for a massive IT outage at a global corporation? This is headline news. And I don't see any difficult contractual issues at all. On the contrary. A massive outage, by definition, means that the contractual obligation…

Security costs money, reliability costs money. If it's not in your contract, you don't pay for it.

if there's any outage that is because the customer didn't ask you do do something that's their fault not yours.

For example would you as an IT outsourcer pay for a redundant datacentre if your contract didn't call for it?

Would you patch all your systems immediately even if it caused availability issues if it wasn't explicitly outlined in the contract?

would you explain to your shareholders that your profits were lower this year because you undertook activities not specified in your contracts because they were good for the security and reliability of the services you managed...?

When outsourcing contracts are bid for there's a common experience of lower costs win. that inevitably leads to items that aren't strictly required being excluded.

Re: British Airways: All flights cancelled amid IT crash

#145

Post by pan1c »

davidf18 wrote: post #62

By using IT outsourcing in India (Tata?), Poland, BA and its parent is revealing some important information: They are cutting corners to the point of risking airline operations. The question is, where else are they unnecessarily cutting corners? Are they cutting unnecessary corners on airplane maintenance? Other quality airlines have outsourced IT or part of their IT operations, but they are careful about choosing th…

I live in Atlanta - and as someone mentions below there have been pretty big IT outages at Delta etc too.

But - without any evidence whatsoever wholesale blaming of Indian and Polish companies seem somewhat unfair. Until, we have a postmortem and know this in more detail - it appears that you just have a axe to grind.

Re: British Airways: All flights cancelled amid IT crash

#146

Post by otterley »

davidf18 wrote: post #62

By using IT outsourcing in India (Tata?), Poland, BA and its parent is revealing some important information: They are cutting corners to the point of risking airline operations. The question is, where else are they unnecessarily cutting corners? Are they cutting unnecessary corners on airplane maintenance? Other quality airlines have outsourced IT or part of their IT operations, but they are careful about choosing th…

We don't know yet what failed, let alone why. What sense does it make to jump to conclusions so quickly?

And what difference does it make whether the error -- if indeed the root cause was an error, which hasn't yet been established -- was made by someone in India vs. someone in Poland vs. someone in the UK?

Re: British Airways: All flights cancelled amid IT crash

#147

Post by nomercy400 »

tomschlick wrote: post #31

Airlines should band together and form working group to redevelop the old system in modern open source tech. Yes it would probably take 5+ years to develop and roll out, but they can't keep maintaining these 50 year old mainframes that cost them tens of millions a year in downtime.

Yes, they should build an even bigger system with more requirements from each of the airlines, and have the same consulting company implement it. What could possibly go wrong?

Re: British Airways: All flights cancelled amid IT crash

#148

Post by raesene9 »

fauigerzigerk wrote: post #139
sgt101 wrote: post #119

Earlier quoted context omitted.

Here is the list of things that are worse for an IT outsourcer. 1. Being more expensive than the other guy.

I doubt that anyone wants to compete solely on price. It's a margins game.

have you been involved in many IT outsourcing agreements? Cost is always a major factor in procurement, and in IT where there can be a strong market for lemons, it's often the key factor.

A company that's more expensive and can't clearly demonstrate in a bid scenario the positive impact of that increase in costs, will lose bids, a lot.

Re: British Airways: All flights cancelled amid IT crash

#149

Post by raesene9 »

tudorconstantin wrote: post #132
raesene9 wrote: post #96

Earlier quoted context omitted.

The key is incentives. An IT outsourcer's goal is generally to fulfill their contractual obligations (to the letter not the spirit) for as little money as possible. So if something like security or reliability isn't clearly stated in contracts (and it's extremely difficult to do that well) it is disregarded. An outsourcing company is not going to spend money it doesn't have to, simple as that. SAAS like gsuite is an…

I've worked in outsourcing for 9 years in Romanian companies as a software developer. I had the impression that the goal was to have the customer satisfied, not to fulfill a contract (as long as the customer was also reasonable, of course) and for pragmatic reasons: a happy customer pays you longer. My longest project was for 4 years, with the same customer. And my former employer keeps working on that project, 2 yea…

I wouldn't say that an outsourcing company wants to do a bad job, of course they don't, but things that are often invisible in the product (like security or reliability) will likely get less focus in a cost sensitive environment.

For example, completing a full security development lifecycle can add 10%+ to the costs of the final product. that's not a cost that a company will incur unless they have to.

In a bid for work, everyone says "we take security seriously" and the client probably can't evaluate the difference between someone who really takes all the necessary steps, and someone who pays lip service to that concept.

So the cheap company (who doesn't spend that 10% extra) looks just as good as the one that does, and they're likely cheaper.

Guess who's more likely to get the work (all other things being equal)...

Re: British Airways: All flights cancelled amid IT crash

#150

Post by a-dub »

Looks like their new (as of 2 yrs) CEO comes from the low-cost airline world: https://en.wikipedia.org/wiki/%C3%81lex_Cruz_(businessman) so that may be a hint...

Although, I've seen this sort of thing before... Usually it starts with a middle management hiring of some alpha-male business asshole who is driven to advance his career and thinks that because he can use a spreadsheet that he's qualified to run IT. He'll then go on to sell upper management on some kind of ridiculous story straight out of some bullshit CIO magazine about how consolidating all the existing best-in-class systems into one system will cut costs and open opportunities for building and mining customer data to increase revenue. He'll get the greenlight and a shit-ton of capital, and then he has to make the decision about whether to build or buy. That question is irrelevant, as our intrepid hero has no idea what the fuck he is doing and will fuck things up regardless of which path he takes. Once the "new system" is fully half baked, he then shoves it out all over the company in some ridiculous balls to the wall no going back roll out plan. Subsequently there will be huge problems, massive lines, pissed off customers, pissed off employees... but this is where our intrepid hero really shines. His mastery of the art of bullshit successfully deflects all blame from himself and his incompetence onto the users/operators (eg, people who are responsible for revenue generating businesses) of his monstrosity. Somehow it is forgotten that outages never happened before and all the revenue loss and customer ill-will is blamed on the operators for not having well-tuned disaster recovery plans in place for manual operation.

Of course the only disasters they ever dealt with were a result of his incompetence, and in a stunning feat of failing upwards, he's destined to helm the company (and then likely others) within a few short years.

Post reply on HN