United Airlines System-Wide Computer Problem • Hacker News Board

Re: United Airlines System-Wide Computer Problem

#101

Post by juliangregorian » Wed, Jul 08, 2015, 4:51 PM UTC

briandear wrote: post #90

driverdan wrote: post #77

Earlier quoted context omitted.

Overall JetBlue and Virgin have been great. AA and US Air have been good. United is the worst.

AA destroyed a piece of baggage of mine. They denied responsibility. I sued and won. Then I couldn't collect because that judgement was essentially nullified because AA was going through bankruptcy. I have 1k status with United -- I fly internationally with them almost monthly. The problems I've had with United have been when bags have to be interlined to Brussels Airlines and occasionally (surprisingly) Lufthansa. I…

That's not accurate about the Air France pilots -- they were striking because the airline was moving to replace them with cheaper pilots.

Re: United Airlines System-Wide Computer Problem

#102

Post by pavel_lishin » Wed, Jul 08, 2015, 4:54 PM UTC

r00fus wrote: post #21

mnw21cam wrote: post #18

Earlier quoted context omitted.

Or just use words (8th July 2015). But yes, YYYY-MM-DD actually sorts correctly.

ISO all the way. Words have issues for different month languages (e.g. Auot vs. August), but otherwise is more elegant (no delimiter needed, e.g. 08JUL15)

http://www.telegraph.co.uk/news/ww1-archive/11721622/Daily-T...

Re: United Airlines System-Wide Computer Problem

#103

Post by geek_slop » Wed, Jul 08, 2015, 4:59 PM UTC

bitwize wrote: post #96

chinathrow wrote: post #10

Nice reminder about a "glitch" happening in one of the datacenters use at the airline I used to work. Went like this: Guy who shows around the new datacenter/ops guy demonstrated how the emergency power off works by lifting the protection plate. Protection plate unhinges suddenly and droppes onto emergency power off button. Hilarity ensues.

A data center where I work had a naked big red switch without a protection plate. Until I flagged up this lack and let the IT people know about it, it was a disaster waiting to happen. There's a plate on the switch now, praise Eris.

Similar situation here. A big red button on the wall in the basement data center - nobody knew what it was for and were afraid to turn it off. One day a night operator decided he was going to flip the switch and see what happened. It cut power to the entire datacenter, including the elevator systems. We spent a few days getting all the systems back online.

Had another instance where a pipe ran from the floor above right over the UPS. Construction workers on the first floor decided to poor some unusued paint into it (figured it was a drain pipe) and of course, we lost power again.

Re: United Airlines System-Wide Computer Problem

#104

Post by misterbwong » Wed, Jul 08, 2015, 5:19 PM UTC

ianhawes wrote: post #40

chinathrow wrote: post #10

Nice reminder about a "glitch" happening in one of the datacenters use at the airline I used to work. Went like this: Guy who shows around the new datacenter/ops guy demonstrated how the emergency power off works by lifting the protection plate. Protection plate unhinges suddenly and droppes onto emergency power off button. Hilarity ensues.

I remember one of the LiveJournal outages (IIRC) was caused by a dude in the datacenter thinking that the emergency power off button opened one of the doors.

Had a similar experience at an old company. One of the light switches in server room was placed next to a switch connected to the UPS of certain servers (I have no idea how/why this even came about).

Confused the heck out of us when we were trying to figure out why some of our servers went on UPS power randomly at night. Turns out we'd get the notifications on nights the cleaning crew decided to flick the lights off then on again.

Re: United Airlines System-Wide Computer Problem

#105

Post by rwestergren » Wed, Jul 08, 2015, 5:24 PM UTC

Perhaps someone wasn't following the bug bounty rules?

Re: United Airlines System-Wide Computer Problem

#106

Post by Xylakant » Wed, Jul 08, 2015, 5:34 PM UTC

temuze wrote: post #85

carnesen wrote: post #76

I worked on United's computer systems for a year (never that one though), and so I get nervous when I see a headline like that. True story: one of their systems still runs on a mainframe that has 9 bits in a byte!

Oh god stories like this scare me. One of my friends used to intern for a very large company that maintained software for flight control towers. His entire summer was spent writing bash tests for these old Fortran apps that kept planes from running into each other. Most of the mainframes still had tape. That's the code that keeps track of our planes.

I must admit I don't take that as a negative thing. Code that's so old had a lot of time for corner cases to be ironed out. If it's maintained properly it's probably less buggy than any rewrite, even though fortran is less enticing than rust as a language.

Re: United Airlines System-Wide Computer Problem

#107

Post by coob » Wed, Jul 08, 2015, 5:36 PM UTC

js2 wrote: post #56

chinathrow wrote: post #10

Nice reminder about a "glitch" happening in one of the datacenters use at the airline I used to work. Went like this: Guy who shows around the new datacenter/ops guy demonstrated how the emergency power off works by lifting the protection plate. Protection plate unhinges suddenly and droppes onto emergency power off button. Hilarity ensues.

At former managed-hosting startup, we would deploy a pair of EMC symmetrixes into our DC cage as storage for our DB tier, with volumes mirrored over the pair. While equipment was being moved into the cage one day, someone accidentally banged the power-switch protection plate on one of the EMCs. Fortunately the power switch had to be held depressed for several seconds to turn off the equipment. Unfortunately, the prot…

This is what happens when Homer Simpson works at your datacentre.

Re: United Airlines System-Wide Computer Problem

#108

Post by imgabe » Wed, Jul 08, 2015, 5:54 PM UTC

drzaiusapelord wrote: post #74

imgabe wrote: post #52

Earlier quoted context omitted.

Not that I know of. My particular flight was delayed because they had to replace a piece of the navigation equipment on the plane. This delay caused me to miss a connecting flight and that's how I ended up in an entirely different city.

This happens with every airline. No airline has magical planes that never break and they all have similar maintenance schedules on the exact same planes running the exact same workload. Blame Boeing for making a crappy plane if they keep breaking down.

I wasn't so annoyed that the flight got delayed and I got stranded in the wrong city, it was the way they handled it. United's response was:

1. Do nothing. You're stuck here. We'll get you on the next flight. Oh, the next available flight isn't for over a week. Sorry. No, we can't pay for your rental car.

2. Upon renting my own car, and writing to customer service to complain I got a $125 voucher for United. Great. Not enough to buy an entire ticket, so it just ensures I'll have to continue giving money to this airline that failed to deliver what I gave them money for in the first place.

3. After several weeks of emails back-and-forth with customer service finally a manager agrees to issue a reimbursement for the rental car as a one-time exception to their policy of never doing that. (but not the $7 worth of gas. I guess they just have to draw a line somewhere? Oh, and they also rescinded the voucher. No big deal since I'm not too keen to fly with them again anyway)

So, yeah. Sometimes shit goes wrong when you travel. Everybody knows that. How airlines act to fix it makes all the difference. If you hamstring all your customer service reps so they can't actually solve someone's problem, it makes something that's already annoying way more frustrating.

Re: United Airlines System-Wide Computer Problem

#109

Post by coderjames » Wed, Jul 08, 2015, 5:54 PM UTC

TeMPOraL wrote: post #83

sopooneo wrote: post #41

Earlier quoted context omitted.

But aren't the insurance companies smart enough to figure this out and start correcting their rates to be much higher?And if they actually have their acts together, wouldn't those same insurance companies start insisting on basic audits of their client's systems? I actually don't know about this stuff, so any correction of my thoughts is appreciated.

A cynic in me feels that someone will figure out the problems with cascade case and insure from failure of insurance companies to pay out insure money. Just like during 2008 financial crisis.

That's what existing reinsurance companies do, if I understand correctly. They insure the insurance companies.

Re: United Airlines System-Wide Computer Problem

#110

Post by scrumper » Wed, Jul 08, 2015, 6:05 PM UTC

scrumper wrote: post #6

That's twice in a short time now - 'coincidence' on Ian Fleming's scale. A third time is enemy action. Airlines do seem like a pretty juicy target for cyber war operations - you can cause a gigantic amount of disruption with a successful attack on a single system.

I don't see any convincing rebuttal. Why are airline systems not a juicy target? Why would a successful attack on the system not cause gigantic disruption? Why have these long-running, stable systems only recently begun failing so severely?

Yes, one is wise to attribute to incompetence over malice, but these systems are demonstrably not run by incompetents: they've been in operation for decades.