Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

flightglobal.com

Post reply on HN
Print view View original

121–130 of 311 posts

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#121

Post by bitwize »

ericjmorey wrote: post #99
ronsor wrote: post #93

Earlier quoted context omitted.

> The general engineering challenge, is how we deal with little-used little-seen functionality (definitely thinking of fault-handling, but there may be other cases) that is totally reasonable when put in place, but has not aged well, and nobody has noticed or realized it, and even if they did it might be hard to convince anyone it's a priority to improve, and the longer you wait the more expensive. The solution to th…

Which company deployed a chaos monkey deamon on their systems? Seemed to improve resiliency when I read about it.

The chaos monkey is there to remind you to always mount a scratch monkey.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#122

Post by CrimsonCape »

throw0101a wrote: post #82
CrimsonCape wrote: post #65

Earlier quoted context omitted.

C dev: "You are telling me that the three digit codes are not globally unique??? And now we have to add more bits to the struct?? That's going to kill our perfectly optimized bit layout in memory! F***! This whole app is going to sh**"

> C dev: "You are telling me that the three digit codes are not globally unique??? They are understood not to be. They are generally known to be regionally unique. The "DVL" code is unique with-in FAA/Transport Canada control, and the "DVL" is unique with-in EASA space. There are pre-defined three-letter codes: * https://en.wikipedia.org/wiki/IATA_airport_code And pre-defined four-letter codes: * https://en.wikipedia…

It seems like tasking a software engineer to figure this out when the industry at large hasn't figured this out just isn't fair.

Best I can see (using Rust) is a hashmap on UTF-8 string keys and every code in existence gets inserted into the hash map with an enum struct based on the code type. So you are forced to switch over each enum case and handle each case no matter what region code type.

It becomes apparent that the problem must be handled with app logic earlier in the system; to query a database of codes, you must also know which code and "what type" of code it is. Users are going to want to give the code only, so there's some interesting mis-direction introduced; the system has to somehow fuzzy match the best code for the itinerary. Correct me if i'm wrong, but the above seems like a mandatory step in solving the problem which would have caught the exception.

I echo other comments that say that there's probably 60% more work involved than your manager realizes.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#123

Post by zer8k »

nightowl_games wrote: post #90
steeeeeve wrote: post #15

You know there's a software engineer somewhere that saw this as a potential problem, brought up a solution, and had that solution rejected because handling it would add 40 hours of work to a project.

I don't know that and I don't like this assumption that only 'managers' make mistakes, or that software engineers are always right. I thinks needlessly adversarial, biased and largely incorrect.

Spoken like a manager.

Look, when you're barking orders at the guys in the trenches who, understandably in fear for their jobs, do the stupid "business-smart" thing, then it is entirely the fault of management.

I can't tell you how many times just in the last year I've been blamed-by-proxy for doing something that was decreed upon me by some moron in a corner office. Everything is an emergency, everything needs to be done yesterday, everything is changing all the time because King Shit and his merry band of boot-licking middle managers decide it should be.

Software engineers, especially ones with significant experience, are almost surely more right than middle managers. "Shouldn't we consider this case?" is almost always met with some parable about "overengineering" and followed up by a healthy dose of "that's not AGILE". I have grown so tired of this and thanks to the massive crater in job mobility most of us just do as we are told.

It's the power imbalance. In this light, all blame should fall on the manager unless it can be explicitly shown to be developer problems. The addage "those who can, do, and those who can't, teach" applies equally to management.

When it's my f@#$U neck on the line and the only option to keep my job is do the stupid thing you can bet I'll do the stupid thing. Thank god there's no malpractice law in software.

Poor you - only one of our jobs is getting shipped overseas.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#124

Post by anigbrowl »

andkenneth wrote: post #98
pvitz wrote: post #70

Earlier quoted context omitted.

Yes, it looks like they should have written "NM" instead of "nm".

No one is using nanometers in aviation navigation. Quite a few aviation systems are case insensitive or all caps only so you can't always make a distinction. In fact, if you say "miles", you mean nautical miles. You have to use "sm" to mean statute miles if you're using that unit, which is often used for measuring visibility.

Indeed, but you can easily imagine a software glitch over what looks like a single location but which the computer sees as two separate ones.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#125

Post by jug »

andyjohnson0 wrote: post #67
jp57 wrote: post #9

FYI: nm = nautical miles, not nanometers.

Even though I knew this was about aviation, I still read nm as nanometres. Now I'm wondering what this says about how my brain works.

Yeah, I went into the article thinking this because I expected someone had created waypoints right on top of each other and in the process also somehow generating the same code for them.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#126

Post by elteto »

nightowl_games wrote: post #90
steeeeeve wrote: post #15

You know there's a software engineer somewhere that saw this as a potential problem, brought up a solution, and had that solution rejected because handling it would add 40 hours of work to a project.

I don't know that and I don't like this assumption that only 'managers' make mistakes, or that software engineers are always right. I thinks needlessly adversarial, biased and largely incorrect.

Agreed. And most of the people with these attitudes have never written actual safety critical code where everything is written to a very detailed spec. Most likely the designers of the system thought of this edge case and required adding a runtime check and fatal assertion if it was ever encountered.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#127

Post by d1sxeyes »

mjevans wrote: post #80
jmvoodoo wrote: post #11

So, essentially the system has a serious denial of service flaw. I wonder how many variations of flight plans can cause different but similar errors that also force a disconnect of primary and secondary systems. Seems "reject individual flight plan" might be a better system response than "down hard to prevent corruption" Bad assumption that a failure to interpret a plan is a serious coding error seems to be the root…

Reject the flight plan would be the last case scenario, but where it should have gone without other options rather than total shutdown. CORRECT the flight plan, by first promoting the exit/entry points for each autonomous region along the route, validating the entry/exit list only, and then the arcs within, would be the least errant method.

You can’t just reject or correct the flight plan, you’re a consumer of the data. The flight plan was valid, it was the interpretation applied by the UK system which was incorrect and led to the failure.

There are a bunch of ways FPRSA-R can already interpret data like this correctly, but there were a combination of 6 specific criteria that hadn’t been foreseen (e.g. the duplicate waypoints, the waypoints both being outside UK airspace, the exit from UK airspace being implicit on the plan as filed, etc).

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#128

Post by kortilla »

zer8k wrote: post #123
nightowl_games wrote: post #90

Earlier quoted context omitted.

I don't know that and I don't like this assumption that only 'managers' make mistakes, or that software engineers are always right. I thinks needlessly adversarial, biased and largely incorrect.

Spoken like a manager. Look, when you're barking orders at the guys in the trenches who, understandably in fear for their jobs, do the stupid "business-smart" thing, then it is entirely the fault of management. I can't tell you how many times just in the last year I've been blamed-by-proxy for doing something that was decreed upon me by some moron in a corner office. Everything is an emergency, everything needs to be…

Your attitude is super antagonistic and your relationship with management is not representative of the industry. I recommend you consider a different job or if this pattern repeats at every job that you reflect on how you interact with managers to improve.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#129

Post by cduzz »

jp57 wrote: post #9

FYI: nm = nautical miles, not nanometers.

I was wondering; it seemed like if the to airports were 36000 angstroms apart (3600 nanometers), it'd be reasonable to give them the same airport code since they'd be pretty much on top of each other.

I've also seen "DANGER!! 12000000 μVolts!!!" on tiny little model railroad signs.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#130

Post by akira2501 »

wyldfire wrote: post #43
FateOfNations wrote: post #3

Good news: the system successfully detected an error and didn't send bad data to air traffic controllers. Bad News: the system can't recover from an error in an individual flight plan, bringing the whole system down with it (along with the backup system since it was running the same code).

> he system can't recover from an error in an individual flight plan, bringing the whole system down with it From the system's POV maybe this is the right way to resolve the problem. Could masking the failure by obscuring this flight's waypoint problem have resulted in a potentially conflicting flight not being tracked among other flights? If so, maybe it's truly urgent enough to bring down the system and force the h…

> obscuring this flight's waypoint problem have resulted in a potentially conflicting flight not being tracked among other flights?

Flights are tracked by radar and by transponder. The appropriate thing to do is just flag the flight with a discontinuity error but otherwise operate normally. This happens with other statuses like "radio failure" or "emergency aircraft."

It's not something you'd see on a commercial flight, but a private IFR flight (one with a flight plan), you can actually cancel your IFR plan mid flight and revert to VFR (visual flight rules) instead.

Some flights take off without an IFR clearance as a VFR flight, but once airborne, they call up ATC and request an IFR clearance already en route.

The system is vouchsafing where it does not need to.

Post reply on HN