Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

flightglobal.com

Post reply on HN
Print view View original

271–280 of 311 posts

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#271

Post by dx034 »

amiga386 wrote: post #86

This is old news, but what's new news is that last week, the UK Civil Aviation Authority openly published its Independent Review of NATS (En Route) Plc's Flight Planning System Failure on 28 August 2023 https://www.caa.co.uk/publication/download/23337 (PDF) Let's look at point 2.28: "Several factors made the identification and rectification of the failure more protracted than it might otherwise have been. These inclu…

> The Level 2 engineer was rostered on-call and therefore was not available on site at the time of the failure. Having exhausted remote intervention options, it took 1.5 hours for the individual to arrive on-site to perform the necessary full system re-start which was not possible remotely.

Which shows that sometimes, remote isn't a viable option. If you have very critical infrastructure, it's advisable to have people physically very close to the data center so that they can access the servers if all other options fail. That's valid for aviation as well as for health care, banks, etc. Remote staff just isn't enough in these situations.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#272

Post by [deleted] »

gopher_space wrote: post #175
crubier wrote: post #155

Earlier quoted context omitted.

Possibly even just waypoint = waypointsMatches[0] Without even mentioning that waypointsMatches might have multiple elements. This is why I always consider [0] to be a code smell. It doesn't have a name afaik, but it should.

Race condition?

[deleted]

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#273

Post by CaptainFever »

crubier wrote: post #155
convivialdingo wrote: post #116

I guarantee that piece of code has a comment like /* This should never happen */ if (waypoints.matchcount > 2) {

Possibly even just waypoint = waypointsMatches[0] Without even mentioning that waypointsMatches might have multiple elements. This is why I always consider [0] to be a code smell. It doesn't have a name afaik, but it should.

Silently ignoring conditions where there are multiple or zero elements?

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#274

Post by cryptonector »

> Just 20s elapsed between the receipt of the flightplan and the shutdown of both FPRSA-R systems, causing all automatic processing of flightplan data to cease and forcing reversion to manual procedures.

That's quite a DoS vulnerability...

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#275

Post by cryptonector »

wyldfire wrote: post #43
FateOfNations wrote: post #3

Good news: the system successfully detected an error and didn't send bad data to air traffic controllers. Bad News: the system can't recover from an error in an individual flight plan, bringing the whole system down with it (along with the backup system since it was running the same code).

> he system can't recover from an error in an individual flight plan, bringing the whole system down with it From the system's POV maybe this is the right way to resolve the problem. Could masking the failure by obscuring this flight's waypoint problem have resulted in a potentially conflicting flight not being tracked among other flights? If so, maybe it's truly urgent enough to bring down the system and force the h…

There is no need to shut down the whole system just because of one flight plan that the system was able to reject. Canceling (or forcing manual updates to) one flight plan is a lot better than canceling 1,500 flights.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#276

Post by throw0101a »

rounce wrote: post #247
skissane wrote: post #131

Earlier quoted context omitted.

> They are understood not to be. They are generally known to be regionally unique. Then why aren’t they namespaced? Attach to each code its issuing authority, so it is obvious to the code that DVL@FAA and DVL@EASA are two different things? Maybe for backward compatibility/ human factors reasons, the code needs to be displayed without the namespace to pilots and air traffic controllers, but it should be a field in the…

Given airport codes are namespaced by ICAO, it seems a bit of an oversight that the same scheme hasn’t been extended to other infrastructure like VORs and waypoints for systems exactly like this one, for cases when uniqueness is required.

> […] like VORs and waypoints for systems exactly like this one, for cases when uniqueness is required.

AIUI, most VORs (and in the past, NDBs) are located at airports. And there is an airport at Deauville (DOL/LFRG):

* https://en.wikipedia.org/wiki/Deauville–Normandie_Airport

But the beacon is elsewhere, apparently in some random(?) field:

* https://www.google.com/maps/place/49°18'38.0%22N+0°18'45.0%2...

* https://opennav.com/navaid/FR/DVL/DEAUVILLE+SAINT+GATIEN

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#277

Post by cryptonector »

akira2501 wrote: post #130
wyldfire wrote: post #43

Earlier quoted context omitted.

> he system can't recover from an error in an individual flight plan, bringing the whole system down with it From the system's POV maybe this is the right way to resolve the problem. Could masking the failure by obscuring this flight's waypoint problem have resulted in a potentially conflicting flight not being tracked among other flights? If so, maybe it's truly urgent enough to bring down the system and force the h…

> obscuring this flight's waypoint problem have resulted in a potentially conflicting flight not being tracked among other flights? Flights are tracked by radar and by transponder. The appropriate thing to do is just flag the flight with a discontinuity error but otherwise operate normally. This happens with other statuses like "radio failure" or "emergency aircraft." It's not something you'd see on a commercial flig…

The appropriate thing to do was to reject the flight plan (remember, the flight plan is processed before the flight starts, and anyways there were hours over U.S. in which the flight could have been diverted if manual resolution was not possible), not to let the flight continue with the apparent discontinuity, nor to shutdown the whole system.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#278

Post by cryptonector »

mannykannot wrote: post #137
aftbit wrote: post #81

Earlier quoted context omitted.

It seems fundamentally unreasonable for the flight processing system to entirely shut itself down just because it detected that one flight plan had corrupt data. Some degree of robustness should be expected from this system IMO.

It does not seem reasonable when you put it like that, but when could it be said with confidence that it only affected just one flight plan? I get the impression that it is only in hindsight that this could be seen to be so. On the face of it, this was just an ordinary transatlantic flight like thousands of others, with no reason to think there was anything unusual about it to make it more vulnerable than the rest -…

Shutting down a flight control system might have other knock-on effects on flight safety. Even if it merely only grounded flights not yet in the air, the resulting confusion might lead to manual mistakes and/or subsequent air lane congestion that might cause collisions.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#279

Post by rounce »

throw0101a wrote: post #276
rounce wrote: post #247

Earlier quoted context omitted.

Given airport codes are namespaced by ICAO, it seems a bit of an oversight that the same scheme hasn’t been extended to other infrastructure like VORs and waypoints for systems exactly like this one, for cases when uniqueness is required.

> […] like VORs and waypoints for systems exactly like this one, for cases when uniqueness is required. AIUI, most VORs (and in the past, NDBs) are located at airports. And there is an airport at Deauville (DOL/LFRG): * https://en.wikipedia.org/wiki/Deauville–Normandie_Airport But the beacon is elsewhere, apparently in some random(?) field: * https://www.google.com/maps/place/49°18'38.0%22N+0°18'45.0%2... * https://o…

There are plenty of navaids that aren’t associated with an airport, there’s probably around a dozen like this just south-east England.

Re: Air traffic failure caused by two locations 3600nm apart sharing 3-letter code

#280

Post by jrochkind1 »

thereddaikon wrote: post #262
Thorrez wrote: post #258

Earlier quoted context omitted.

Isn't what you're proposing exactly what led to this being a major problem? The automated systems disabled themselves, so people had to use the manual way, which was much less efficient, and 1,500 flights had to be cancelled.

They are referring to air crew procedures, not ATC. WHen the crew of an aircraft encounter a failure that doesn't have a common simple response, they consult a procedure book. This is something professional crews are well acquainted with and used to. The problem in the article was with the air traffic control system. They did not have a proper failback procedure and it caused major disruptions.

Their fallback procedure was "do it the manual way". Just like for the pilots. They thought it was a proper one...
Post reply on HN