Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UK air traffic control meltdown

jameshaydon.github.io

Post reply on HN
Print view View original

441–450 of 459 posts

Re: UK air traffic control meltdown

#441

Post by sheepshear »

zaphar wrote: post #149
d1sxeyes wrote: post #147

Earlier quoted context omitted.

Well, if the primary is known not to be in a good state, you might as well fail over and hope that the issue was a fried disk or a cosmic bit flip or something. The real safety feature is the 4 hour lead time before manual processing becomes necessary. One of the key safety controls in aviation is “if this breaks for any reason, what do we do”, not so much “how do we stop this breaking in the first place”.

I'm no aviation safety controls expert but it seems to me that there are two types of controls that should be in place: 1. Process controls: What do we do when this breaks for any reason. 2. Engineering controls: What can we do to keep this from breaking in the first place? Both of them seem to be somewhat essential for a truly safe system.

Everyone uses slightly different terminology and groups things differently but this will give you the gist.

https://en.m.wikipedia.org/wiki/Hierarchy_of_hazard_controls

Re: UK air traffic control meltdown

#442

Post by epolanski »

seabass-labrax wrote: post #436
epolanski wrote: post #309

Earlier quoted context omitted.

Quite few non major airports are still heavily pen and paper reliant methods to some degree. An example are islands that serve few flights per week and can't justify heavy update investments. Airplanes are generally spaced by hours and you need to do your math about where the airplanes are by hand. But again there's so little planes that risks are minimal.

Indeed, but the set of aerodromes that are large enough to have a tower controller but not large enough to have their own radar surveillance is shrinking all the time. Radar is getting cheaper and what with ADS-C and TA/RA, a big reason to have ATC even without radar is vanishing (namely that of preventing collisions close to the airport). Oceanic control is probably the closest you can get nowadays to routine ATC wi…

I think an island in the middle of the Atlantic that is mostly used for refueling is exactly that kind of airport.

Can't remember the name but I'm quite sure it belongs to Portugal.

Re: UK air traffic control meltdown

#443

Post by WalterBright »

fransje26 wrote: post #410
WalterBright wrote: post #322

> the backup system applied the same logic to the flight plan with the same result Oops. In software, the backup system should use different logic. When I worked at Boeing on the 757 stab trim system, there were two avionics computers attached to the wires to activate the trim. The attachment was through a comparator, that would shut off the authority of both boxes if they didn't agree. The boxes were designed with:…

As as side note, too bad they knowingly didn't reuse such an approach for the MAX..

The MAX system relied on the pilot remembering the stab trim cutoff switch and what it was for.

Re: UK air traffic control meltdown

#444

Post by WalterBright »

f1shy wrote: post #388
WalterBright wrote: post #322

> the backup system applied the same logic to the flight plan with the same result Oops. In software, the backup system should use different logic. When I worked at Boeing on the 757 stab trim system, there were two avionics computers attached to the wires to activate the trim. The attachment was through a comparator, that would shut off the authority of both boxes if they didn't agree. The boxes were designed with:…

I would be very interested in knowing which languages were used. Do you know which were? Thanks

One of them was Pascal. This was around 1980 or so.

Re: UK air traffic control meltdown

#445

Post by WalterBright »

jojobas wrote: post #354
WalterBright wrote: post #322

> the backup system applied the same logic to the flight plan with the same result Oops. In software, the backup system should use different logic. When I worked at Boeing on the 757 stab trim system, there were two avionics computers attached to the wires to activate the trim. The attachment was through a comparator, that would shut off the authority of both boxes if they didn't agree. The boxes were designed with:…

Wouldn't trim be an number of which a significant tolerance is permissible at any given time? Or does "agree" mean "within a preset tolerance"?

Naturally, any comparator would have some slack in it to account for variations. Even CPU internals have such slack, that's why there's a "clock" to synchronize things.

Re: UK air traffic control meltdown

#446

Post by WalterBright »

bdavbdav wrote: post #424
WalterBright wrote: post #379

Earlier quoted context omitted.

I neglected to mention there was a third party that reviewed the algorithms to verify they weren't the same. Nothing is perfect, though, and the pilot is the backup for failure of that system. I.e. turn off the stab trim system.

Is this still the case for simple algorithms?

I don't know as much about modern avionics.

Re: UK air traffic control meltdown

#447

Post by NBJack »

deadfish wrote: post #269
NBJack wrote: post #255

Earlier quoted context omitted.

Yes. It would cascade into: Changes in how ATCs operate Changes in how pilots operate Changes in how airplanes receive these instructions (including the flight software itself, safety systems, etc.) Changes in how airplanes are tested Changes in how pilots are trained Etc. In this case, the refactoring requires changes to hardware, software, training, manufacturing, and humans.

Pretty sure that is still not the meaning of refactoring. As I understand it refactoring should mean no changes to the external interface but changes to how it is implemented internally.

We can pontificate on how to define the scope of a system here. I will only state that, from the perspective of a consumer, you could consider this a Service on which the interface of find flight, book flight, etc. would appear to be the same while the connections internal to each of the above modules would have to account for the change.

Functionally, I suppose it's the equivalent of upgrading an ID field that was originally declared as an unsigned 32 bit integer to a wider 64 bit representation. We may not be changing anything fundametal in the functionality, but every boundary interface, protocol, and storage mechanism must now suffer through a potentially painful modification.

Re: UK air traffic control meltdown

#449

Post by jameshh »

For those of you still following this story, the flight plan that triggered the chaos has been identified!

https://chaos.social/@russss/111048524540643971!

> Tonight we were wondering why nobody had identified the flight which caused the UK air traffic control crash so we worked it out. It was FBU (French Bee) 731 from LAX/KLAX to ORY/LFPO.

> It passed two waypoints called DVL on its expanded flight plan: Devil's Lake, Wisconsin, US, and Deauville, Normandy, FR (an intermediate on airway UN859).

> https://www.flightaware.com/live/flight/FBU731/history/20230...

> Credit to @marksteward and @benelsen for doing much of the legwork here.

Re: UK air traffic control meltdown

#450

Post by samus »

crabbone wrote: post #440
samus wrote: post #435

Earlier quoted context omitted.

> If programmer didn't know that repetitions are allowed, they wouldn't appear in the input to the fuzzer as well. It totally would. The point of a fuzzer is to test the system with every technically possible input, to avoid bias and blind spots in the programmer's thinking. Furthermore, assuming that no duplicates exist is a rather strong assumption that should always be questioned. Unless you know all about the bus…

No... that's not the point of fuzzing... You cannot write individual functions in such a way that they keep revalidating input handed to them. Because then, invariably, the validations will be different function to function, and once you have an error in your validation logic, you will have to track down all function that do this validation. So, functions have to make assumptions about input, if it doesn't come from…

I am not going to comment the first paragraph since you turned my words around.

> How do you even come up with this? Do you write your code in such a way that any time it pulls a value from a dictionary, you iterate over the dictionary keys to make sure that they are unique?

A dictionary in my program is under my control and I can be sure that the key is unique since... well, I know it's a dictionary. I have no such knowledge about data coming from external systems.

> There are plenty of things that are meant to be unique by design. The function in question wasn't meant to check if the points were unique. For all we know, the function might have been designed to take a map and the data was lost even before this function started processing it...

"Meant to be" and "actually are" can be very different things, and it's the responsibility of a programmer to establish the difference, or to at least ask pointed questions. Actually, the programmers did the correct thing by not sweeping this unexpected problem under the rug. The reaction was just a big drastic, and the system did not make it easy for the operators to find out what went wrong.

Edit: as we have seen, input can be valid, but still not be processable by our code. That not fine, but it's a fact of life since specs are often unclear or incomplete. Also, the rules can actually change without us noticing. In these cases, we should make it as easy as possible to figure out what went wrong.

Post reply on HN