Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UK air traffic control meltdown

jameshaydon.github.io

Post reply on HN
Print view View original

211–220 of 459 posts

Re: UK air traffic control meltdown

#211

Post by H8crilA »

xvector wrote: post #185
noman-land wrote: post #166

Earlier quoted context omitted.

My jaw kept dropping with each new bullet point.

Same, is aviation technology really this primitive?

It is mostly quite primitive, but it also works amazingly well. For example ILS or VOR or ATC audio comms can all be received and read correctly using hardware built from entry level ham radio knowledge. Altimeters still require a manual input of pressure. Fuel levels can be checked with sticks.

Kinda the opposite of a modern web/mobile app, complicated, massively bloated and breaks rather often :).

Re: UK air traffic control meltdown

#212

Post by seabass-labrax »

Rochus wrote: post #67

This is apparently just an opinion, no additional inside information than we had from the report ( https://news.ycombinator.com/item?id=37401981 ), isn't it? EDIT: downvoting this question instead of responding is a pretty strange reaction.

You are correct, but it's an opinion that bridges the gap editorially between those knowledgable about ATC but not data, and those knowledgable about data but not ATC. This is a valuable service to provide, as both fields are rather complex.

Re: UK air traffic control meltdown

#213

Post by PeterStuer »

raverbashing wrote: post #154
adrianmonk wrote: post #131

Earlier quoted context omitted.

Because the code classified it as a "this should never happen!" error, and then it happened. The code didn't classify it as a "flight plan has bad data" error or a "flight plan data is OK but we don't support it yet" error. If a "this should never happen!" error occurs, then you don't know what's wrong with the system or how bad or far-reaching the effects are. Maybe it's like what happened here and you could have co…

And that's why I never (or very rarely) put "this should never happen" exceptions anymore in my code Because you eventually figure out that, yes, it does happen

So what does your code do when you did not handle the this should never happen exception? Exit and print out a stacktrace to stdout?

Re: UK air traffic control meltdown

#214

Post by johnklos »

So the "engineering teams" couldn't tail /var/log/FPRSA-R.log and see the cause of the halt?

I've had servers and software that I had never, ever used before stop working, and it took a lot less than four hours to figure out what went wrong. I've even dealt with situations where bad data caused a primary and secondary to both stop working, and I've had to learn how to back out that data and restart things.

Sure, hindsight is easy, but when you have two different systems halt while processing the same data, the list of possible causes shrinks tremendously.

The lack of competence in the "engineering teams" tells us lots about how horribly these supposedly critical systems are managed.

Re: UK air traffic control meltdown

#215

Post by epanchin »

sam0x17 wrote: post #163
reactordev wrote: post #28

So they forgot to "geographically disparate" fence their queries. Having built a flight navigation system before, I know this bug. I've seen this bug. I've followed the spec to include a geofence to avoid this bug.

Why on earth do they not have GUIDs for these navigation points if the names are not globally unique and inter-region routes are commonplace?

What three words would be a better solution than a guid, as transmittable over radio.

Re: UK air traffic control meltdown

#216

Post by throwaway894345 »

drachir91 wrote: post #207

What ticked me is that when the primary system threw in the towel, an EXACT SAME system took over and ran the exact same code on the exact same data as the primary. I know that with code and algorithms it's not always the case but even then you know what doing the same thing over and over expecting different results defines... Yes, it can be argued that the software should've had more graceful failure modes and this…

> Aside from cost. I don't care about anyones cost-cutting incentives in mission-critical systems. Sorry capitalism...

Capitalism is happy to have redundancy in mission critical systems all the time. Why would it care here?

Re: UK air traffic control meltdown

#217

Post by seabass-labrax »

fbdab103 wrote: post #96
rglover wrote: post #72

Earlier quoted context omitted.

Great. It should be fixed by replacing the FORTRAN systems with a modern solution. It's not that it can't be done, it's that the engineers don't bother to start the process (which is a side-effect of bad incentive structure at the employment level).

I guess we should rewrite it in Rust. Airplane logistics feels like one of the most complicated systems running today. A single airline has to track millions of entities: planes, parts, engineers, luggage, cargo, passengers, pilots, gate agents, maintenance schedules, etc. Most of which was created all before best-practices were a thing. Not only is the software complex, but there are probably millions of devices in…

It would still be valuable to replace components piece-by-piece, starting with rigorously defining internal data structures and publically providing schemas for existing data structures so that companies can incorporate them.

I would like to point out that the article (and the incident) does not relate to airline systems; it is to do with Eurocontrol and NATS and their respective commercial suppliers of software.

Re: UK air traffic control meltdown

#218

Post by teleforce »

codeulike wrote: post #153

This is a great post. My reading of it: - waypoint names used around the world are not unique - as a sortof cludge, "In order to avoid confusion latest standards state that such identical designators should be geographically widely spaced." - but still you might get the same waypoint name used twice in a route to mean different places - the software was not written with that possibilty in mind - route did not compute…

Thanks for the summary and TL;DR.

Essentially this is down to the lack of proper namespace, who'd have thought aerospace engineer need to study operating systems! I've a friend who's a retired air force pilot and graduated from Cranfield University, UK foremost post graduate institution for aerospace engineering with their own airport for teaching and research [1]. According to him he did study OS in Cranfield, and now I finally understand why.

Apparently based on the other comments, the standard for namespace is already available but currently it's not being used by the NATS/ATC, hopefully they've learnt their lessons and start using it for goodness sake. The top comment mentioned about the geofencing bug, but if NATS/ATC is using proper namespace, geofencing probably not necessary in the first place.

[1] Cranfield University:

https://en.wikipedia.org/wiki/Cranfield_University

Re: UK air traffic control meltdown

#220

Post by slingnow »

johnklos wrote: post #214

So the "engineering teams" couldn't tail /var/log/FPRSA-R.log and see the cause of the halt? I've had servers and software that I had never, ever used before stop working, and it took a lot less than four hours to figure out what went wrong. I've even dealt with situations where bad data caused a primary and secondary to both stop working, and I've had to learn how to back out that data and restart things. Sure, hind…

Damn, if only you had been there to instantly save the day by just running that simple command!
Post reply on HN