Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Air traffic system failure caused by computer memory shortage

reuters.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Air traffic system failure caused by computer memory shortage

#11

Post by cstuder »

gonzo wrote: post #4
hudibras wrote: post #3

Earlier quoted context omitted.

They didn't have enough memory to cope with an infinite number of inputs. Obviously a hardware problem...

is there a plane that flies at infinite altitude? Is there an aircraft of concern to the FAA that flies above FL1000? One or two historic flights have been that high, so raise the ceiling to 150,000' still not infinite.

Yeah, let's hardcode a maximum altitude somewhere deep in the code and be completely surprised when, a couple of years in the future, newer, higher-flying planes/shuttles disappear from the system.

Re: Air traffic system failure caused by computer memory shortage

#13

Post by andreasvc »

cstuder wrote: post #11
gonzo wrote: post #4

Earlier quoted context omitted.

is there a plane that flies at infinite altitude? Is there an aircraft of concern to the FAA that flies above FL1000? One or two historic flights have been that high, so raise the ceiling to 150,000' still not infinite.

Yeah, let's hardcode a maximum altitude somewhere deep in the code and be completely surprised when, a couple of years in the future, newer, higher-flying planes/shuttles disappear from the system.

No one said anything about burying it, it can be at the top of the main configuration file if you insist. The basic problem is that one quantity was limited (memory), without dealing with another (altitude). It makes perfect sense to limit both, while you're at it. But of course most important of all, test and study the effects of going over those limits ...

Re: Air traffic system failure caused by computer memory shortage

#14

Post by vince_refiti »

damian2000 wrote: post #2

Sounds more like a bug rather than a lack of memory to me ... The flight plan did not contain an altitude for the flight, one of the sources said. While a controller entered the usual altitude for a U-2 plane - about 60,000 feet - the system began to consider all altitudes between ground level and infinity. The conflict generated error messages and caused the system to begin cycling through restarts.

This scenario wasn't unit/functional tested then.

Re: Air traffic system failure caused by computer memory shortage

#15

Post by ars »

jmnicolas wrote: post #12

What's the point of flying the most stealthy plane on earth if you have to register a flight plan ? I thought such a plane would be 100% autonomous and be able to "see" and avoid other planes.

> flying the most stealthy plane

A U2 is not at all stealthy. You must be thinking of some other plane.

> if you have to register a flight plan

It's not a military operation in a foreign country, it's a routine flight in the home country.

> I thought such a plane would be 100% autonomous

The U2 was made long before such computers existed. They do have drones now.

> and be able to "see" and avoid other planes.

It wasn't the plane with the problem, it was ground control.

This must be a world record: A faulty assumption in every single sentence!

Re: Air traffic system failure caused by computer memory shortage

#16

Post by andreasvc »

huhtenberg wrote: post #8

I would guess that it was some form of an numeric overflow, signed/unsigned issue or out of bound array access, because the only abnormal thing here is U2's altitude.

Have you read the article? The cause is explained quite explicitly, and it was the very absence of the altitude not any of the things you mention.

Re: Air traffic system failure caused by computer memory shortage

#17

Post by andreasvc »

kijin wrote: post #5

said Dan Kaminsky, co-founder of the White Ops security firm and an expert in attacks based on over-filling areas of computer memory. That sounds more like buffer overrun than OOM. One is a Heartbleed-class vulnerability, the other is a pretty straightforward DoS situation. But of course why would we expect Reuters to be able to distinguish between the two...

That's irrelevant to the rest of the article, it's only how his expertise is characterized.

Re: Air traffic system failure caused by computer memory shortage

#18

Post by ars »

MattHeard wrote: post #10
damian2000 wrote: post #2

Sounds more like a bug rather than a lack of memory to me ... The flight plan did not contain an altitude for the flight, one of the sources said. While a controller entered the usual altitude for a U-2 plane - about 60,000 feet - the system began to consider all altitudes between ground level and infinity. The conflict generated error messages and caused the system to begin cycling through restarts.

It sounds like the altitude existence assumption caused cascading errors which in turn exhausted the system memory.

That's not what the article said. They said that since it didn't know the true altitude the computer assumed the plane could be at every altitude, and checked every single possible one.

This was made worse by a flight plan that had the plane going in and out of the area, so there were a tremendous number of permutations of possible flight paths.

The computer attempted to allocate memory to calculate every one of them and ran out of memory.

Re: Air traffic system failure caused by computer memory shortage

#19

Post by andreasvc »

jmnicolas wrote: post #12

What's the point of flying the most stealthy plane on earth if you have to register a flight plan ? I thought such a plane would be 100% autonomous and be able to "see" and avoid other planes.

It's not the most stealthy plane on earth, in fact it has been shot down over the Soviet Union and China in historical cases, because of being tracked by radar. Moreover it was flying over American airspace, it makes sense to register it.

Re: Air traffic system failure caused by computer memory shortage

#20

Post by andreasvc »

awjr wrote: post #6

Having worked on ATC, this really does sound like a software bug. Fixing such a bug on a delivered system would require an immense amount of costly testing. Adding memory is the cheap option. The other thing to consider is that some of these systems are rather old and could easily have less than a gig of ram.

That sounds very worrying. The fact that this issue occurs shows that they didn't prepare for OOM conditions, and another issue could easily cause a different OOM condition they didn't prepare for. It's not inconceivable that there could be an issue which gobbles up an amount of memory you throw at it. It is a tragic irony that the fact that testing is costly may be the very reason this condition was not tested for.
Post reply on HN