Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A corrupt file led to the FAA ground stoppage – also found in backup system

cnn.com

Post reply on HN
Print view View original

351–360 of 402 posts

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#351

Post by Tangurena2 »

metacritic12 wrote: post #196
gnarbarian wrote: post #105

Earlier quoted context omitted.

I spent a lot of time at the FAA writing software. (6+ years) there is a huge culture of process, policy and not a whole lot of thinking or analysis or actually understanding the problems that they are working on. it is maddening. imagine a spreadsheet with 700 lines in it telling you that you need to do ABCDEFG each of those lines is instructing you to write a document detailing a procedure with the chain of custody…

Agencies like the FAA are notoriously risk adverse. Basically the motivations of most employees seems like, "if I mess up once, I get fired; if I don't produce any movement, I can't get fired". Naturally, the output is glacial progress and the introduction of tons of safety-theatre procedures (that get the implementers promotions for "increasing a culture of safety"). We all know this from the subject the FAA regulat…

The early aviation industry had a very substantial "move fast and break stuff" mentality. YouTube has plenty of videos of those folks. However "break stuff" usually meant smearing aircraft and bodies all over the landscape. Much of FAA's regulation is to try to prevent killing "too many" people. If the engine on your car stops, you can usually pull over to the side of the road. If the engine on your aircraft stops, you're going to be landing soon and hopefully not landing on a building full of people. And if you're really lucky when the engine stops, all the people sitting in the back can walk away.

The FAA is frequently called a "tombstone agency" - who only act long after fatal accidents and the bodies have been buried.

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#352

Post by berniedurfee »

alexb_ wrote: post #271
huslage wrote: post #219

You all seem to think this is similar in value or operation to a web app. It is not. It is a safety-critical system that requires very stringent operational and development guidelines ON PURPOSE. The idea that the FAA shouldn't be risk averse in this system is absolutely ridiculous. The complexity of operating the airspace of an entire nation is nothing to scoff at and the importance of the NOTAM system should not be…

You've got it exactly right. There are a lot of people here who are completely deluded into the "move fast and break things" mindset not realizing that sometimes you really do not want to move fast, because you REALLY do not want to break things. A corrupted file throwing up panics like this is a good thing, because you don't want corrupted files to pass through like everything is fine.

Move fast and break things, unless people are involved.

Please, don’t break people.

People are non-fungible.

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#353

Post by berniedurfee »

cdiamand wrote: post #270
huslage wrote: post #219

You all seem to think this is similar in value or operation to a web app. It is not. It is a safety-critical system that requires very stringent operational and development guidelines ON PURPOSE. The idea that the FAA shouldn't be risk averse in this system is absolutely ridiculous. The complexity of operating the airspace of an entire nation is nothing to scoff at and the importance of the NOTAM system should not be…

The FAA has risk reduction baked into it's very heart and soul and this is evident the more you learn about how the system operates (go look at a flying textbook). It is not perfect organization, but I think it deserves more credit than it is receiving in this thread.

Really. The outage was impactful, but it was remediated in a few hours.

That ain’t bad considering how infrequent incidents are in light of the immense complexity of the operation.

Much less complex and much more modern systems go down regularly and sometimes don’t come back up for much longer.

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#354

Post by [deleted] »

metacritic12 wrote: post #212
avar wrote: post #209

Earlier quoted context omitted.

We know what it's like in software because we have e.g. the example of how the Space Shuttle's flight control software was developed and audited. It's weeks of meetings, tests and change processes etc. obsessing over changing a single instruction in assembly. Which also suggests to me that the problem isn't per-se the glacial progress & approval processes, but that they're using the wrong glacial processes. If this F…

I would settle for the government moving bureaucracy from safety-theatre to actual safety.

[deleted]

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#355

Post by CuriousCosmic »

njovin wrote: post #287
haswell wrote: post #279

Earlier quoted context omitted.

This is a sweeping claim with really nothing to back it up. Are you saying this from an inside knowledge of the FAA, or is this just an opinion? On the surface, the relative safety of air travel and the lack of major stoppages over a span of 22 years seems like a major counter example. You’re making this statement emphatically and authoritatively, though, so I’m curious to understand where that certainty comes from a…

Two examples: 1. The FAA basically handed their risk-management keys over to Boeing when authorizing the 737-MAX, contributing to those deaths ( https://www.newyorker.com/news/our-columnists/how-boeing-and... ) 2. The FAA's pilot medical vetting process, while thorough, is behind the times. There are people who took ADHD medicine in high school that are unable to obtain a medical certificate due to the FAA's overly-s…

Regarding #2, while there are a lot of issues with the FAA's medical process, you can absolutely get cleared to fly after having previously taken ADHD medication.

What you have to do is:

1. Not take those meds for at least 1-2 years.

2. Show documentation that getting off the meds hasn't impeded your performance. This generally means showing a stable work history if you've been off them for a long time or documents showing no change in performance between before you stopped taking them and x months after if you recently got off them.

3. Take an FAA ADHD re-evaluation.

Then they'll clear you. It's an annoying process but it's absolutely doable.

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#356

Post by justinsaccount »

avgDev wrote: post #349
justinsaccount wrote: post #296

Earlier quoted context omitted.

People like to think that the alternative to "move fast and break things" is "move slowly and not break things" but it's not, it's "move slowly, break things anyway, then take days to resolve the problem because you never learned how to move fast".

When I hear "move fast and break things", I am reminded of a guy that I worked with. He delivered work fast......full of bugs...0 planning...and his daily ritual was to just keep patching the "pile of shit" he put together. He has now moved on and we sometimes chat, he works for a huge corp, still sucks at writing SQL.

And as I said, people think the alternative to "fast, full of bugs, 0 planning" is automatically "slow, no bugs, lots of planning" but it's often "slow, lots of planning, just as many bugs"

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#357

Post by sclarisse »

bobbylarrybobby wrote: post #253
sclarisse wrote: post #217

Earlier quoted context omitted.

There’s a reason for that. Freight has priority because the freight companies actually own the rails, and actually use them, and it’s a significant contributor to the US economy. Amtrak is an unwelcome guest that they are required by law to put up with. If they wanted better service, Amtrak could pay for it, except the big cross country routes are already crazy unprofitable. The actual sane solution is to shut down t…

Well, another option is that the government could pay for Amtrak’s track the way they do for roads. We don't make car manufacturers or taxi/bus companies pay for the roads, so I don't see why it's on Amtrak to pay for their own rail.

On the highways, the government has taxed motor vehicle fuels at the federal and state level (supplying occasional infusions from the general fund) and occasionally set up tolls for specific projects, generally setting higher for more intensive use (trucks). Thus bus and truck operators pay for use of the highway, perhaps occasionally joined by taxpayers generally.

By contrast, on the railroads, Burlington Northern Santa Fe or whoever purchased the rail from the decrepit husk of bankruptcy that preceded it, and upgraded it, and installed all the new safety equipment, and maintained it over time. The capital involved here is, by and large, private.

So there shouldn’t be any surprise that the situations are different.

I don’t know why you want to drag in manufacturers, though. It seems to muddy the water.

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#358

Post by avgDev »

justinsaccount wrote: post #356
avgDev wrote: post #349

Earlier quoted context omitted.

When I hear "move fast and break things", I am reminded of a guy that I worked with. He delivered work fast......full of bugs...0 planning...and his daily ritual was to just keep patching the "pile of shit" he put together. He has now moved on and we sometimes chat, he works for a huge corp, still sucks at writing SQL.

And as I said, people think the alternative to "fast, full of bugs, 0 planning" is automatically "slow, no bugs, lots of planning" but it's often "slow, lots of planning, just as many bugs"

When you are in a complex spiderweb you simply cannot move fast. If you are moving fast you are not looking at everything and it will blow up in your face.

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#359

Post by sclarisse »

_ea1k wrote: post #300
sclarisse wrote: post #217

Earlier quoted context omitted.

There’s a reason for that. Freight has priority because the freight companies actually own the rails, and actually use them, and it’s a significant contributor to the US economy. Amtrak is an unwelcome guest that they are required by law to put up with. If they wanted better service, Amtrak could pay for it, except the big cross country routes are already crazy unprofitable. The actual sane solution is to shut down t…

Doesn't Amtrak pay for that? I have a feeling if there was an effort to shut them down and replace it with buses, the freight companies would block it. They benefit from the relationship.

This feeling is grossly at odds with the reality: hostility between host railroads and Amtrak is well known as the order of the day. Amtrak pays a well-below-market sweetheart rate, for trains that often miss their slots and snarl operations. No host railroad would miss them.

Re: A corrupt file led to the FAA ground stoppage – also found in backup system

#360

Post by berniedurfee »

tristor wrote: post #347
huslage wrote: post #219

You all seem to think this is similar in value or operation to a web app. It is not. It is a safety-critical system that requires very stringent operational and development guidelines ON PURPOSE. The idea that the FAA shouldn't be risk averse in this system is absolutely ridiculous. The complexity of operating the airspace of an entire nation is nothing to scoff at and the importance of the NOTAM system should not be…

FAA regulations, more so than nearly anything else regulated in the US, are written in blood. They are not written out of government corruption. Many many many hundreds of people have died in airplane accidents since airplanes were invented, and the reason why your average person can board a commercial flight and act like they just got on the bus (with a /better/ risk profile than a bus on public roads) is exactly du…

Mentor Pilot on YouTube does a great job of discussing aircraft incidents and how the RCA almost always informs new regulations.
Post reply on HN