Live data from Hacker News

Starliner faced “catastrophic” failure before software bug found

arstechnica.com

21–30 of 64 posts

Re: Starliner faced “catastrophic” failure before software bug found

#21
post #2

> According to the source, Boeing patched a software code error just two hours before the vehicle reentered Earth's atmosphere. Had the error not been caught, the source said, proper thrusters would not open during the reentry process, and the vehicle would have been lost. Uh, that's extremely concerning for a CREWED capsule.

Man I hope the QA person who found that bug was rewarded..

Re: Starliner faced “catastrophic” failure before software bug found

#22
post #6

Earlier quoted context omitted.

> You can't always simulate every possible failure mode. Things fail on the ground that wouldn't be possible during normal operation and vice versa. This should be concerning, then: https://spaceflightnow.com/2019/11/04/boeing-starliner-pad-a... > “Boeing is not going to do an in-flight abort test,” said Jon Cowart, deputy manager of the mission management office for NASA’s commercial crew program, before the pad abo…

You'd think that after two air plane disasters, they'd tread carefully

> Finally, before the meeting ended, the chair of the safety panel, Patricia Sanders, noted yet another ongoing evaluation of Boeing. "Given the potential for systemic issues at Boeing, I would also note that NASA has decided to proceed with an organizational safety assessment with Boeing as they previously conducted with SpaceX," she said.

This is a welcome development.

Re: Starliner faced “catastrophic” failure before software bug found

#24
post #16

On a certain blog that is completely outside acceptable standards for wrongthink and political correctness, a very popular topic of late is why NASA seems to have it in for Elon Musk Personally and SpaceX generally. The commonly stated reasons, and I will be paraphrasing and transliterating freely, are: HR culture defining administration wide objectives and methods and reasoning, professional embarrassment over langu…

> On a certain blog that is completely outside acceptable standards for wrongthink and political correctness What does this statement really add besides political signaling?

Well for one, it made me curious what blog

Re: Starliner faced “catastrophic” failure before software bug found

#25
post #3
post #2

> According to the source, Boeing patched a software code error just two hours before the vehicle reentered Earth's atmosphere. Had the error not been caught, the source said, proper thrusters would not open during the reentry process, and the vehicle would have been lost. Uh, that's extremely concerning for a CREWED capsule.

Doesn't that depend on the testing schedule? If the schedule called for simulations to be run in parallel with the live test, then it's an expected outcome. It should be _expected_ that every test will find a problem. Since this was uncrewed, there was no risk (other than to the uncompleted tests possibly requiring a second flight) to running them in parallel and porting across fixes for any problems that were found.…

It kind-of depends on what sort of issue you're finding in your test. Complex systems like spacecraft often have unexpected interaction effects which only testing can reveal. I would call these the 'good' kind of test learnings - ones that an army of great engineers wouldn't be able to predict.

This is editorializing but it looks like Boeing didn't uncover very complicated interactions - they failed at a more basic level of competency - timer synchronization for the launch issue and then a major software bug for an important orbital maneuver. Those sorts of issues really should be sorted out on the ground using hardware simulators. Furthermore, the timer synchronization failure prevented testing of the docking hardware, further delaying the overall program.

For a human rated vehicle, personally I think you should have at least one full-up, fully nominal test before you send anyone along for the ride.

Re: Starliner faced “catastrophic” failure before software bug found

#26
post #8

"Given the potential for systemic issues at Boeing, I would also note that NASA has decided to proceed with an organizational safety assessment with Boeing as they previously conducted with SpaceX"

Ah so just the fact that Boeing has been around for half a century doesn't absolve them from being subject to safety reviews? Nice to see some sense.

Re: Starliner faced “catastrophic” failure before software bug found

#27

On a certain blog that is completely outside acceptable standards for wrongthink and political correctness, a very popular topic of late is why NASA seems to have it in for Elon Musk Personally and SpaceX generally. The commonly stated reasons, and I will be paraphrasing and transliterating freely, are: HR culture defining administration wide objectives and methods and reasoning, professional embarrassment over langu…

FYI the Space Shuttle flew 10 missions for the DoD, making it far more relevant to the military than the Saturn V.

Re: Starliner faced “catastrophic” failure before software bug found

#28
post #3

Earlier quoted context omitted.

Doesn't that depend on the testing schedule? If the schedule called for simulations to be run in parallel with the live test, then it's an expected outcome. It should be _expected_ that every test will find a problem. Since this was uncrewed, there was no risk (other than to the uncompleted tests possibly requiring a second flight) to running them in parallel and porting across fixes for any problems that were found.…

Boeing took the "we'll do very rigorous engineering up front and prove everything on paper" approach, where SpaceX took the "we'll prove it works by actually launching it" approach (which isn't to say SpaceX isn't operating with engineering rigor). In theory Boeing ran all their simulations over the past couple years, and this flight should have just been a formality. As it turns out, Boeing is running into a lot of…

This is likely an oversimplification. There are multiple organizations, even within Boeing Defense & Space, that write their own flight software. All stovepiped and largely working in parallel. This doesn't even include the commercial folks, infamous for the 737 Max. My understanding is that the St. Louis teams are better regarded, and the folks that worked on DARPA HACMS deserve some credit, but they seem to be outliers. Boeing's culture doesn't seem to prioritize modern software development methods or software rigor on the whole. Functional testing should be the last layer of bug-hunting techniques, not the first or primary. The issue seen on their capsule didn't surprise me at all. Other BDS software groups use utterly outdated software development methods and we should all be a little bit worried.

Re: Starliner faced “catastrophic” failure before software bug found

#29
post #5
post #2

> According to the source, Boeing patched a software code error just two hours before the vehicle reentered Earth's atmosphere. Had the error not been caught, the source said, proper thrusters would not open during the reentry process, and the vehicle would have been lost. Uh, that's extremely concerning for a CREWED capsule.

It depends on if the crew would have been able to control those thrusters themselves. Obviously you want the entire system to be autonomous enough that no crew interaction is required but things do happen and the crew needs to be able to act and fix the problem if capsule can't do it nor the ground. When the Starliner started a burn at the wrong time, a crew would have been able to stop it and prevent the loss of fue…

In many spacecraft failure modes, the crew would be unconscious.

Re: Starliner faced “catastrophic” failure before software bug found

#30
post #14

On a certain blog that is completely outside acceptable standards for wrongthink and political correctness, a very popular topic of late is why NASA seems to have it in for Elon Musk Personally and SpaceX generally. The commonly stated reasons, and I will be paraphrasing and transliterating freely, are: HR culture defining administration wide objectives and methods and reasoning, professional embarrassment over langu…

I'm confused by what you mean. NASA has provided SpaceX with lots of expertise and assistance. And you are aware that Starliner is a Boeing vehicle, not SpaceX?

I'm not sure who is confused about what, but with the Starliner, Starhopper, and Starship, it seems easy to get confused.
Post reply on HN