Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Should We Do to Prevent Software from Failing?

sloanreview.mit.edu

Post reply on HN
Print view View original

81–90 of 128 posts

Re: What Should We Do to Prevent Software from Failing?

#81

Post by carlmr »

ozim wrote: post #55
AnimalMuppet wrote: post #29

Earlier quoted context omitted.

Even in the mechanical world, chicken coops aren't engineered to the same standards as skyscrapers. In software, we can build to skyscraper standards when we need to. But most of our software is done like we're building a chicken coop. And that's actually appropriate for some of it - your web app probably isn't the software equivalent of a skyscraper.

I don't know how Boeing is doing their software but 737 Max is nowhere near chicken coop. I kind of believe they use something like MISRA. Article also is not saying anything about web applications. What I have seen on my own eyes was for example airport parking lot in Eindhoven which is also not a chicken coop: https://nltimes.nl/2017/09/25/eindhoven-airport-parking-gara... On the other hand I have never seen collap…

The problem is that you can be nearly 100% Misra compliant and still have shit software.

It takes away about 90% of the footguns you have in C and C++, but those 10% are just never going to be covered by a coding standard.

Furthermore it's not only about using the worst language for safety in a corset. The disagreeing AoAs, the lack of a warning signal, the weird UI to disable Mcas. Nothing in MISRA or similar coding standards prevents these.

It's good old common sense that you sadly can't teach (although you should maybe pay for a design course for your software developers). If not at least read the design of everyday things. Most of the design problems are the same that downed these two planes.

Re: What Should We Do to Prevent Software from Failing?

#82

Post by airbreather »

pg_bot wrote: post #39

Speaking as someone who builds mission critical software, licensing is not the answer to this problem. Occupational licensing only serves to raise barriers to entry and crowd out competition without raising quality. To put my opinion bluntly, Bernie Madoff had a license. The things you should do to prevent catastrophic failure: - Reduce your attack surface as much as possible. - Automate your infrastructure, human in…

Still a big difference between mission critical (money) and safety critical (lives).

Safety critical encourages use of low variability languages, not just any language will do.

Also expects that software will fail at some point, so encourages architecture that is tolerant of this where the consequences warrant, eg redundancy and diversity such as 2oo3 voting a la airbus and space shuttle.

So many other things, but it always annoys me when the "IT" guys get involved in the safety software engineering just because there is a PC used to program the device - they just don't get that there is no suitable attitude of "don't worry how or why it got lost, just send that email again". Tell that to the widow.

Re: What Should We Do to Prevent Software from Failing?

#83

Post by airbreather »

gtirloni wrote: post #36

Civil engineers aren't asked to design a small 2-story building that can be retrofitted as a 300-story skyscraper when the time comes. Or connect two buildings 15km apart over the weekend.

Or make all the windows suddenly round, two days before the tenants move in...

Re: What Should We Do to Prevent Software from Failing?

#84

Post by djakjxnanjak »

Don’t make excuses to yourself when you know there is a problem with your code. Don’t say “well, it will only fail in these edge cases and I know not to use it that way.” You’ll forget, or someone else will use it the wrong way and it will break. The easiest time to fix it is when you are already working on the code and first notice the problem, not later.

Re: What Should We Do to Prevent Software from Failing?

#85

Post by gchamonlive »

Vector919 wrote: post #60
_bxg1 wrote: post #53

Earlier quoted context omitted.

The problem isn't that executives don't understand, it's that they don't care. Capitalism's incentives do not encourage the practice that's best for society. The only industries where care is taken are the ones that have been regulated to be as such. And then there are cases like Boeing, where that regulatory structure decayed and the safety practices decayed in tandem.

This reminds me of one of the first conversations I had with my manager when I started my current job as a software engineer. I used to work at a large financial company who had an intensely rigorous approach to verifying it's software in both manual and automated ways, as well as a much more careful approach to writing code to begin with. When I started here (a much smaller startup delivering marketing software for…

> it's really all about what your industry is optimizing for, and what would cost you more money

Sure, those were not critical systems, they could be patched with minimal hasse to clients. However you can't really predict catastrophe costs. It could range from minor usability issues to hard to trace calculation errors that could jeopardise the whole production operation. If the company doesn't work work with prototypes it should invest heavily in test coverage, integration test and delivery pipelines

Re: What Should We Do to Prevent Software from Failing?

#86

Post by nerpderp82 »

jonahss wrote: post #13

Licensing isn't the solution here. Software changes too quickly, while institutionalized tests and textbooks, and certifications move incredibly slowly. Treating this industry the same as others will hopelessly slow down the already glacial pace of government software, while innovation will continue outside the system. Software is a different realm, as other commenters point out. When faced with a problem in the soft…

The math doesn't change. High reliability software has the same quality as high reliability knowledge, namely it has been proven logically to be correct.

Re: What Should We Do to Prevent Software from Failing?

#87

Post by lioeters »

pg_bot wrote: post #39

Speaking as someone who builds mission critical software, licensing is not the answer to this problem. Occupational licensing only serves to raise barriers to entry and crowd out competition without raising quality. To put my opinion bluntly, Bernie Madoff had a license. The things you should do to prevent catastrophic failure: - Reduce your attack surface as much as possible. - Automate your infrastructure, human in…

My favorite from your insightful recommendations: actively try to break your own systems.

"Chaos monkeys" is a wonderful phrase and concept I picked up somewhere, as a strategy for testing and improving the resilience of systems. Since I learned of it, I've been introducing chaotic elements to test a wider range of inputs/behaviors, and indeed it has uncovered failing edge cases and helped to make application logic more robust.

Re: What Should We Do to Prevent Software from Failing?

#88

Post by nerpderp82 »

carlmr wrote: post #78
bcheung wrote: post #24

Earlier quoted context omitted.

We can also reduce the effort to write better code. Certain language and tooling features make quality easier.

I think that's the key. There was ADA now there's Rust. If you can have GC on your system there are plenty of other languages that make strong static typing and other compile time guarantees that beat any analyzed C++.

Our safety critical languages need to be simpler, so they are amenable to mechanical proofs and exhaustive model checking. Dimensional units would be a good start.

https://gmpreussner.com/research/dimensional-analysis-in-pro...

Re: What Should We Do to Prevent Software from Failing?

#89

Post by nineteen999 »

lioeters wrote: post #87
pg_bot wrote: post #39

Speaking as someone who builds mission critical software, licensing is not the answer to this problem. Occupational licensing only serves to raise barriers to entry and crowd out competition without raising quality. To put my opinion bluntly, Bernie Madoff had a license. The things you should do to prevent catastrophic failure: - Reduce your attack surface as much as possible. - Automate your infrastructure, human in…

My favorite from your insightful recommendations: actively try to break your own systems. "Chaos monkeys" is a wonderful phrase and concept I picked up somewhere, as a strategy for testing and improving the resilience of systems. Since I learned of it, I've been introducing chaotic elements to test a wider range of inputs/behaviors, and indeed it has uncovered failing edge cases and helped to make application logic m…

It's the name of Netflix's tool.

https://netflix.github.io/chaosmonkey/

Re: What Should We Do to Prevent Software from Failing?

#90

Post by jacobsenscott »

_bxg1 wrote: post #16
efficax wrote: post #10

The analogy between software "engineering" and structural engineering is used a lot to point out the failures of software. I think this analogy is a category mistake caused entirely by the misapplication of the word "engineer" to software developers. We have already solved large areas of common classes of bugs at the language level, through strong static typing, memory safety and safe concurrency (Rust is the vanguar…

None of the above changes the discussion. It's a very elaborate way of saying "writing correct software is hard". You know what's also hard to do correctly? Designing airplanes. Building skyscrapers. It's not that their designs are made flawless, it's that (in theory) the sheer volume of man-hours spent pouring over every minute detail of them, incorporating lessons learned from every failure that's ever happened in…

Writing software is not engineering - engineering is building the same thing over and over again and you slowly learn how to do it the right way. There are incremental improvements along the way - stronger lighter materials, better tools, etc. But the folks who built the Roman aqueducts could get up to speed pretty quickly on the Golden Gate Bridge.

The principles of flight don't change. The fundamentals of building a skyscraper or a bridge don't change. There are hundreds of years of learning behind those things that are still relevant today.

Building airplanes and bridges and skyscrapers is not hard at all. People do it all the time with a high success rate. The hardest part is scraping up the capital and managing the logistics. I'm sure none of that knowledge is required to get any kind of engineering certification.

If you redesigned every bridge and airplane from scratch, from first principles, you would have a tremendous failure rate no matter how many certifications you held. That's what software is - every project is a brand new thing that has never been done before - there isn't 200 years of learning to fall back on.

One of the worst things that could happen to the software industry is for bureaucrats and other people who know nothing about software to really believe it is an engineering discipline and try and regulate it as such. We'll all be forced to do a fantastic amount of busy work to meet arbitrary and archaic "engineering standards", and the software will actually decline in quality. You'll be fined $50,000 if you use tabs rather than spaces, or don't use yoda compares in your C. But as long as you are using spaces and yoda compares you won't be liable for a failure. Stuff basically like that.

Maybe when we have 200 years of experience writing software, rather than 50 or so, we can start to think about formalizing it as an engineering job.

Post reply on HN