Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Programmers who want to change how we code before catastrophe strikes

theatlantic.com

Post reply on HN
Print view View original

71–80 of 274 posts

Re: Programmers who want to change how we code before catastrophe strikes

#71

Post by IncRnd »

tener wrote: post #37
chroem- wrote: post #24

The dangerous unreliability of software is an accountability problem, not a technology problem. Engineering disciplines are regulated so that engineers are held personally and even criminally responsible if they endanger property or lives. Software, being the newcomer, has no such regulation. Instead we have people trying to claim that software deserves a free pass from due diligence because "software is hard," when…

Broken software does not typically kill people or makes you loose money (time or some other resources, maybe). In contrast bridges that fall apart tend to kill people in a nasty way. They are also quite hard to reboot.

Broken software has the capability to regularly kill people or cause people to lose money. It happens too often.

Re: Programmers who want to change how we code before catastrophe strikes

#72

Post by brians »

skywhopper wrote: post #58

This article is silly in a lot of ways. The real problem of software engineering is not "how do you prove this code follows algorithm X exactly?" It's "how do you know what algorithm X needs to be in sufficient detail to implement it?" It's "when you realize you were wrong about algorithm X, how do you change your existing, working code to implement the new algorithm X-prime, without interruption?" It's "what do we d…

The article's not silly. Leveson and Hamilton's works cover that explicitly. Start with Engineering a Safer World's Intent Specifications, move on to Safeware, and look at the attention that goes into writing the right TLA+ specs.

Lists like "Government regulations, flaky hardware…" look a lot like the upper levels of the hierarchical control systems contemplated there. It didn't make it into a pop science piece, but it's absolutely core to the work at issue.

You can get a copy of the book at http://sunnyday.mit.edu/safer-world/index.html .

Re: Programmers who want to change how we code before catastrophe strikes

#73

Post by coldtea »

Bartweiss wrote: post #43
chroem- wrote: post #24

The dangerous unreliability of software is an accountability problem, not a technology problem. Engineering disciplines are regulated so that engineers are held personally and even criminally responsible if they endanger property or lives. Software, being the newcomer, has no such regulation. Instead we have people trying to claim that software deserves a free pass from due diligence because "software is hard," when…

> The only difference is that these other engineers are motivated to be significantly more thorough about their jobs. Hang on. Sure, all engineering has hurdles. But how many engineering projects face regular enemy action? The last time somebody actively tried to compromise a system I was building was two weeks ago. How many engineers live like that? We hold civil engineers responsible when bridges collapse unprompte…

>* Hang on. Sure, all engineering has hurdles. But how many engineering projects face regular enemy action? The last time somebody actively tried to compromise a system I was building was two weeks ago. How many engineers live like that? We hold civil engineers responsible when bridges collapse unprompted. We also hold them responsible if their bridges fail from some minor, anticipated harm like vandalism or a car crash. We don't hold them responsible for building bridges that can be blown up.*

Even more so, how many engineering problems have fuzzy requirements that on top of that change constantly and arbitrarily?

If someone thinks building bridges is the same as writing some big software, they really have no idea about working in the trenches.

Re: Programmers who want to change how we code before catastrophe strikes

#74

Post by wpietri »

chroem- wrote: post #24

The dangerous unreliability of software is an accountability problem, not a technology problem. Engineering disciplines are regulated so that engineers are held personally and even criminally responsible if they endanger property or lives. Software, being the newcomer, has no such regulation. Instead we have people trying to claim that software deserves a free pass from due diligence because "software is hard," when…

Motivation seems far from the only difference.

If we divide making into "builds with atoms" and "builds with bits", both sorts of things are generally unregulated. It's when you get into "builds life-critical things with atoms" that you see significant regulation. If you build a house or an office building, your engineering is regulated. If you build a doghouse or an office filing cabinet, it isn't.

I agree that societal accountability for engineers working on life-critical system, software and hardware both, is vital. For most atom-making and bit-making, though, I think stringent professional licensing is overkill. Instead I'd rather we started with accountability for the people who have much more control over the outcomes than individual software engineers: executives and managers.

Re: Programmers who want to change how we code before catastrophe strikes

#75

Post by schoen »

simlevesque wrote: post #18
hwayne wrote: post #16

> He began collaborating with Gerard Berry, a computer scientist at INRIA, the French computing-research center, on a tool called Esterel INRIA is also maintains Coq, TLAPS, and much of TLA+. INRIA is a scary scary place.

> INRIA is a scary scary place. Why ?

I assume scary as in "scar(il)y smart", in the sense that it's (almost) disturbing to see their sophistication and competence.

Re: Programmers who want to change how we code before catastrophe strikes

#76

Post by wpietri »

DrScump wrote: post #2

“Software engineers don’t understand the problem they’re trying to solve, and don’t care to.” In environments where management judges by (and is judged by) other metrics, this result is inevitable.

More specifically, I think those engineers are trained not to care. The have tried caring, experienced unpleasantness from their "superiors", and learned to stop.

But this is reversible. I've done coaching in large-company environments and it's not at all hard to wake developers up, to get them to care again. The hard part is changing the environment so that caring is rewarded, not punished.

Re: Programmers who want to change how we code before catastrophe strikes

#77

Post by arximboldi »

skywhopper wrote: post #58

This article is silly in a lot of ways. The real problem of software engineering is not "how do you prove this code follows algorithm X exactly?" It's "how do you know what algorithm X needs to be in sufficient detail to implement it?" It's "when you realize you were wrong about algorithm X, how do you change your existing, working code to implement the new algorithm X-prime, without interruption?" It's "what do we d…

I think you might enjoy this paper, I did: http://www1.cs.columbia.edu/~angelos/Misc/p271-de_millo.pdf

Re: Programmers who want to change how we code before catastrophe strikes

#78

Post by blackth0rn »

Bartweiss wrote: post #50
tener wrote: post #37

Earlier quoted context omitted.

Broken software does not typically kill people or makes you loose money (time or some other resources, maybe). In contrast bridges that fall apart tend to kill people in a nasty way. They are also quite hard to reboot.

An excellent framing I saw recently was "I can tell the people who write these articles don't actually want secure code, because they aren't paying me to write it." There's plenty of carefully-written, rigorously-tested code out there. Want to write software for air travel, drug manufacturing, or space flight? You'll have no choice but to write safe, first-rate code. Want a smartphone weather app written to those sta…

Isn't this a tooling and knowledge issue? If I was as productive with TLA+ or model based design as I am with React-Native/Ionic/Swift etc. then we could have a smartphone weather app written to those standards for the same price.

Re: Programmers who want to change how we code before catastrophe strikes

#79

Post by gipp »

skywhopper wrote: post #58

This article is silly in a lot of ways. The real problem of software engineering is not "how do you prove this code follows algorithm X exactly?" It's "how do you know what algorithm X needs to be in sufficient detail to implement it?" It's "when you realize you were wrong about algorithm X, how do you change your existing, working code to implement the new algorithm X-prime, without interruption?" It's "what do we d…

Everything you said after "this article is silly" kinda seems like just restating the exact point of the article.

The central idea is that the massive costs of writing code that adheres to a specification takes away resources from the task of understanding the problem space well enough to create a correct specification. That is, incidental complexity overwhelms essential complexity in modern software engineering. The various avenues of study mentioned in the article are attempts at addressing that imbalance.

Which is pretty much what you said.

Re: Programmers who want to change how we code before catastrophe strikes

#80

Post by AnimalMuppet »

kmicklas wrote: post #56

Disappointing to see such a long article and no mention of type theory, or any other work from the "correct by construction" school of formal methods. It's all normie model checking, TLA+ etc.

Type theory is much less than "correct by construction" formal methods. Type theory is great at preventing a whole class of bugs (an operation on a value for which that operation doesn't make sense). But it's inadequate for the larger problem, of whether the operation is the correct one.

Formal methods can ensure that the code matches a formally written spec, for all the aspects of the code that are covered by the formal method. Note well: That's not all aspects of the code. And you have the little problem of (correctly) creating the formal spec. This move the problem up a layer, but the problem doesn't go away.

Even given a formal spec, though, it's my impression that formal methods are s l o w. Does anyone have data on this?

Post reply on HN