Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Therac-25 Incident (2021)

thedailywtf.com

Post reply on HN
Print view View original

241–250 of 307 posts

Re: The Therac-25 Incident (2021)

#241

Post by BeFlatXIII »

bongodongobob wrote: post #188
skinwill wrote: post #64

Earlier quoted context omitted.

Around here we have Epic. If you want a good scare, look up their corporate Willy Wonka-esq jail/campus and their policy of zero remote work.

Their hiring is insane too. Years ago I applied for a Sys admin I position. After 3 rounds of interviews, they gave me a test to take. Except it was something you'd give a senior dev. They specced out a fake language, super esoteric too, it was like a combination of brainfuck and assembly, and asked me to solve problems with it, and debug code written in it. I could have toughed my way through it, but I was so dissat…

You sure it was a fake language and not just ANSI MUMPS?

Re: The Therac-25 Incident (2021)

#242

Post by herval »

fesenjoonior wrote: post #238
herval wrote: post #161

Earlier quoted context omitted.

you don't need "high quality engineers" to have high-quality outputs. And vice-versa - lots of places with very high quality engineers produce terribly low-quality software

> you don't need "high quality engineers" to have high-quality outputs. [citation needed]

37signals and craigslist are two examples (and they’re open about their engineering being sub-par). If you consider FB products “high quality”, it’s another example (the average FB developer is anything but “high quality”, by most definitions). Palantir is another example, with a horde of junior engineers and famous for bad practices (yet here they are commanding the US military). And so on and so on. The inverse is also true - plenty of stellar teams producing irrelevant or low impact products.

Re: The Therac-25 Incident (2021)

#243

Post by [deleted] »

philjohn wrote: post #230
WalterBright wrote: post #208

Earlier quoted context omitted.

I'm going to disagree. I have years of experience at Boeing designing aircraft parts. The guiding principle is that no single failure should cause an accident. The way to accomplish this is not "write quality software", nor is it "test the software thoroughly". The idea is "assume the software does the worst possible thing. Then make sure that there's an independent system that will prevent that worst case." For the…

This. One of the biggest things I see in junior engineers that I mentor (working in backend high throughput, low latency, distributed systems) is not working out all of the various failure modes your system will likely encounter. Network partitions, primary database outage, caching layer outage, increased latency ... all of these things can throw a spanner in the works, but until you've experienced them (or had a str…

[deleted]

Re: The Therac-25 Incident (2021)

#244

Post by herval »

ChrisMarshallNY wrote: post #167
herval wrote: post #161

Earlier quoted context omitted.

you don't need "high quality engineers" to have high-quality outputs. And vice-versa - lots of places with very high quality engineers produce terribly low-quality software

I guess we see things differently. They don't need to be especially talented engineers, but, in my experience (and I actually have quite a bit of it, in this area), they need to be dedicated to a culture of Quality. And it is entirely possible for very talented engineers to produce shite. I've seen exactly that.

A culture of quality doesn’t require particularly skilled individuals to function.

That’s in fact the thesis for the entire Deming management philosophy, and in line with what I’m saying (you can produce high quality with a good process or a good culture, you don’t necessarily need high caliber individuals)

Re: The Therac-25 Incident (2021)

#245

Post by wat10000 »

kccqzy wrote: post #210
wat10000 wrote: post #112

Earlier quoted context omitted.

The lesson is not to write a mess like that. It might seem obvious, but it has to be learned.

Software engineering has advanced in the past few decades that the kind of code considered a "mess" has expanded.

We’ve invented entirely new ways to write bad code.

Re: The Therac-25 Incident (2021)

#246

Post by msy »

benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

I couldn't disagree more. Outside of exotic scenarios where things like formal proofing are possible and economically viable I've never seen a process that prevents bugs, only culture. Good engineering cultures are also often ones with well defined and tested processes and good testing practices but it's the culture and people giving a shit that makes the difference, not the other way around.

Good product cultures are ones where natural communication between the field and engineering would mean issues get reported back up and make their way to the right people. No process will compensate for people not giving a shit.

Re: The Therac-25 Incident (2021)

#247

Post by smarks »

wocram wrote: post #27
michaelt wrote: post #6

I'd be interested in knowing how many of y'all are being taught about this sort of thing in college ethics/safety/reliability classes. I was taught about this in engineering school, as part of a general engineering course also covering things like bathtub reliability curves and how to calculate the number of redundant cooling pumps a nuclear power plant needs. But it's a long time since I was in college. Is this sort…

This was part of our Systems Engineering class, something like this: https://web.mit.edu/6.033/2014/wwwdocs/assignments/therac25....

Thanks for posting this. The Leveson article is IMO the definitive study on this incident. I’m going to repost at top level for emphasis.

Re: The Therac-25 Incident (2021)

#248

Post by smarks »

I believe the definitive analysis of the Therac-25 incident was written by Nancy Leveson, first in IEEE Computer,[1] and later as an appendix of her book.[2] The appendix is freely available as a PDF on the web [3][4] and probably other places. Many people here are asking questions about what happened and how it came about. The answers to many of these questions can be found there. I strongly recommend that anyone who is serious about safety and wants to learn more about this incident read Leveson’s analysis.

[1]: N. G. Leveson and C. S. Turner, "An investigation of the Therac-25 accidents," in Computer, vol. 26, no. 7, pp. 18-41, July 1993.

[2]: Nancy Leveson. Safeware: System Safety and Computers. Addison-Wesley, 1995.

[3]: http://sunnyday.mit.edu/papers/therac.pdf

[4]: https://web.mit.edu/6.033/2014/wwwdocs/papers/therac.pdf

Re: The Therac-25 Incident (2021)

#249

Post by mitthrowaway2 »

slavik81 wrote: post #163
Tenemo wrote: post #105

The full 1993 report linked in the article has an intetesting statement regarding software developer certfication in the "Lessons learned" chapter: > Taking a couple of programming courses or programming a home computer does not qualify anyone to produce safety-critical software. Although certification of software engineers is not yet required, more events like those associated with the Therac-25 will make such certi…

I am a licensed professional software engineer in Canada. It's been fifteen years since I first registered with my professional association, but I will probably not be renewing my license this year as it's not providing any real benefit to my career. Two decades ago there was a lot of talk about turning software development into a structured engineering discipline, but that plan seems to have largely been abandoned.

I've had some discussions with my engineering regulator in Canada. It's clear they have no idea what software engineering even is or who should be regulated or why. I tried to get them to provide some examples of what would and would not count as software engineering, but they couldn't.

Re: The Therac-25 Incident (2021)

#250

Post by bombcar »

WalterBright wrote: post #209
bell-cot wrote: post #101

Earlier quoted context omitted.

>> the real failure in the story of the Therac-25 from my understanding, is that it took far too long for incidents to be reported, investigated and fixed. > the earlier (manually operated) version of the machine did have the same fault. But it also had a failsafe fuse that blew so the fault never materialized. #1 virtue of electromechanical failsafes is that their conception, design, implementation, and failure mode…

> And PHB's all know that picking up nickles in front of the steamroller is how you get to the C-suite. Blaming it on PHB's is a mistake. There were no engineering classes in my degree program about failsafe design. I've known too many engineers who were insulted by my insinuations that their design had unacceptable failure modes. They thought they could write software that couldn't possibly fail. They'd also tell me…

Failsafe design is actually really fun when you start looking at all the scenarios and such.

But one key component is that IF a failsafe is triggered, it needs to be investigated as if it killed someone; because it should NEVER have triggered.

Without that part of the cycle, eventually the failsafe is removed or bypassed or otherwise ineffective, and the next incident will get you.

Post reply on HN