Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Therac-25 Incident (2021)

thedailywtf.com

Post reply on HN
Print view View original

121–130 of 307 posts

Re: The Therac-25 Incident (2021)

#121

Post by franktankbank »

Fr3dd1 wrote: post #34
AdamN wrote: post #16

Earlier quoted context omitted.

This is true but there also needs to be good developers as well. It can't just be great process and low quality developer practices. There needs to be: 1/ high quality individual processes (development being one of them), 2/ high quality delivery mechanisms, 3/ feedback loops to improve that quality, 4/ out of band mechanisms to inspect and improve the quality.

I would argue that a good process always has a good self correction mechanism built in. This way, the work done by a "low quality" software developer (this includes almost all of us at some point in time), is always taken into account by the process.

The process that makes this work would be so onerous to create. Would you think you could do this to make a low quality machinist be able to build a high quality technical part? What would this look like? Quite a lot like machine code which doesn't really reduce the requirements does it? It actually just shifted the onerous requirement somewhere else.

Re: The Therac-25 Incident (2021)

#122

Post by ycombobreaker »

vemv wrote: post #25

My (tragically) favorite part is, from wikipedia: > A commission attributed the primary cause to generally poor software design and development practices, rather than singling out specific coding errors. Which to me reads as "this entire codebase was so awful that it was bound to fail in some or other way".

Sibling reply notes the "process" is the problem, amd I would second that. I would also like to add, it's perfectly possible to produce a high quality code base with poor practices. This can happen with very small, expert teams. However, certain qualities become high-variance, which becomes a hazard over time.

Re: The Therac-25 Incident (2021)

#123

Post by hiccuphippo »

haunter wrote: post #75

My "favorite" part: >One failure occurred when a particular sequence of keystrokes was entered on the VT100 terminal that controlled the PDP-11 computer: If the operator were to press "X" to (erroneously) select 25 MeV photon mode, then use "cursor up" to edit the input to "E" to (correctly) select 25 MeV Electron mode, then "Enter", all within eight seconds of the first keypress and well within the capability of an…

And we have a concept, optimistic updates, for making the ui look responsive while the updates happen in the background and reconcile later. I can only hope they know when not to use it.

Re: The Therac-25 Incident (2021)

#124

Post by ozim »

pjmlp wrote: post #102
benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

The worst part is that many devlopers think that by not working with high integrity systems, such quality levels don't apply to them. Wrong, any software failure can have huge consequences in someone's life, or company, by preventing some critical flow to take place, corrupting data related to someone's life, professional or medical record, preventing a payment on some specific goods that had to be acquired on that m…

Hey don’t blame developers.

It is business who requests features ASAP to cut costs and and then there are customers who don’t want to pay for „ideal software” but rather have every software for free.

Most devs and QA workers I know want to deliver best quality software and usually are gold plating stuff anyway.

Re: The Therac-25 Incident (2021)

#125

Post by roeles »

0xDEAFBEAD wrote: post #98
benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

Honestly I wish instead of the Therac-25, we were discussing a system which made use of unit testing and defensive coding, yet still failed. That would be more educational. It's too easy to look at the Therac-25 and think "I would never write a mess like that".

One instance that crosses my mind often is the airbus a320 incident at Hamburg in 2008. Everything was done right there, but the requirements were wrong.

Despite all the procedures and tests, the software still managed to endanger the lives of the passengers.

Re: The Therac-25 Incident (2021)

#126

Post by varjag »

scott_w wrote: post #106
varjag wrote: post #90

Earlier quoted context omitted.

My takeaway from observing different teams over years is the talent by a huge margin is the most important component. Throw a team of A performers together and it really doesn't matter what process you make them jump through. This is how a waterfall team got the mankind to the Moon with handwoven core memory but an agile team 10x the size can't fix the software for a family car.

You conflated, misrepresented and simply ignored so many things in your statement that I really don’t know where to start rebutting it. I’d say at least compare SpaceX to NASA with space exploration but, even then, I doubt you have anywhere near enough knowledge of both programmes to be able to properly analyse, compare and contrast to back up your claim. Hell, do you even know if SpaceX or Tesla are even using an ag…

Hell, do you even know if SpaceX or Tesla are even using an agile methodology for their system development?

What I've been saying is methodology is mostly irrelevant, not that waterfall is specifically better than agile. Talent wins over the process but I can see how this idea is controversial.

I’d need to see some real examples of high talent, no process, teams compared to low talent, high process, teams, then some mixture of the groups to make a fair statement. Even then, how do you measure talent?

Yep, even if I made it my life's mission to run a formal study on programmer productivity (which I clearly won't) that wouldn't save the argument from nitpicking.

Re: The Therac-25 Incident (2021)

#127

Post by franktankbank »

ChrisMarshallNY wrote: post #120
benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

I worked for a company that manufactured some of the highest-Quality photographic and scientific equipment that you can buy. It was expensive as hell, but our customers seemed to think it was worth it. > It's the end result of a process In my experience, it's even more than that. It's a culture .

A culture of high-quality engineering, no doubt. Made up of: high quality engineers!

Re: The Therac-25 Incident (2021)

#128

Post by ozim »

anal_reactor wrote: post #71
quietbritishjim wrote: post #37

Earlier quoted context omitted.

Right, but if everyone is low quality then there's no one to do that correction. That may seem a bit hypothetical but it can easily happen if you have a company that systematically underpays, which I'm sure many of us don't need to think hard to imagine, in which case they will systematically hire poor developers (because those are the only ones that ever applied).

Sad truth is that average dev is average, but it's not polite to say this out loud. This is particularly important at scale - when you are big tech at some point you hit a wall and no matter how much you pay you can't attract any more good devs, simply because all good devs are already hired. This means that corporate processes must be tailored for average dev, and exceptional devs can only exist in start-ups (or her…

Huh, sad truth?

But average construction worker is also average and average doctor as well.

World cannot be running on „best of the best” - just wrap your head around the fact whole economy and human activities are run by average people doing average stuff.

Re: The Therac-25 Incident (2021)

#129

Post by thijson »

tedggh wrote: post #104

TL;DR The Therac-25 was a radiation therapy machine built by Atomic Energy Canada Limited in the 1980s. It was the first to rely entirely on software for safety controls, with no hardware interlocks. Between 1985 and 1987, at least six patients received massive overdoses of radiation, some fatally, due to software flaws. One major case in March 1986 at the East Texas Cancer Center involved a technician who mistyped t…

I remember my computer science professor talking about this, how critical safety can be in software. Another example he gave was the refueling machine at a nuclear power plant, it had fell off the tracks and broke the pipe that goes through the reactor due to a software bug. Also he mentioned the software in his pacemaker.

Engineers in other fields need to sign off on designs, and can be held liable if something goes wrong. Software hasn't caught up to that yet.

Re: The Therac-25 Incident (2021)

#130

Post by snkline »

I was kinda shocked by the results of his informal survey, because this was a big focus of my ethics course in college. I guess a lot of developers either didn't get a CS degree, or their degree program didn't involve an ethics course.
Post reply on HN