Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Therac-25 Incident (2021)

thedailywtf.com

Post reply on HN
Print view View original

261–270 of 307 posts

Re: The Therac-25 Incident (2021)

#261

Post by jeltz »

herval wrote: post #242
fesenjoonior wrote: post #238

Earlier quoted context omitted.

> you don't need "high quality engineers" to have high-quality outputs. [citation needed]

37signals and craigslist are two examples (and they’re open about their engineering being sub-par). If you consider FB products “high quality”, it’s another example (the average FB developer is anything but “high quality”, by most definitions). Palantir is another example, with a horde of junior engineers and famous for bad practices (yet here they are commanding the US military). And so on and so on. The inverse is…

FB products are low quality as is 37 signals. I don't think you necessarily need geniuses to build good products but your examples are bad.

Re: The Therac-25 Incident (2021)

#262

Post by waste_monk »

kulahan wrote: post #207
franktankbank wrote: post #127

Earlier quoted context omitted.

A culture of high-quality engineering, no doubt. Made up of: high quality engineers!

Unfortunately, software developers are the absolute most offensive use of the word "engineer", because 99.9% of the stuff this field makes is a competition to take the most unique approach to a solution, then getting it bandaged together with gum and paperclips. If this industry wants to be respected, it should start trying to be actual engineers. There should be tons and tons of standards which are enforced legally,…

My university offered (not sure if it still exists) a software engineer degree that was part of the school of engineering, had a mix of courses from the school of engineering and school of math and IT, and at the end of it you would be eligible to take the local equivalent of the PE exam and become a Real Engineer (I think technically you would be in the field of mechatronics).

IMO this should be the standard - software engineer should be a protected title, and everyone else would be titled some flavour of software developer or similar.

Re: The Therac-25 Incident (2021)

#263

Post by b_e_n_t_o_n »

bzzzt wrote: post #58
b_e_n_t_o_n wrote: post #47

Earlier quoted context omitted.

invariants are so useful to enforce even for toy projects. they should never be triggered outside of dev, but if they do sometimes it's better to just let it crash.

Making sure the beam is off before crashing would be better though.

For sure :P

Re: The Therac-25 Incident (2021)

#264

Post by Gud »

sgerenser wrote: post #239
darepublic wrote: post #224

Earlier quoted context omitted.

16000 - 25000 rads right. Not safe under any circumstance?

Completely safe as long as the block of metal was in place. So you couldn’t just prevent the machine from putting out that much energy, you had to prevent it from doing that without the block in place.

So there should have been an interlocking system

Re: The Therac-25 Incident (2021)

#265

Post by fulafel »

WalterBright wrote: post #208
benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

I'm going to disagree. I have years of experience at Boeing designing aircraft parts. The guiding principle is that no single failure should cause an accident. The way to accomplish this is not "write quality software", nor is it "test the software thoroughly". The idea is "assume the software does the worst possible thing. Then make sure that there's an independent system that will prevent that worst case." For the…

The GP didn't propose processes in the sw engineering part though but "the real failure in the story of the Therac-25 from my understanding, is that it took far too long for incidents to be reported, investigated and fixed"

Re: The Therac-25 Incident (2021)

#266

Post by maweki »

SCdF wrote: post #160
maweki wrote: post #72

Earlier quoted context omitted.

But there is still a difference here. Provenance and proper traceability would have allowed the subpostmasters to show their innocence and prove the system failable. In the Therac-25 case, the killing was quite immediate and it would have happened even if the correct radiation dose was recorded.

I don't understand the distinction here. > Provenance and proper traceability would have allowed But there wasn't those things, so they couldn't, so they were driven to suicide. Bad software killed people. It being slow or fast doesn't seem to matter.

Slow killing software can be made more secure by adding the possibility for human review.

Fast killing software is too fast for that.

Re: The Therac-25 Incident (2021)

#267

Post by SCdF »

maweki wrote: post #266
SCdF wrote: post #160

Earlier quoted context omitted.

I don't understand the distinction here. > Provenance and proper traceability would have allowed But there wasn't those things, so they couldn't, so they were driven to suicide. Bad software killed people. It being slow or fast doesn't seem to matter.

Slow killing software can be made more secure by adding the possibility for human review. Fast killing software is too fast for that.

I'm really trying to understand your point, but I am failing.

It sounds like you're saying that you shouldn't care as much about the quality of "slow killing software" because in theory it can be made better in the future?

But... it wasn't though? Horizon is a real software system that real developers like you and me built that really killed people. The absolutely terrible quality of it was known about. It was downplayed and covered up, including by the developers who were involved, not just the suits.

I don't understand how a possible solution absolves the reality of what was built.

Re: The Therac-25 Incident (2021)

#268

Post by dwedge »

technofiend wrote: post #256
philjohn wrote: post #230

Earlier quoted context omitted.

This. One of the biggest things I see in junior engineers that I mentor (working in backend high throughput, low latency, distributed systems) is not working out all of the various failure modes your system will likely encounter. Network partitions, primary database outage, caching layer outage, increased latency ... all of these things can throw a spanner in the works, but until you've experienced them (or had a str…

I recently had to argue a junior into leaving the health check frequency alone on an ECS container: the regular log entries annoyed her and she didn't know how to filter logs, so her solution was to take healthchecks down to every five minutes, as just one example of trying to talk to people about the unhappy path.

That sounds more like a disaster waiting to happen than a junior. I find it difficult to believe that she didn't know the purpose of the healthcheck, so it sounds like breaking (someone else's problem) instead of addressing gaps in ability

Re: The Therac-25 Incident (2021)

#269

Post by pjmlp »

ozim wrote: post #169
pjmlp wrote: post #132

Earlier quoted context omitted.

Being a real Software Engineer, those that actually have the proper title, eventually with the final examination, means being able to deliver the best product within the set of given constraints. Also, speaking out when the train is visibly going against a wall.

You mean like those in the article: https://edition.cnn.com/2025/08/27/us/alaska-f-35-crash-acci...

Yeah, and I bet responsabilities will be taken care of, unlike in most software projects.

Re: The Therac-25 Incident (2021)

#270

Post by Aloha »

WalterBright wrote: post #220
vjvjvjvjghv wrote: post #216

Earlier quoted context omitted.

In general I agree but there is bit more complexity. I work in medical devices and there are plenty of situations where a certain output is ok in some circumstance but deadly in another. That makes a stopgap a little more tricky. I agree with the previous poster that the feedback from the field is lacking a lot. A lot of doctors don’t report problems back because they are used to bad interfaces. And then the feedback…

At Boeing there's a required "failure analysis" document listing all the failure modes and why they won't cause a crash by themselves.

Agreed - This is essentially the corner stone of systems failure analysis - something I wish architects thought about more in the software space.

I'm a product manager for an old (and if I'm being honest somewhat crusty) system of software, the software is buggy, all of it is, but its also self healing and resilient, so while yes, it fails with somewhat alarming regularity, with very lots and lots concerning looking error messages in the logs, but it never causes an outage because it self heals.

Good systems design isn't making bug free software or a bug free system, but rather a system where a total outage requires N+1 (maybe even N+N) things to fail before the end user notices. Failures should be driven by at most, edge cases - basically where the system is being operated outside of its design parameters, and those parameters need to reflect the real world and be known by most stakeholders in the system.

My gripe with software engineers sometimes, they're often too divorced from real users and real use cases, and too devoted to the written spec over what their users actually need to do with the software - I've seen some very elegant (and on paper, well designed) systems fall apart because if simple things like intermittent packet jitter, or latency swings (say between 10ms and 70ms) - these are real world conditions, often encountered by real world systems, but these spec driven systems fall apart once confronted with reality.

Post reply on HN