Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Therac-25 Incident (2021)

thedailywtf.com

Post reply on HN
Print view View original

231–240 of 307 posts

Re: The Therac-25 Incident (2021)

#231

Post by layman51 »

WalterBright wrote: post #208
benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

I'm going to disagree. I have years of experience at Boeing designing aircraft parts. The guiding principle is that no single failure should cause an accident. The way to accomplish this is not "write quality software", nor is it "test the software thoroughly". The idea is "assume the software does the worst possible thing. Then make sure that there's an independent system that will prevent that worst case." For the…

I might also add that apparently, older versions of the machine had physical “hardware interlocks” that would make accidents less likely no matter what the software was doing. So the older software was probably just thought to be reliable, but it had a physical mechanism that was helping it to not kill someone. On a less serious note that’s part of why car doors might still have keyholes even if normally they open in a fancy way with electronic fobs.

Re: The Therac-25 Incident (2021)

#232

Post by jldugger »

0xDEAFBEAD wrote: post #98
benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

Honestly I wish instead of the Therac-25, we were discussing a system which made use of unit testing and defensive coding, yet still failed. That would be more educational. It's too easy to look at the Therac-25 and think "I would never write a mess like that".

Perhaps this is why the cover of my software correctness book in undergrad used a series of stills from the arianne-5 disaster[1] for the cover.

[1]: https://en.wikipedia.org/wiki/Ariane_5#Notable_launches

Re: The Therac-25 Incident (2021)

#233

Post by bobmcnamara »

pjmlp wrote: post #159
bobmcnamara wrote: post #155

Earlier quoted context omitted.

In my country those are a myth. We had it as a professional engineering classification for a while but I'm not sure if anyone ever completed it. They cancelled it several years ago.

And in US anyone can call themselves whatever they feel like. If you want professional quality, we're the first line of actually making it happen, blaming others won't change anything.

Turns out there could be up to 81 US professional software engineers!

https://ncees.org/ncees-discontinuing-pe-software-engineerin...

Re: The Therac-25 Incident (2021)

#234

Post by chuckadams »

NortySpock wrote: post #199
greazy wrote: post #68

Earlier quoted context omitted.

It's still an issue. I've heard stories of EMR system going down forcing staff to use pen and paper. It boggles my mind that such systems don't have redundancy. These are commercial products being deployed.

The redundancy is pen and paper. The EMR just helps teams coordinate faster, pull up records faster, etc. When I worked at Cerner years ago (now owned by Oracle), there were rumors that the Cerner EMR still could barely handle DST* spring forward, but could not handle DST fall back (where the 01:00 hour is repeated) -- you had do preemptively switch to pen-and-paper for the hours around the switch. I assume this was…

The idea of Daylight Savings Time goes back to Ben Franklin, and a lot of Europe does it too. I think it’s an obsolete notion nowadays, but it’s not exclusively a Dumb Yank Idea.

Re: The Therac-25 Incident (2021)

#235

Post by lmm »

simulator5g wrote: post #205
V__ wrote: post #94

Earlier quoted context omitted.

> Engineers are legally obligated to report unsafe conduct, activities or behaviours of others that could pose a risk to the public or the environment. [1] If software "engineers" want to be taken seriously, then they should also have the obligation to report unsafe/broken software and refuse to ship unsafe/broken software. The developers are just as much to blame as the post office: > Fujitsu was aware that Horizon…

I don't think it's fair to blame individual developers for a systemic failure. Its not their fault there is no governing body to award or remove the title of "software engineer" and promote the concept of a software engineer refusing to do something without harming their career. Other engineering disciplines have laws, lobbied for by their governing body, that protect the ability of individual engineers to prevent hi…

> Its not their fault there is no governing body to award or remove the title of "software engineer" and promote the concept of a software engineer refusing to do something without harming their career.

Those governing bodies didn't form by magic. If you look at how hostile people on this site are to the idea of unionization or any kind of collective organisation, I'd say a large part of the problem with software is individual developers' attitudes.

Re: The Therac-25 Incident (2021)

#236

Post by credit_guy »

benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

I'm not sure. Most software (by orders of magnitude) is not critical software like the software running that X-ray machine. In general, if your software fails, a page loads too slow, or a report comes with lots of NaN's, or some batch job does not run at the right time, and someone needs to start it manually, etc. The cases where someone dies because of a software quality issue are very rare, and the developers working on that type of software know who they are and what their duties are (I hope).

Re: The Therac-25 Incident (2021)

#237

Post by bobmcnamara »

mrguyorama wrote: post #194
bobmcnamara wrote: post #114

Earlier quoted context omitted.

Time and time again the introduction of electronic throttle control has spiked the number of reported unintended acceleration incidents. There's a chart here that shows it clearly for Toyota's rollout: https://www.embedded.com/unintended-acceleration-and-other-e...

That chart is not clear. The Camry, the Solara, and the RAV4 are all the exact same engine hardware and software, at nearly all times. Especially the Solara, which is just a Camry with a shinier body. It uses an identical engine, throttle body, and ECU flash and is even considered a "Camry Solara". That Camry "Unintended accelerations" jumped while Solara didn't means that it isn't the hardware. Instead, they all sta…

> The brakes on all Toyotas are fully hydraulic

No, most are hydraulic with vacuum boost.

If you aren't expecting it, the increased force required to drive the brakes hydraulically from the pedal without boost assist is significant and can be surprising. I assume most folks haven't had an engine fail going downhill, but for a large pickup I was standing on the brake pedal and had to push my leg down by pulling on the steering wheel to stop it.

> That seems.... hard to believe

Anyhow, the vacuum booster is driven from the engine airflow. At wide open throttle the vacuum available to the booster is minimal because the restriction is as open as it can be.

You can test how it feels by rolling at a medium speed in a parking lot, shifting into neutral, and killing the engine. The vacuum reservoir may provide you one or two brake pumps and then you're on your own.

Another test: after shutting down your car after a regular drive, try depressing the brake to the floor a few times. You'll soon exhaust the boost reservoir and the brake pedal will become very stiff now that it's fallen back to full hydraulic operation. In this condition if you hold the pedal halfway down when you start the car you'll feel the brake boost kick in soon as the engine starts.

Re: The Therac-25 Incident (2021)

#238

Post by fesenjoonior »

herval wrote: post #161
franktankbank wrote: post #127

Earlier quoted context omitted.

A culture of high-quality engineering, no doubt. Made up of: high quality engineers!

you don't need "high quality engineers" to have high-quality outputs. And vice-versa - lots of places with very high quality engineers produce terribly low-quality software

> you don't need "high quality engineers" to have high-quality outputs.

[citation needed]

Re: The Therac-25 Incident (2021)

#239

Post by sgerenser »

darepublic wrote: post #224
vjvjvjvjghv wrote: post #216

Earlier quoted context omitted.

In general I agree but there is bit more complexity. I work in medical devices and there are plenty of situations where a certain output is ok in some circumstance but deadly in another. That makes a stopgap a little more tricky. I agree with the previous poster that the feedback from the field is lacking a lot. A lot of doctors don’t report problems back because they are used to bad interfaces. And then the feedback…

16000 - 25000 rads right. Not safe under any circumstance?

Completely safe as long as the block of metal was in place. So you couldn’t just prevent the machine from putting out that much energy, you had to prevent it from doing that without the block in place.

Re: The Therac-25 Incident (2021)

#240

Post by rowanG077 »

benrutter wrote: post #5

> software quality doesn't appear because you have good developers. It's the end result of a process, and that process informs both your software development practices, but also your testing. Your management. Even your sales and servicing. If you only take one thing away from this article, it should be this one! The Therac-25 incident is a horrifying and important part of software history, it's really easy to think t…

I think the opposite. The only reason software quality emerges is because of good developers. It's a prerequisites. Process helps good developers deliver quality. But there is no process that allows a team of bad developers deliver quality. you can't squeeze blood from a stone.
Post reply on HN