Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Worst Computer Bugs in History: Therac-25 (2017)

blog.bugsnag.com

Post reply on HN
Print view View original

101–110 of 114 posts

Re: Worst Computer Bugs in History: Therac-25 (2017)

#101

Post by pjmlp »

anonuser123456 wrote: post #83
pjmlp wrote: post #44

Earlier quoted context omitted.

>Probably not. As a company you just disclaim liability in your terms of service. Thankfully EULAs are void in Europe. It is all a matter how big the customer base gets, I am hoping eventually we get something like that EU wide. > The market is still the ultimate decider for quality; if you build a crappy product, expect to get innovated out. If that was true 1 € shops wouldn't exist, but even those products have mor…

> Thankfully EULAs are void in Europe. It's not so clear cut that you should be thankful. The ability of companies to dictate the terms of which users can use their software, affects their risk calculation to produce the product in the first place. It is very likely that very useful but imperfect software will not be written because the risk / reward balance is tilted. Remember, you always have the ability to reject…

Than let it be so, software shouldn't be a special snowflake with special rules regarding commercial obligations.

Re: Worst Computer Bugs in History: Therac-25 (2017)

#102

Post by pjmlp »

ksdale wrote: post #84
pjmlp wrote: post #8

Earlier quoted context omitted.

Lawful punishment of bad quality software needs to be a thing, just like in other industries. Only then will most companies actually start to care about software quality in their development processes.

Other industries are also free to make terrible products, they're just not allowed to hurt people. In that regard, software isn't that different. Crappy software just doesn't physically injure people very often (compared to like, lawnmowers), and that's where the most serious legal liability for products comes from. Monetary damage from software gets worked out the same way any contract dispute gets worked out, or th…

Most people surely wouldn't buy a physical product that doesn't work as expected, or in the worst case they will return it and expect to be fully refunded.

If everyone did the same thing for broken software, instead of being conditioned that broken software is unavoidable, the quality across the industry would be much better.

Re: Worst Computer Bugs in History: Therac-25 (2017)

#104

Post by Matumio »

therein wrote: post #71
ecpottinger wrote: post #60

Earlier quoted context omitted.

I have friend who has 40 years programming experience, he is building a computer controlled milling machine in his basement. When I asked him about the limit switches it turns out they are read by software only and the software will turn off power to the motor controllers if a limit switch is activated. I asked why he does not wire the switches to cut power directly to be on the safe side. His answer "It's to much bo…

In a situation like that, I wouldn't blame him. Consider while building his milling machine how many of these situations he will come across. If he had to make sure there was a hardware failswitch, it would simply not scale. 3D printers are like this too. They have mechanical limit switches [0] that are read only by software. So if there is a bug in the software, nothing is stopping it from pushing the hardware limit…

Most 3D printers don't have massive printing heads. If they drive into the end-stops, the motors will likely just skip steps and be stuck. They are not designed to apply much force.

I'm much more worried about the heating element. Its temperature is usually controlled by the same cpu that also does motion control and g-code parsing. If anything locks up the CPU the heat might not be turned off in time, and (because you also want fast startup) there is enough power available to melt something. At the very least you would get nasty fumes from over-heated plastics, and maybe even teflon tape, which often is part of the print head. At worst it could start a fire.

Re: Worst Computer Bugs in History: Therac-25 (2017)

#105

Post by ioquatix »

Matumio wrote: post #104
therein wrote: post #71

Earlier quoted context omitted.

In a situation like that, I wouldn't blame him. Consider while building his milling machine how many of these situations he will come across. If he had to make sure there was a hardware failswitch, it would simply not scale. 3D printers are like this too. They have mechanical limit switches [0] that are read only by software. So if there is a bug in the software, nothing is stopping it from pushing the hardware limit…

Most 3D printers don't have massive printing heads. If they drive into the end-stops, the motors will likely just skip steps and be stuck. They are not designed to apply much force. I'm much more worried about the heating element. Its temperature is usually controlled by the same cpu that also does motion control and g-code parsing. If anything locks up the CPU the heat might not be turned off in time, and (because y…

As a 3D printing enthusiast, I can confirm your fears. It's all in software and while there are good control systems, nothings perfect. I had the hotbed fail and it was smoking when I found it.

Re: Worst Computer Bugs in History: Therac-25 (2017)

#106

Post by umvi »

gmueckl wrote: post #68
btrettel wrote: post #63

Earlier quoted context omitted.

Yes, that would be ideal. My impression is that CAD software usually integrates with another software to do stress analysis, e.g., FEA software, so it's more complicated than you've described, but very possible.

How far advanced are methods to automatically setup FEA simulations on arbitrary inputs? The parameter space for FEA methods is pretty large and things like meshing can go terribly wrong and lead to utterly wrong results. Trying to run simulations in the background while a building is being designed seems like a goal to strife for, but to be successful, the software needs to be able to perform equivalently to an expe…

I mean, even if the software could do something like flag the hyatt bridge redesign as failing to meet weight specs, that would be a win. I could be wrong but it seems like certain civil engineering projects (such as indoor bridges) would be pretty simple to check the math on compared to a building subject to wind, etc.

Re: Worst Computer Bugs in History: Therac-25 (2017)

#107

Post by jsjohnst »

perl4ever wrote: post #99
jsjohnst wrote: post #59

Earlier quoted context omitted.

Still doesn’t negate the fact the technician should’ve still checked. Again, not saying the technician is at fault at all, but they aren’t free of being involved in the death either. I’ll use a terrible analogy to make the point, if someone tells you “that’s bad” if you pull the trigger on a revolver playing Russian roulette and you pull the trigger five times without any apparent consequence despite being informed “…

"if someone tells you “that’s bad” if you pull the trigger on a revolver playing Russian roulette and you pull the trigger five times without any apparent consequence despite being informed “don’t do that” each time" I'm not sure that's a valid analogy. Isn't it the case rather that the revolver said "that's bad" but a person said "it's fine, ignore that, it always does that"? Which, in my experience, happens all the…

> Which, in my experience, happens all the time in the workplace.

If you are doing work that could immediately cause someone’s death, I feel you have a duty to double check and not just go with the flow. Ymmv and I’m not claiming my POV is more right, just that it’s my POV.

Re: Worst Computer Bugs in History: Therac-25 (2017)

#108

Post by Digital-Citizen »

classichasclass wrote: post #5

Therac is one of the reasons I get nervous about "health hacking." Yes, people can verifiably benefit from some of the advancements made in this movement, like the DIY diabetic insulin pump, and yes, I prefer to see such advancements be open source than locked up in proprietary designs and trade secrets. And there probably is room in health regulation for trimming the red tape anyway even for innovations originating…

I'd say Brad Kuhn got it right in http://ebb.org/bkuhn/blog/2016/08/13/does-not-kill.html including "[Y]our proprietary software has killed people, both quickly and slowly, and your attacks on GPLv3 and software freedom are not only unwarranted, they are clearly part of a political strategy to divert attention from your own industry's bad behavior and graft unfair blame onto FLOSS".

Re: Worst Computer Bugs in History: Therac-25 (2017)

#109

Post by paulie_a »

anonuser123456 wrote: post #83
pjmlp wrote: post #44

Earlier quoted context omitted.

>Probably not. As a company you just disclaim liability in your terms of service. Thankfully EULAs are void in Europe. It is all a matter how big the customer base gets, I am hoping eventually we get something like that EU wide. > The market is still the ultimate decider for quality; if you build a crappy product, expect to get innovated out. If that was true 1 € shops wouldn't exist, but even those products have mor…

> Thankfully EULAs are void in Europe. It's not so clear cut that you should be thankful. The ability of companies to dictate the terms of which users can use their software, affects their risk calculation to produce the product in the first place. It is very likely that very useful but imperfect software will not be written because the risk / reward balance is tilted. Remember, you always have the ability to reject…

As a customer I've found just ignoring all EULAs to be effective on the flip side. They are meaningless in my opinion and I don't give a crap about what it says. I'll use the software as I want.

Re: Worst Computer Bugs in History: Therac-25 (2017)

#110

Post by binbag »

A lot of comments here seem to think having a hardware failsafe to backup the software failsafes is the key thing. In fact it doesn't matter about it being hardware. Hardware can fail too. The keys are 1) redundancy and 2) having multiple different failure modes. Adding hardware failsafe gives you both these, but you get the same level of safety by introducing any second failsafe that used a different method, including a separate software based technique as long as its failure mode is dissimilar and uncorrelated to the first. The best method is to add multiple different methods based on completely different technologies.
Post reply on HN