Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“I Want to Know What Code Is Running Inside My Body”

backchannel.com

Post reply on HN
Print view View original

101–110 of 217 posts

Re: “I Want to Know What Code Is Running Inside My Body”

#101

Post by TickleSteve »

nitrogen wrote: post #98
TickleSteve wrote: post #97

Earlier quoted context omitted.

contrary to popular opinion.... Obscurity is good practice as one layer of a layered defence system. See "Defence in Depth" https://en.wikipedia.org/wiki/Defense_in_depth_(computing) "Defense in depth is originally a military strategy that seeks to delay rather than prevent the advance of an attacker by yielding space to buy time". We have to acknowledge that no system is perfect, there will always be holes, therefor…

True, but history has demonstrated countless times that closed source code doesn't provide near enough obscurity to deter hackers, and automated fuzzing tools make it even easier.

Of course, obscurity on its own is not a defence.

its one part of a whole system, and in that, if it does delay or deter (even slightly) then it has worked.

Re: “I Want to Know What Code Is Running Inside My Body”

#102

Post by seehafer »

jordigh wrote: post #92
seehafer wrote: post #90

Earlier quoted context omitted.

As someone who has been on the receiving end of several FDA audits, I would really like to know on what basis you're saying all of this, because everything in my 10 years of experience of doing this is contrary to what you've said.

Maybe we worked in different fields? I only did 3 years, only received one FDA audit, but many other audits from pharmaceutical companies. I was in the more diagnostic side, not therapeutic, although we did have some safety checks where we had to quickly raise an alarm if our analysis showed a potential medical emergency. Did the FDA actually check your software or just your documentation? Did the auditors very caref…

> Maybe we worked in different fields?

Class III devices (pacemakers, deep brain stimulators, insulin pumps)

> Did the FDA actually check your software or just your documentation?

In a submission, occasionally. In an audit, it depended on the nature of an audit. I've been part of one audit that was due to persistent SW failure and yes, they checked code.

> Did you find the process of writing documentation instrumental in ensuring that your software was carefully tested?

No, but that's not really the point of documentation.

> Did you ensure that your tests were reproducible and comprehensive?

Yes, we could kill people if we screwed something up.

> They were more concerned with making sure signatures and dates were correct and that our documentation didn't make us look bad.

This practice is common, I grant. Doesn't mean it's universal or right.

> The FDA doesn't really give a damn about what I really worked on, do they?

FDA heavily triages activity based on risk. If your product is life-sustaining they will definitely be all up in your business.

Re: “I Want to Know What Code Is Running Inside My Body”

#103

Post by lukasLansky »

TickleSteve wrote: post #97
cooper12 wrote: post #75

Earlier quoted context omitted.

The sad part is that the companies will use this security by obscurity argument against open sourcing.

contrary to popular opinion.... Obscurity is good practice as one layer of a layered defence system. See "Defence in Depth" https://en.wikipedia.org/wiki/Defense_in_depth_(computing) "Defense in depth is originally a military strategy that seeks to delay rather than prevent the advance of an attacker by yielding space to buy time". We have to acknowledge that no system is perfect, there will always be holes, therefor…

Theoretically yes, but the problem with obscurity is that it creates moral hazard by lowering visibility of all other measures that are or are not taken to protect the system. It's not unreasonable to decide that such extra layer of protection is not worth of making your decision makers subject to being able to cut corners as there is no feedback loop for them.

Re: “I Want to Know What Code Is Running Inside My Body”

#104

Post by mankash666 »

woodruffw wrote: post #13
mankash666 wrote: post #4

Reasons to NOT open up the code: 1> Loss of competitive advantage 2> Open source is not necessarily any safer (heartbleed bug ... ) 3> If software for the pacemaker is allowed to be updated like that on a computer, someone will update it with buggy software that can cause adverse side effects. Who owns the liability in that case?

Even if we assume that these arguments are sound (which they aren't), they do not appear to provide a proper justification for letting a human being die . Put another way: If you had to explain Marie Moe's death to one of her relatives, which of your three points would make the relative understand your position?

Your argument that opening up the source code might have prevented her death is facetious.

Re: “I Want to Know What Code Is Running Inside My Body”

#105

Post by ComodoHacker »

kiba wrote: post #8
SubiculumCode wrote: post #5

Sure this is about pacemakers, but cant we say something similar about the rest of our body?

Unfortunately, it's a black box made from nature that we have to reverse engineer.

And it's full of bugs and vulnerabilities our adversaries of all kinds are actively using.

On the good side, bugs are fixed and vulnerabilities closed eventually. But the process is so slow... It spans numerous minor releases.

Re: “I Want to Know What Code Is Running Inside My Body”

#106

Post by vacri »

jordigh wrote: post #83
vacri wrote: post #57

Earlier quoted context omitted.

I used to work at a diagnostic medical equipment company. Rules were slightly different for us (diagnostic rather than therapeutic = lower bar) and we had FDA audits. They were basically making sure we had written down our processes to an adequate degree, and checking that we actually followed what we wrote down. They most definitely were not doing code audits (which isn't their job or area of expertise).

> and checking that we actually followed what we wrote down We never got that. For the most part, checking that we followed what we wrote down involved reading more documents that said that we did what we wrote down. The auditors weren't even allowed to walk around freely in our office, nor even be left alone without supervision, and we were all carefully coached on how to answer interviews with non-compromising resp…

Our FDA inspector hung around the QA manager and staff, didn't go into R&D much at all, and spent some time on the factory floor checking processes.

I guess that they function as regular police do - the actual experience is less than ideal and leaves a lot to be desired, but without any at all, the world would be horrible.

Re: “I Want to Know What Code Is Running Inside My Body”

#107

Post by TickleSteve »

lukasLansky wrote: post #103
TickleSteve wrote: post #97

Earlier quoted context omitted.

contrary to popular opinion.... Obscurity is good practice as one layer of a layered defence system. See "Defence in Depth" https://en.wikipedia.org/wiki/Defense_in_depth_(computing) "Defense in depth is originally a military strategy that seeks to delay rather than prevent the advance of an attacker by yielding space to buy time". We have to acknowledge that no system is perfect, there will always be holes, therefor…

Theoretically yes, but the problem with obscurity is that it creates moral hazard by lowering visibility of all other measures that are or are not taken to protect the system. It's not unreasonable to decide that such extra layer of protection is not worth of making your decision makers subject to being able to cut corners as there is no feedback loop for them.

Its only a moral hazard if you don't trust the people who are certifying the system and therefore aren't subject to the obscurity.

The trust question is the problem with obscurity. Do you trust the people making it obscure?

In this particular case, where safety-critical standards are relatively well known (within the industry) and not themselves obscured, they deserve to be trusted.

Re: “I Want to Know What Code Is Running Inside My Body”

#108

Post by adrianN »

TickleSteve wrote: post #99

You most certainly don't want people to be able to modify safety critical code within a pacemaker. What most developers don't realise is the level of engineering strictness that goes into anything safety-related. The rules and regulations related to anything that affects the human body is in a different league than what most developers are familiar with. What is a problem here, is that the design (not the code) appar…

Industrial control systems are also safety critical systems that have to adhere to very similar regulations as medical systems. Yet, when you ask hackers in that field you'll quickly learn that they have terrible code and abysmal security. Rules and regulations do very little to improve code quality and security, imho. Most things in these regulations are either best practices that any software engineer does (write tests, for example, or do code reviews), or just checkboxes for the QA to check.

I'd argue that over time safety critical systems have worse code than normal software because refactoring is almost impossibly expensive due to all the paperwork that each software change ensues.

Re: “I Want to Know What Code Is Running Inside My Body”

#109

Post by adrianN »

TickleSteve wrote: post #107
lukasLansky wrote: post #103

Earlier quoted context omitted.

Theoretically yes, but the problem with obscurity is that it creates moral hazard by lowering visibility of all other measures that are or are not taken to protect the system. It's not unreasonable to decide that such extra layer of protection is not worth of making your decision makers subject to being able to cut corners as there is no feedback loop for them.

Its only a moral hazard if you don't trust the people who are certifying the system and therefore aren't subject to the obscurity. The trust question is the problem with obscurity. Do you trust the people making it obscure? In this particular case, where safety-critical standards are relatively well known (within the industry) and not themselves obscured, they deserve to be trusted.

Just like we trusted the people certifying VW's engine control software?

Re: “I Want to Know What Code Is Running Inside My Body”

#110

Post by flukus »

adrianN wrote: post #108
TickleSteve wrote: post #99

You most certainly don't want people to be able to modify safety critical code within a pacemaker. What most developers don't realise is the level of engineering strictness that goes into anything safety-related. The rules and regulations related to anything that affects the human body is in a different league than what most developers are familiar with. What is a problem here, is that the design (not the code) appar…

Industrial control systems are also safety critical systems that have to adhere to very similar regulations as medical systems. Yet, when you ask hackers in that field you'll quickly learn that they have terrible code and abysmal security. Rules and regulations do very little to improve code quality and security, imho. Most things in these regulations are either best practices that any software engineer does (write t…

>Rules and regulations do very little to improve code quality and security, imho.

In fact, I'd say the restrict them. Adding a tonne of paperwork is a great way to prevent people from "noticing" problems.

Post reply on HN