Live data from Hacker News

“I Want to Know What Code Is Running Inside My Body”

backchannel.com

51–60 of 217 posts

Re: “I Want to Know What Code Is Running Inside My Body”

#51
post #31
post #26

Philosophy - She doesn't know the code that's running on machines inside her body. I don't even know the code that's running my heart. And yet, I trust it.

The latter has undergone far more testing than the former.

The bug fix turnaround makes waterfall look downright lean, though.

Re: “I Want to Know What Code Is Running Inside My Body”

#52
post #48

Earlier quoted context omitted.

Maybe in a complete vacuum, but in reality, having access to source certainly makes it easier to look for vulnerabilities, and if the same software is in many devices, the cost of finding vulnerabilities is amortized. Security by obscurity obviously doesn't stop a determined attacker, but it does raise the barrier to entry for script kiddies.

No. Tell me, how many vulnerabilities are running wild on Linux, the software that powers... well, pretty much anything (including the servers through which you read this content)? Even if you find a vulnerability, it gets patched within hours and it may take a day or two for it to be distributed to everyone. > Security by obscurity [...] does raise the barrier to entry for script kiddies. Which script can help you f…

If the same vulnerability is present across a large range of devices, a public exploit has a much larger impact.

I'm not advocating for security by obscurity in the slightest because on balance I think it's bad, but we should acknowledge that publishing your source does change the potential cost of mounting an attack in various scenarios, and some of them might actually favor obscurity.

Nobody except the most determined attacker will attack a device with some custom, unpublished code. On the other hand, popular software have a variety of exploits in the wild because their popularity makes them more attractive targets, one consequence of which is enabling script kiddies (since the hard work can be outsourced).

> it gets patched within hours and it may take a day or two for it to be distributed to everyone

This doesn't work for embedded devices. Hence why it might not be a great idea to publish their vulnerabilities, or tell the whole world that you're running on old vulnerable source.

Re: “I Want to Know What Code Is Running Inside My Body”

#53

Sure this is about pacemakers, but cant we say something similar about the rest of our body?

Not similar at all actually. You're talking about software written over a couple of years (by humans) that is supporting our current biology versus biological processes that have "optimized" us over tens of thousands of years.

Re: “I Want to Know What Code Is Running Inside My Body”

#54

Does anyone know if at least the FDA is allowed to review the source code for pacemakers? Or is it a complete blackbox? Personally I would be appalled if even the FDA is not allowed to.

A google search turned up "General Principles of Software Validation; Final Guidance for Industry and FDA Staff"[1]

My understanding is that they don't review the code, but they do review all of the validation that goes into making sure the code does what it should.

[1]http://www.fda.gov/RegulatoryInformation/Guidances/ucm085281...

Re: “I Want to Know What Code Is Running Inside My Body”

#55
post #28

Does anyone know if at least the FDA is allowed to review the source code for pacemakers? Or is it a complete blackbox? Personally I would be appalled if even the FDA is not allowed to.

With or without a warrant...? EDIT: Not sure what's up w/ the downvotes. There are well-established ways for regulatory agencies (whether FDA, FCC, etc) to obtain firmware for devices -- and it almost always involves a warrant under extraneous circumstances -vs- proactively receiving proprietary code.

What if they had to subpoena a tomato grown in a field next to the toxic waste dump? What if they opted not to examine that tomato because they didn't have the resources to issue, process, and support, the lengthy bureaucratic process involved in such things?

Re: “I Want to Know What Code Is Running Inside My Body”

#56

Earlier quoted context omitted.

Maybe in a complete vacuum, but in reality, having access to source certainly makes it easier to look for vulnerabilities, and if the same software is in many devices, the cost of finding vulnerabilities is amortized. Security by obscurity obviously doesn't stop a determined attacker, but it does raise the barrier to entry for script kiddies.

Anyone you're likely to classify as a "script kiddy" is not going to be able to read the kind of code going into embedded devices like a pacemaker to a deep enough level to find any problems. And if they can the software is really problematic, most likely. Security by obscurity is never a good idea, but especially not when it might prevent a white hat from finding a bug that would allow a malicious actor to remotely…

I agree with you in general, but since we're talking about embedded devices that can't be updated, here's a concrete scenario:

1) White hat finds a vulnerability in the source code which applies to a large number of devices. 2) Source is patched but vulnerable devices exist in wild

Now all an attacker needs to do is find a vulnerable device; because the source code is public like OP suggests, figuring out which devices are vulnerable is trivial.

Unless I'm missing something drastic, this is actually a problem in the embedded space where obscurity seems to help.

Re: “I Want to Know What Code Is Running Inside My Body”

#57

Does anyone know if at least the FDA is allowed to review the source code for pacemakers? Or is it a complete blackbox? Personally I would be appalled if even the FDA is not allowed to.

I used to work at a diagnostic medical equipment company. Rules were slightly different for us (diagnostic rather than therapeutic = lower bar) and we had FDA audits. They were basically making sure we had written down our processes to an adequate degree, and checking that we actually followed what we wrote down. They most definitely were not doing code audits (which isn't their job or area of expertise).

Re: “I Want to Know What Code Is Running Inside My Body”

#58
post #36

Earlier quoted context omitted.

Do you want to control the device, or do you want someone else to control your devices? If you're OK with someone else (who probably doesn't have your well-being anywhere in their list of priorities) controlling all of your devices, then the answer is no, you are not required to have access to the source code and output data. EDIT: I didn't realize this was such a controversial statement. I stand by it, though; even…

In theory, open sourcing pacemaker software make sense, but in practice, the pool of people qualified to review and edit that software may not be very large. Most people would still be depending on someone else to control their devices in any case.

If I had a pacemaker I would most definitely review the source code for it if I could get access to it. Even if it meant learning golang or ARM assembly or whatever the kids use to build pacemakers these days.

I'm sure I'd lack a lot of the heart mechanics side of the equation, but I would be very highly incentivised to hunt out programming bugs.

Re: “I Want to Know What Code Is Running Inside My Body”

#59
post #14

Earlier quoted context omitted.

> If your company takes ~2 years to develop a pacemaker's software, it's not to your advantage to let your competitors catch up. Why should the patient who has the pacemaker implanted care? This seems like a clear situation in which the patient's interests trump everybody else's. Pacemaker manufacturers should be competing in how well their devices meet patient needs. Closed source doesn't meet a key patient need.

Medical device software is subject to a level of rigor in development process that little else is. Exactly what patient need do you believe open source would meet that is not being met by the current closed-source development process?

The need to avoid Therac-25 [1] type bugs in the proprietary code that are fatal to the patient. Open source in conjunction with a bug bounty would make me far more confident than simply trusting that some large corporate has got their shit wired tight when history throws up so many counter-examples.

[1] https://en.wikipedia.org/wiki/Therac-25

Re: “I Want to Know What Code Is Running Inside My Body”

#60
post #26

Philosophy - She doesn't know the code that's running on machines inside her body. I don't even know the code that's running my heart. And yet, I trust it.

Except your code and her code are quite similar, so introspection has significant predictive value.
Post reply on HN