Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“I Want to Know What Code Is Running Inside My Body”

backchannel.com

Post reply on HN
Print view View original

141–150 of 217 posts

Re: “I Want to Know What Code Is Running Inside My Body”

#141

Post by TickleSteve »

michaelt wrote: post #129
TickleSteve wrote: post #107

Earlier quoted context omitted.

Its only a moral hazard if you don't trust the people who are certifying the system and therefore aren't subject to the obscurity. The trust question is the problem with obscurity. Do you trust the people making it obscure? In this particular case, where safety-critical standards are relatively well known (within the industry) and not themselves obscured, they deserve to be trusted.

As long as "independent certification" companies are selected in a competitive market and paid by the system makers, they can't remove moral hazard - only shift it around. After all, if you're a system maker, why would you hire hardasses who have rejected your products in the past? And if you're a certification house, why would you $$$ on many hours from experienced engineers when you could use fewer hours and junior…

Very true... that is a real effect...

But the government(s) also employs similar agencies to perform the enforcement of the certifications.

Its not a perfect system, but its a lot better than the developer-on-the-street realises.

Re: “I Want to Know What Code Is Running Inside My Body”

#142

Post by TeMPOraL »

nitrogen wrote: post #98
TickleSteve wrote: post #97

Earlier quoted context omitted.

contrary to popular opinion.... Obscurity is good practice as one layer of a layered defence system. See "Defence in Depth" https://en.wikipedia.org/wiki/Defense_in_depth_(computing) "Defense in depth is originally a military strategy that seeks to delay rather than prevent the advance of an attacker by yielding space to buy time". We have to acknowledge that no system is perfect, there will always be holes, therefor…

True, but history has demonstrated countless times that closed source code doesn't provide near enough obscurity to deter hackers, and automated fuzzing tools make it even easier.

It also demonstrates that obscurity can significantly reduce the number of attack attempts that are made against you. See e.g. why people move SSH to non-standard ports - raising the entry bar for the attackers has some value.

Re: “I Want to Know What Code Is Running Inside My Body”

#143

Post by TickleSteve »

throwanem wrote: post #133
TickleSteve wrote: post #112

Earlier quoted context omitted.

Certification most certainly isn't a joke. It involves a hell of a lot of work, enough to double or triple the software portion of a project. Whatever certification you have been involved with may have been a joke. Safety-critical certification most certainly is not. BTW: I'm s/w architect on a set of smart meters. And they most certainly are not safety critical. (IMO They should be, but thats a separate issue). ...a…

Not Wifi, but IIRC a guy called Chris Roberts was able to leverage a wireline connection to an underseat IFE controller into FADEC control on multiple aircraft. Last I heard of the case, he might be going to prison for it, so apparently at least some people find him credible. Similarly, witness recent revelations about car hackability - I forget if it was Blackhat or DEFCON where a couple of guys demoed a fully remot…

A few points here:

- Given physical access, all bets are off.

- "safety critical" refers to software that has a SIL level(https://en.wikipedia.org/wiki/Safety_integrity_level). Note the list of IEC/ANSI/EN standards.

- Yes, the hack of the braking and steering system via the entertainment system is a breakdown of the certification system. That should have been found earlier. This is why critical systems are air-gapped. I personally have never worked on automotive systems so am not familiar with the regulations there, but I was/am extremely surprised this was possible.

Re: “I Want to Know What Code Is Running Inside My Body”

#144

Post by Illniyar »

throwanem wrote: post #138
Illniyar wrote: post #91

Earlier quoted context omitted.

I think the difference between these two are 2-fold - first is ownership and second is personal. A pacemaker is something you bought and owned, when flying in a plane you are buying a service, this is similar to earlier discussions about being able to change your car's software under DMCA etc... A pacemaker is also personal, in that it's something that only you have and for your specific pacemaker the only affected p…

> A pacemaker is something you bought and owned Sure about that? How much did you pay for, and how much did your insurance cover? What's the proportion look like? Before you downvote, note: I don't like this argument. I find it downright horrifying. But I can't imagine no one will ever make it in a serious way, so it bears considering how to respond.

I guess bought is subjective here, but ownership is not. And you can still own something if you got it as a gift or through other means.

If no one can take it from you (or you need to return it) without your consent then you probably own it.

It might be a little vague legally, but we aren't talking about legal issues here (since it's perfectly legal to not have access to the sourcecode of something you own), but rather a perception issue.

Re: “I Want to Know What Code Is Running Inside My Body”

#145

Post by ef4 »

TickleSteve wrote: post #99

You most certainly don't want people to be able to modify safety critical code within a pacemaker. What most developers don't realise is the level of engineering strictness that goes into anything safety-related. The rules and regulations related to anything that affects the human body is in a different league than what most developers are familiar with. What is a problem here, is that the design (not the code) appar…

> everything is open to them, source included

Certifiers typically don't review source code. They are operating at least two levels of abstraction away: they review documents that describe procedures that are supposed to ensure quality.

> If the code was open-sourced, don't expect to find lots of buffer overflow attack vectors

That is a big fat citation needed. Researchers working without access to source code have demonstrated multiple vulnerabilities in safety-critical medical devices. We have no way of knowing how many more they would find if they also had access to the source.

Re: “I Want to Know What Code Is Running Inside My Body”

#146

Post by TeMPOraL »

throwanem wrote: post #136
ben_jones wrote: post #55

Earlier quoted context omitted.

What if they had to subpoena a tomato grown in a field next to the toxic waste dump? What if they opted not to examine that tomato because they didn't have the resources to issue, process, and support, the lengthy bureaucratic process involved in such things?

To extend the metaphor, if the farm that grew the tomato is selling them to people to eat, I don't see that a subpoena is or should be required. But it breaks down anyway, because a tomato's source code is right there in it, with no opaque binary blobs to worry about trying to decompile.

Nitpick, but what you call tomato's "source code" actually is an opaque binary (or rather, quaternary) blob.

Re: “I Want to Know What Code Is Running Inside My Body”

#147

Post by TeMPOraL »

jsmthrowaway wrote: post #94
gravypod wrote: post #33

Earlier quoted context omitted.

Yea it's got some cool genetic algorithms.

One could frame the entire computing industry as a distributed genetic algorithm, executed by the real computers in order to understand themselves and the environment around them. One could further posit that we don't really have a good handle on the right fitness function yet. (I realize this sounds like a low-effort joke, but think about it for a second.)

Well, computing today seems like primordial soup flowing through the pipes set up by Moloch[0] - we keep doing random shit, somewhat directed by economic incentives.

[0] - http://slatestarcodex.com/2014/07/30/meditations-on-moloch/

Re: “I Want to Know What Code Is Running Inside My Body”

#148

Post by TeMPOraL »

throwanem wrote: post #138
Illniyar wrote: post #91

Earlier quoted context omitted.

I think the difference between these two are 2-fold - first is ownership and second is personal. A pacemaker is something you bought and owned, when flying in a plane you are buying a service, this is similar to earlier discussions about being able to change your car's software under DMCA etc... A pacemaker is also personal, in that it's something that only you have and for your specific pacemaker the only affected p…

> A pacemaker is something you bought and owned Sure about that? How much did you pay for, and how much did your insurance cover? What's the proportion look like? Before you downvote, note: I don't like this argument. I find it downright horrifying. But I can't imagine no one will ever make it in a serious way, so it bears considering how to respond.

I don't like this argument but for another reason - it implies a horrible direction things will most definitely go. The pacemaker will become a service. Just like your car, your house and your washing machine. Oh sorry, not your anymore - soon we'll all be renting them, because there's every business incentive for that to happen, and close to zero incentives that would stop it.

Re: “I Want to Know What Code Is Running Inside My Body”

#149

Post by TickleSteve »

munificent wrote: post #131
TickleSteve wrote: post #115

Earlier quoted context omitted.

industrial control != safety critical. "industrial control" covers both certified and non-certified code, you will have to be more specific than that for the purposes of this conversation. Quite often, cases such as the pacemaker are flaws in system design , not code . (e.g. no secure messaging, probably due to the lack of awareness when it was designed). That is not to say that safety-critical code is perfect... jus…

> That is not to say that safety-critical code is perfect... just that it has a lot more rigour and inspection involved than run-of-the-mill website code. I had assumed that as well until all of the horror stories around Toyota's firmware came to light. https://en.wikipedia.org/wiki/2009%E2%80%9311_Toyota_vehicle...

Unfortunately, we are human...

Yes, the toyota case is a well publicised case. Consider though, the number of safety critical systems that are out there performing perfectly everyday. Of course, that is not proof of much, but the fact that you can name the Toyota case (and probably the Therac 25 case) means that the process generally works.

Re: “I Want to Know What Code Is Running Inside My Body”

#150

Post by TickleSteve »

xenophonf wrote: post #135
TickleSteve wrote: post #112

Earlier quoted context omitted.

Certification most certainly isn't a joke. It involves a hell of a lot of work, enough to double or triple the software portion of a project. Whatever certification you have been involved with may have been a joke. Safety-critical certification most certainly is not. BTW: I'm s/w architect on a set of smart meters. And they most certainly are not safety critical. (IMO They should be, but thats a separate issue). ...a…

I'm not sure, but your correspondent may be referring to Hugo Teso's 2013 presentation for Hack in the Box, http://conference.hitb.org/hitbsecconf2013ams/hugo-teso/ , or possibly to design revisions of the Boeing 777.

If so:

"The described technique cannot engage or control the aircraft's autopilot system using the FMS or prevent a pilot from overriding the autopilot," the FAA's statement explained. "Therefore, a hacker cannot obtain 'full control of an aircraft' as the technology consultant has claimed."

"The statement went on to explain that although Teso may have been able to exploit aviation software running on a simulator, as he described in his presentation, the same approach wouldn't work on software running on certified flight hardware."

http://www.theregister.co.uk/2013/04/13/faa_debunks_android_...

Post reply on HN