Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“I Want to Know What Code Is Running Inside My Body”

backchannel.com

Post reply on HN
Print view View original

111–120 of 217 posts

Re: “I Want to Know What Code Is Running Inside My Body”

#111

Post by j-pb »

TickleSteve wrote: post #99

You most certainly don't want people to be able to modify safety critical code within a pacemaker. What most developers don't realise is the level of engineering strictness that goes into anything safety-related. The rules and regulations related to anything that affects the human body is in a different league than what most developers are familiar with. What is a problem here, is that the design (not the code) appar…

Software certification is a joke, and mostly consists of checking off a few boxes on workflow and development methods.

Smart electricity meters are safety critical and I know students writing on it in their summer jobs. Airplanes are safety critical and they accidentally bridged the on plane wifi into the fly by wire system.

Unless this stuff is written in COQ and formally verified, I won't trust it a bit.

Re: “I Want to Know What Code Is Running Inside My Body”

#112

Post by TickleSteve »

j-pb wrote: post #111
TickleSteve wrote: post #99

You most certainly don't want people to be able to modify safety critical code within a pacemaker. What most developers don't realise is the level of engineering strictness that goes into anything safety-related. The rules and regulations related to anything that affects the human body is in a different league than what most developers are familiar with. What is a problem here, is that the design (not the code) appar…

Software certification is a joke, and mostly consists of checking off a few boxes on workflow and development methods. Smart electricity meters are safety critical and I know students writing on it in their summer jobs. Airplanes are safety critical and they accidentally bridged the on plane wifi into the fly by wire system. Unless this stuff is written in COQ and formally verified, I won't trust it a bit.

Certification most certainly isn't a joke.

It involves a hell of a lot of work, enough to double or triple the software portion of a project.

Whatever certification you have been involved with may have been a joke.

Safety-critical certification most certainly is not.

BTW: I'm s/w architect on a set of smart meters. And they most certainly are not safety critical. (IMO They should be, but thats a separate issue).

...and no Wifi was "accidently" bridged. evidence?

Re: “I Want to Know What Code Is Running Inside My Body”

#113

Post by TickleSteve »

adrianN wrote: post #109
TickleSteve wrote: post #107

Earlier quoted context omitted.

Its only a moral hazard if you don't trust the people who are certifying the system and therefore aren't subject to the obscurity. The trust question is the problem with obscurity. Do you trust the people making it obscure? In this particular case, where safety-critical standards are relatively well known (within the industry) and not themselves obscured, they deserve to be trusted.

Just like we trusted the people certifying VW's engine control software?

Thats the point.

Do we trust them or not?

The problem with VW was not a 'bug'. it was a malicious design.

Re: “I Want to Know What Code Is Running Inside My Body”

#114

Post by ashitlerferad »

visarga wrote: post #96
yostrovs wrote: post #77

Earlier quoted context omitted.

Because Linux was supposed to become a great thing, but instead it remained a paradise for geeks to do what they think is best. Software built to make money, on the other hand, was built to improve things like ease of use, aesthetics, and buyer's happiness, because that's what buyers were looking for. The open-source people never really cared about the dumb people and lay folks, the ignoramuses that didn't care to le…

I was under the impression that Mac OS, iOS, Android and recently Windows all run "Linux" under the hood nowadays. Not to mention the presence of Linux in the cloud, which is essential for the functioning of most websites, apps and mobile devices. Am I wrong? Basically, it's easier to tell where there is no Linux than the opposite. That surely doesn't sound like such a failure. It's as if all the other OSes and syste…

Only Android actually uses Linux. macOS/iOS are proprietary Apple things with bits of mach and FreeBSD. The Windows thing you are probably thinking of is that Microsoft reimplemented various Linux APIs, so software built for Linux can run under Windows without Linux being involved.

Re: “I Want to Know What Code Is Running Inside My Body”

#115

Post by TickleSteve »

adrianN wrote: post #108
TickleSteve wrote: post #99

You most certainly don't want people to be able to modify safety critical code within a pacemaker. What most developers don't realise is the level of engineering strictness that goes into anything safety-related. The rules and regulations related to anything that affects the human body is in a different league than what most developers are familiar with. What is a problem here, is that the design (not the code) appar…

Industrial control systems are also safety critical systems that have to adhere to very similar regulations as medical systems. Yet, when you ask hackers in that field you'll quickly learn that they have terrible code and abysmal security. Rules and regulations do very little to improve code quality and security, imho. Most things in these regulations are either best practices that any software engineer does (write t…

industrial control != safety critical.

"industrial control" covers both certified and non-certified code, you will have to be more specific than that for the purposes of this conversation.

Quite often, cases such as the pacemaker are flaws in system design, not code. (e.g. no secure messaging, probably due to the lack of awareness when it was designed).

That is not to say that safety-critical code is perfect... just that it has a lot more rigour and inspection involved than run-of-the-mill website code.

True, the expense and overhead does indeed affect the level of change that is acceptable to a business, but given that change is allowed, the quality of that change is what we're discussing here and all I'm saying is that there is a lot more rigour involved than most developers here are aware of.

Re: “I Want to Know What Code Is Running Inside My Body”

#116

Post by TickleSteve »

flukus wrote: post #110
adrianN wrote: post #108

Earlier quoted context omitted.

Industrial control systems are also safety critical systems that have to adhere to very similar regulations as medical systems. Yet, when you ask hackers in that field you'll quickly learn that they have terrible code and abysmal security. Rules and regulations do very little to improve code quality and security, imho. Most things in these regulations are either best practices that any software engineer does (write t…

>Rules and regulations do very little to improve code quality and security, imho. In fact, I'd say the restrict them. Adding a tonne of paperwork is a great way to prevent people from "noticing" problems.

When the paperwork is describing the system design from high level down to low-level to sufficient detail that an external auditor can use it to examine the source then no, that is not true.

The security/safety processes (as applicable) are examined at every level from system-design all the way down to implementation.

Its not perfect, but its best practice we have for secure/safe product development.

Re: “I Want to Know What Code Is Running Inside My Body”

#117

Post by MichaelBurge »

visarga wrote: post #96
yostrovs wrote: post #77

Earlier quoted context omitted.

Because Linux was supposed to become a great thing, but instead it remained a paradise for geeks to do what they think is best. Software built to make money, on the other hand, was built to improve things like ease of use, aesthetics, and buyer's happiness, because that's what buyers were looking for. The open-source people never really cared about the dumb people and lay folks, the ignoramuses that didn't care to le…

I was under the impression that Mac OS, iOS, Android and recently Windows all run "Linux" under the hood nowadays. Not to mention the presence of Linux in the cloud, which is essential for the functioning of most websites, apps and mobile devices. Am I wrong? Basically, it's easier to tell where there is no Linux than the opposite. That surely doesn't sound like such a failure. It's as if all the other OSes and syste…

Windows is not Linux "under the hood".

Re: “I Want to Know What Code Is Running Inside My Body”

#118

Post by nyan4 »

jordigh wrote: post #61
randyrand wrote: post #20

By extension should every device I own require me to have access to the source code and output data? Not a rhetorical question.

Y'know, back in the early 2000's and the days of Slashdot, it was quite common to find people who advocated for free software everywhere. Now we find people who like yourself have to specify that the radical position that all software should be free is something worthy of serious consideration. That they're not joking or trying to be deliberately provocative. What happened to us? Why did we go from boasting about ins…

> Why did we go from [...] talking about how all software will some day be free to being afraid to seriously consider the proposition?

HN is clearly not attracting much of the FLOSS crowd.

Re: “I Want to Know What Code Is Running Inside My Body”

#119

Post by lordnacho »

TickleSteve wrote: post #97
cooper12 wrote: post #75

Earlier quoted context omitted.

The sad part is that the companies will use this security by obscurity argument against open sourcing.

contrary to popular opinion.... Obscurity is good practice as one layer of a layered defence system. See "Defence in Depth" https://en.wikipedia.org/wiki/Defense_in_depth_(computing) "Defense in depth is originally a military strategy that seeks to delay rather than prevent the advance of an attacker by yielding space to buy time". We have to acknowledge that no system is perfect, there will always be holes, therefor…

I think you're right that it can be used as a layer, but the reason we admonish against security-by-obscurity is that when you hide something, you often put less work into securing it properly.

It's like when you leave a key for someone under a door mat. You don't often consider that the door might be easily kicked in by an intruder.

Re: “I Want to Know What Code Is Running Inside My Body”

#120

Post by mchaver »

TickleSteve wrote: post #99

You most certainly don't want people to be able to modify safety critical code within a pacemaker. What most developers don't realise is the level of engineering strictness that goes into anything safety-related. The rules and regulations related to anything that affects the human body is in a different league than what most developers are familiar with. What is a problem here, is that the design (not the code) appar…

Software Freedom Law Centers claims that the FDA requirements for medical devices put most of the responsibility of quality assurance on the manufacturer. It does not seem like the FDA provides much of an assurance of quality. Note: this is not my field of expertise, there is good chance I am very wrong, just following a short paper trail of laws, there maybe more laws (or practices) that I am missing that may indicate different FDA regulation controls for medical devices.

https://www.softwarefreedom.org/resources/2010/transparent-m...

https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFR...

Post reply on HN