Infosec's inability to quantify risk • Hacker News Board

Re: Infosec's inability to quantify risk

#21

Post by DCKing » Sat, Jul 25, 2015, 4:25 PM UTC

jdsnape wrote: post #5

I'm not sure I agree - maybe I'm lucky to work in an organisation where we have a good understanding that infsec is all about risk. We build this into pretty much everything we do, and the only way to get stuff done is by demonstrating some reduction in risk. Where we do have a problem, and it's an industry-wide problem, is that there is no real widely applicable methodology for evaluating and quantifying security ri…

Indeed. The suggestion that risk quantification is absent from information security is completely alien to me. I work in information security and I work on risk quantification almost every day.

It is true that there is no objectively agreed on method for risk quantification. But that's somewhat of a red herring. There doesn't have to be an objectively agreed on method for risk quantification to be useful.

Re: Infosec's inability to quantify risk

#22

Post by roflc0ptic » Sat, Jul 25, 2015, 4:25 PM UTC

PowerPete wrote: post #10

Not a fan of this post. Yes, Charlie Miller and Chris Valasek's stunt had relatively low risk, but the difference is that they were risking the lives of innocent people who had nothing to do with it. You don't get to "quantify risk" for others who didn't ask for it, who are completely unassociated with you and your activities. "Business leaders quantify and prioritize risk, but we don't, so our useless advice is igno…

> "You don't get to "quantify risk" for others who didn't ask for it, who are completely unassociated with you and your activities."

To take an example from the article: when you change lanes in a car, you're incurring risk for others "who didn't ask for it, and are completely unassociated with and your activities." I agree that it's generally unacceptable to incur risk to others' lives, but in actual fact we do it often.

Not totally related to your point: on the whole people seem to ignore existing systematic risk and privilege preventing non-systematic risk. Interactions with cars kill so many people! The unmitigated disaster that is climate change - a product of using cars - is in all probability going to kill so many more people! But here we are, having a discussion moralizing about the way some security researchers did or did not misuse their car (fwiw, I'm in the "misused" camp).

The problem here might be irresponsible use of cars by two bad actors, but it might be larger?

Re: Infosec's inability to quantify risk

#23

Post by [deleted] » Sat, Jul 25, 2015, 4:30 PM UTC

[deleted]

Re: Infosec's inability to quantify risk

#24

Post by icebraining » Sat, Jul 25, 2015, 4:30 PM UTC

logicallee wrote: post #7

Infosec is easy to quantify: every security hole should be treated as though the lives of every human on Earth depended on it; also, 100% of software has security holes. If you take the logical conclusion of these two statements, it's obvious that the only winning move is not to play.

every security hole should be treated as though the lives of every human on Earth depended on it

What possible reason would lead you to do that?

Re: Infosec's inability to quantify risk

#25

Post by kbenson » Sat, Jul 25, 2015, 4:32 PM UTC

maxerickson wrote: post #20

kbenson wrote: post #16

What apologetics for this demo seem to gloss over, or worse, not realize is the real problem, is not just the increased risk. It's the increased risk and removal of choice from all the unwitting participants. > In college, I owned a poorly maintained VW bug that would occasionally lose power on the freeway, such as from an electrical connection falling off from vibration. I caused more risk by not maintaining my car…

It isn't that apologetic: In hindsight, it's obvious to everyone that Valasek and Miller went too far. It's not encouraging them to double down and do more live traffic tests, it's encouraging other people to calibrate their reactions a little bit, so as to not lose sight of the very real benefit that came out of the research.

I took it as half apologetic, but really, for that I was addressing what I saw in general from arguments from apologetics on this issue, not specifically this reaction.

> It's not encouraging them to double down and do more live traffic tests, it's encouraging other people to calibrate their reactions a little bit, so as to not lose sight of the very real benefit that came out of the research.

Sure, I think it's extremely important research. I think it's great they did it. But I think the outcry is important too, so as to not lose sight of the very real benefit that comes out of keeping public safety in mind when performing experiments. It sucks that these guys have to be a poster child for this, when a slap on the wrist would suffice, but I think it's important that both sides of the issue aren't discounted because of the other side.

Re: Infosec's inability to quantify risk

#26

Post by pyre » Sat, Jul 25, 2015, 4:35 PM UTC

nly wrote: post #6

This disturbs me. Basically I think he's saying we should accept flawed and unproven technologies in the name of progress because "eh, any new risk is small compared to the risks we already face anyways". The difference between a software hack and bad driving is one of responsibility for the risks we face. When people are negligent, we humans like to see people taking responsibility and facing justice. This is why th…

> Who will be responsible for these deaths?

There is not always a responsible party. What about the child that runs out into the road, and you have no chance to avoid them? Will you decry self-driving cars when this incident happens to them?

Searching for "who is responsible" when the car is self-driving is self-defeating. When the car is driven by an algorithm, the algorithm can be improved. One death could prevent many more. In the case of human drivers, some people refuse to learn from their mistakes, but self-driving cars don't have this issue, and the effects of that learning are much broader than a single driver once the 'lesson' has been learned.

Re: Infosec's inability to quantify risk

#27

Post by blincoln » Sat, Jul 25, 2015, 4:36 PM UTC

sprkyco wrote: post #15

An inability to quantify risk is attributing a skill set to an industry which is probably not responsible for quantifying risk. Infosec researchers should only be beholden to identifying and detailing risks. There are infosec subsets that require better skills to identify risks, but those subsets are the one's more responsible for quantifying risk in an appropriate manner not the ENTIRE industry. Quantifying risk is…

I completely agree. To make things worse, I'm not even sure that it's practical to use an actuarial approach in a way that will produce remotely valid results.

Actuaries based calculations of probability on past occurrences. This is why car insurance is calculated using a huge variety of variables like age of the driver, make/model of the car, etc.

So first of all, where is the data about past compromises going to come from? Just about no organization is willing to share it unless there is a legal requirement to do so. Unless something dramatic changes, that means only the most high-profile events from other organizations will become part of the pool of data to work with.

More importantly, however, IMO there are far too many variables in order to do anything meaningful with that data. When the exploitability of a particular vulnerability can depend on everything from the specific type of hardware the OS hosting the app is hosted on, to the choice of database back-end, so a single configuration setting in an XML or other type of file, how is that even trackable, let alone something that can be calculated with any sort of accuracy?

Re: Infosec's inability to quantify risk

#28

Post by philsnow » Sat, Jul 25, 2015, 4:37 PM UTC

Author equates the risk of one person operating a single vehicle that sometimes loses power to the risk incurred when a whole fleet of cars could be subverted by somebody buying a few hundred dollars worth of hardware and tinkering for a bit.

Re: Infosec's inability to quantify risk

#29

Post by icebraining » Sat, Jul 25, 2015, 4:38 PM UTC

perlgeek wrote: post #17

> In hindsight, it's obvious to everyone that Valasek and Miller went too far. Not at all; they didn't create new risks, they just exposed existing risks. And the security community's reaction isn't their fault, either. And sometimes an industry needs a wakeup call to take a topic serious.

If doing something risky doesn't increase the risk of bad consequences happening, what does? Whether they "created new risks" seems beyond the point of the criticism - this isn't about exposing the flaw, it's about reproducing it in an unsafe way.

Re: Infosec's inability to quantify risk

#30

Post by icebraining » Sat, Jul 25, 2015, 4:40 PM UTC

philsnow wrote: post #28

Author equates the risk of one person operating a single vehicle that sometimes loses power to the risk incurred when a whole fleet of cars could be subverted by somebody buying a few hundred dollars worth of hardware and tinkering for a bit.

No, OP is equating it to the trial the researchers did with a single vehicle.