Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? • Hacker News Board

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#21

Post by Avshalom » Fri, Aug 17, 2018, 9:08 PM UTC

Anderkent wrote: post #13

Sidestepping comedy for a bit, there are a lot of inscrutable systems that we connect to 'things that matter' all the time. The financial systems themselves are pretty damn inscrutable. Corporations are very often inscrutable.

I feel like you're making his point for him.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#22

Post by stouset » Fri, Aug 17, 2018, 9:09 PM UTC

wyldfire wrote: post #15

Avshalom wrote: post #3

James Mickens is pure gold https://mickens.seas.harvard.edu/wisdom-james-mickens

This guy looks like the one who wrote those satire magazine-style articles.

He is that guy. Mickens is a legend.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#23

Post by eli_gottlieb » Fri, Aug 17, 2018, 9:10 PM UTC

Anderkent wrote: post #13

Sidestepping comedy for a bit, there are a lot of inscrutable systems that we connect to 'things that matter' all the time. The financial systems themselves are pretty damn inscrutable. Corporations are very often inscrutable.

To quote, "This has made a lot of people very angry, and been widely regarded as a bad move."

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#24

Post by ballenf » Fri, Aug 17, 2018, 9:37 PM UTC

a1369209993 wrote: post #16

KZeillmann wrote: post #7

Regarding the "we don't know how this stuff works" point, doesn't the FDA approve a ton of drugs where we don't know the exact mechanism of how it works? Do we need to know exactly and precisely how something works to know _that_ it works?

The overwhelming majority of medical treatments don't have inteligent humans actively trying to maliciously sabatoge them. Many drugs can be made horrendously lethal or otherwise dangerous with little effort (often just by significantly increasing the dosage) but we don't need to care very much because it's not possible to silently untracably apply that effort from arbitrarily far away, and there usually isn't anythi…

At the end of Mickens' talk he suggests putting black box devices behind smart firewalls and routers. I'd say that's the equivalent role of doctors.

That is, the trifecta of bad decisions is black box functionality connected to an internet of hate (or unfiltered/tested input data) and given levers of power in society. Take away any one of those 3 and you're probably ok.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#25

Post by tptacek » Fri, Aug 17, 2018, 9:41 PM UTC

Anderkent wrote: post #13

Sidestepping comedy for a bit, there are a lot of inscrutable systems that we connect to 'things that matter' all the time. The financial systems themselves are pretty damn inscrutable. Corporations are very often inscrutable.

The word Mickens uses is "interpretable", which financial infrastructure is, and ML models are not.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#26

Post by dmoy » Fri, Aug 17, 2018, 9:43 PM UTC

CharlesW wrote: post #2

"Using case studies involving machine learning and other hastily-executed figments of Silicon Valley's imagination, I will explain why computer security (and larger notions of ethical computing) are difficult to achieve if developers insist on literally not questioning anything that they do since even brief introspection would reduce the frequency of git commits." For anyone who hasn't heard a James Mickens talk, do…

Every time I watch or listen to a james mickens thing, it physically pains me that he is SO CLOSE to my office, but there's basically no chance in hell of us poaching him. Oh well, at least we get glorious comedic writing and talks at a frequent basis.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#27

Post by badloginagain » Fri, Aug 17, 2018, 9:49 PM UTC

Anderkent wrote: post #13

Sidestepping comedy for a bit, there are a lot of inscrutable systems that we connect to 'things that matter' all the time. The financial systems themselves are pretty damn inscrutable. Corporations are very often inscrutable.

There is also a lot of regulation around those entities because they're known to be inscrutable and are expected to be if no regulation existed.

Technologists tend to think that tech is value-neutral, and will therefore give good outcomes.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#28

Post by TallGuyShort » Fri, Aug 17, 2018, 9:50 PM UTC

KZeillmann wrote: post #7

Regarding the "we don't know how this stuff works" point, doesn't the FDA approve a ton of drugs where we don't know the exact mechanism of how it works? Do we need to know exactly and precisely how something works to know _that_ it works?

In some cases, no. But let's say there was a drug for 99% of people, but totally messed up 1% of people. Well then it's probably really worth understanding how to predict or avoid that. Because the same thing happens with image classifiers. Google had an embarrassing incident where their classifier was very impressive, until it got it wrong and it was really embarrassing. Do you just accept that sometimes image classifiers act racist? Or would it be nice to have a tool that can highlight what parts of the picture contributed most to the classification, so you can identify pictures that would have prevented that error in the training set?

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#29

Post by Animats » Fri, Aug 17, 2018, 9:53 PM UTC

Fixing security is quite possible.

Install a backdoor, go to jail for "exceeding authorized access".

Fail to fix an security bug, get sued for negligence.

Make it public policy that license contracts cannot override those responsibilities.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#30

Post by alasdair_ » Fri, Aug 17, 2018, 10:03 PM UTC

Animats wrote: post #29

Fixing security is quite possible. Install a backdoor, go to jail for "exceeding authorized access". Fail to fix an security bug, get sued for negligence. Make it public policy that license contracts cannot override those responsibilities.

>Make it public policy that license contracts cannot override those responsibilities.

This would be a disaster for open source. Who wants to write software for free if you can get sued for a bug?