Live data from Hacker News

Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

usenix.org

11–20 of 184 posts

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#11
post #8
post #3

James Mickens is pure gold https://mickens.seas.harvard.edu/wisdom-james-mickens

He is like that in real life, too... I am in his department, and every single social event ends with him surrounded by a group of people, listening to his hilarious rants. At the same time, he's a great teacher, too. Somehow you learn things while listening to 90 minutes of his stand-up comedy.

He was like that at MSR also. Microsoft lost someone great when he moved to Harvard.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#12
post #2

"Using case studies involving machine learning and other hastily-executed figments of Silicon Valley's imagination, I will explain why computer security (and larger notions of ethical computing) are difficult to achieve if developers insist on literally not questioning anything that they do since even brief introspection would reduce the frequency of git commits." For anyone who hasn't heard a James Mickens talk, do…

He's the guy who wrote the Slow Winter. He's hilarious.

https://www.usenix.org/system/files/1309_14-17_mickens.pdf

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#16

Regarding the "we don't know how this stuff works" point, doesn't the FDA approve a ton of drugs where we don't know the exact mechanism of how it works? Do we need to know exactly and precisely how something works to know _that_ it works?

The overwhelming majority of medical treatments don't have inteligent humans actively trying to maliciously sabatoge them. Many drugs can be made horrendously lethal or otherwise dangerous with little effort (often just by significantly increasing the dosage) but we don't need to care very much because it's not possible to silently untracably apply that effort from arbitrarily far away, and there usually isn't anything to gain from doing so even if it were possible.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#17

Sidestepping comedy for a bit, there are a lot of inscrutable systems that we connect to 'things that matter' all the time. The financial systems themselves are pretty damn inscrutable. Corporations are very often inscrutable.

But corporations and financial systems can often be analyzed in terms of the incentives of the humans involved, or at least the human incentives pushing back against changes to the crappy status quo. When they behave badly, they usually do so in predictable ways because there are humans in the loop somewhere.

By contrast, there is nothing remotely resembling a human mind anywhere in machine learning, and the failure modes are often, by our standards, insane (like thinking a picture of random noise is a cat). That creates a whole new level of danger when connecting to "things that matter".

And to the extent we already connect inscrutable systems to things that matter, we should be trying to make that problem better, not worse. "When you're in a hole, the first thing to do is stop digging"

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#18
post #12
post #2

"Using case studies involving machine learning and other hastily-executed figments of Silicon Valley's imagination, I will explain why computer security (and larger notions of ethical computing) are difficult to achieve if developers insist on literally not questioning anything that they do since even brief introspection would reduce the frequency of git commits." For anyone who hasn't heard a James Mickens talk, do…

He's the guy who wrote the Slow Winter. He's hilarious. https://www.usenix.org/system/files/1309_14-17_mickens.pdf

In exchange for their last remaining bits of entropy, the branches cast evil spells on future genera- tions of processors ... The point is that the branches, those vanquished foes from long ago, would have the last laugh.

More true than ever, now.

Re: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

#20

Regarding the "we don't know how this stuff works" point, doesn't the FDA approve a ton of drugs where we don't know the exact mechanism of how it works? Do we need to know exactly and precisely how something works to know _that_ it works?

No, but drugs spend literal years in testing through various models and animal subjects before even being considered for years more of human trials. Then the benefits are weighed up against the known side-effects and if they stack up the drug is approved. Even after that sometimes the analysis is wrong and the drug turns out to be ineffective or even harmful.

Technology companies churn out things with barely a few weeks of testing at times and no oversight.

Post reply on HN