Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tesla Autopilot Crash: Why We Should Worry About a Single Death

spectrum.ieee.org

Post reply on HN
Print view View original

61–70 of 77 posts

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#61

Post by greglindahl »

banku_brougham wrote: post #56

The responses in this thread all seem to hold the assumption that these flotillas of self driving cars will have a low failure rate. The Tesla crash makes it easy to imagine a 100 car freeway pile-ups caused by AI drivers discovering novel ways to crash. Driving headlong into the side of a trailer because the color looked like the sky? The possibilities are endless, and the subsequent media attention and reactionary…

Are you saying there hasn't already been a 100-car freeway pileup caused by (now-standard) cruise control? Or that human-driven cars don't drive into the side of a trailer because the driver didn't see it?

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#62

Post by espadrine »

> According to research, we generally believe that we’re above-average in intelligence, and that we’re above-average in driving skills. (By definition of “average,” obviously that’s impossible.)

It actually is possible. It is a common mistake to confuse statistical average and median. For instance, 75% of the following numbers (1, 1, 1, 0) are above average. More than 90% of humans have an above average number of legs.

Sure, it is difficult for the general public to understand academic results. Still, level zero of academic reporting is to link to the original paper, instead of playing a game of Chinese whispers by linking to an article that links to an article that links to the paper.

What the paper really points to is that people assume by default that strangers are worse at driving than they are, which ironically is what is happening with autopilot instead of strangers.

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#63

Post by SomeCallMeTim »

aetherson wrote: post #31
SomeCallMeTim wrote: post #26

Earlier quoted context omitted.

The article is arguing for inaction in the Trolley Problem [1]. In short: You have a choice between saving five people and letting one other person die, or doing nothing and letting five people die. Which is a controversial position, but not stupid exactly. I would call it misguided, and certainly not a very utilitarian position. And given the options I personally would vote for the utilitarian answer: Save five peop…

Do you similarly advocate that we should, right now, today, in real life policy, set up a program where we check people for matches to multiple people needing organs, and if one healthy person's organs can save the lives of 2+ people, murder that healthy person and take his/her organs?

Are you implying that the correct response is to let the five people die in the Trolley Problem, then? To avoid getting your moral hands dirty?

No, obviously I don't advocate that we go around arbitrarily murdering people to potentially improve the lives of others. But there's a huge difference between searching out and selecting specific people and killing them and simply changing the number and identity of people who will be killed by random events.

It's the difference between premeditated murder and a no-fault accident. Policy/law decisions must necessarily make choices like this, flipping the switch from tracks that typically have five or five hundred people standing on them to tracks that typically have one, for the good of the five or five hundred.

Like air bags that kill some people but save thousands more. Or vaccines that kill (a very small number) of kids (typically with allergies) but save millions.

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#64

Post by branchan »

uola wrote: post #59
madmax96 wrote: post #42

>We can’t cut corners just because we want to rush a life-saving product to market. But there's a huge difference between a self-driving car and a novel drug. These cars are using statistical learning mechanisms -- it is impossible to prove that the car will always function correctly. >Yes, it could be that autonomous cars will save many more lives than they take. But “the ends justify the means” is a dangerous appro…

Computer software can also be updated, yet almost everything is broken. Rushing self-driving cars is not pragmatic, it's keeping the lack of safety culture that driving already has. Whatever lives you save in the short run is likely to be lost many times over on the margin of the standard we expect from self-driving cars. Pragmatic safety would be things like speed and distance enforcement. That's how self-driving is…

Can you elaborate on 'everything is broken'?

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#65

Post by branchan »

SomeCallMeTim wrote: post #26
fennecfoxen wrote: post #3

Article argues, "autopilot could let people who are blind and disabled drive now, where they couldn't drive before. but they could never get killed in a car accident while driving before, so this is ethically questionable." Article's logic is dumber than a sack of hair.

The article is arguing for inaction in the Trolley Problem [1]. In short: You have a choice between saving five people and letting one other person die, or doing nothing and letting five people die. Which is a controversial position, but not stupid exactly. I would call it misguided, and certainly not a very utilitarian position. And given the options I personally would vote for the utilitarian answer: Save five peop…

This sounds ridiculous. The car's job should always be to protect the occupant of the vehicle. Can you imagine what would happen to a car company if it was known that the car willfully choose to kill the customer if it meant that multiple other lives would be saved?

Another scenario, imagine you are driving on a freeway but a couple people decide to hop onto the freeway and cross it on foot. Would you rather just run them over or ram your own car and kill yourself just so that 2 people aren't harmed?

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#66

Post by aetherson »

SomeCallMeTim wrote: post #63
aetherson wrote: post #31

Earlier quoted context omitted.

Do you similarly advocate that we should, right now, today, in real life policy, set up a program where we check people for matches to multiple people needing organs, and if one healthy person's organs can save the lives of 2+ people, murder that healthy person and take his/her organs?

Are you implying that the correct response is to let the five people die in the Trolley Problem, then? To avoid getting your moral hands dirty? No, obviously I don't advocate that we go around arbitrarily murdering people to potentially improve the lives of others. But there's a huge difference between searching out and selecting specific people and killing them and simply changing the number and identity of people w…

What I'm implying is that the Trolley Car Dilemma is a toy example that is very divorced from actual life, which makes consequentialism seem more compelling than other examples closer to implementable policy do.

The fundamental conceit that you can add and subtract lives, or happiness, or the like as though they were values in a formula is core to consequentialism, and I don't think that people are actually willing to follow through with the implications of that when it stops being at a safe remove.

I don't understand the bright line you're drawing between "murder someone by pointing a runaway trolley that would otherwise absolutely miss them in their direction" and "murder someone by harvesting their organs."

You can (and, I'd argue, should) believe that air bags are a good idea without believing that the reason that air bags are a good idea is that you can add and subtract lives. For example, you could say, "If YOU get into an accident, it is far more likely that an airbag will save you than it will kill you," which is a different thing than saying, "An air bag will kill YOU, but it will save two other people." And there are many other moral theories which would suggest that air bags are good for reasons other than that "I killed Bob but saved Alice and Charlie" is a generalizable rule for moral good.

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#67

Post by SomeCallMeTim »

aetherson wrote: post #66
SomeCallMeTim wrote: post #63

Earlier quoted context omitted.

Are you implying that the correct response is to let the five people die in the Trolley Problem, then? To avoid getting your moral hands dirty? No, obviously I don't advocate that we go around arbitrarily murdering people to potentially improve the lives of others. But there's a huge difference between searching out and selecting specific people and killing them and simply changing the number and identity of people w…

What I'm implying is that the Trolley Car Dilemma is a toy example that is very divorced from actual life, which makes consequentialism seem more compelling than other examples closer to implementable policy do. The fundamental conceit that you can add and subtract lives, or happiness, or the like as though they were values in a formula is core to consequentialism, and I don't think that people are actually willing t…

>I don't understand the bright line you're drawing between "murder someone by pointing a runaway trolley that would otherwise absolutely miss them in their direction" and "murder someone by harvesting their organs."

The difference is key to understanding policy decisions like this.

You absolutely use math with lives when you're making policy. Right down to calculating lives with dollars.

If there is a fixed budget of $100M to apply to saving people's lives, you ideally will divvy the money around in such a way that the most people live. You don't extend one person's life by 1 month using extreme measures that cost $50M and let a hundred die who would could have had their lives extended by years for $50k each.

To some degree the original question is a real-world policy example of the trolley problem: Do we allow companies like Tesla to save thousands of lives, possibly your own, in exchange for indemnifying them against the (ideally) few lives lost (also potentially your own) when the technology fails? That sounds identical to the air bag policy: In both cases the life saved could be yours. You might not even get into an accident if there were enough self-driving cars, so not only would you live where you might have died, you might even avoid an inconvenience.

I'm not talking about seeking out people and killing them. That is committing murder for an alleged "greater good", and the ends don't justify the means. I'm talking about redirecting a trolley that is going to kill five random people and instead making it kill only one random person. And we can talk about focusing on further advances to make sure that one person doesn't die (statistically speaking) either, but I think that throwing the switch, in a policy sense, is always the right first step.

I think the moral argument that flipping the trolley switch is wrong is bankrupt, and that it does map onto the real world and real world policy decisions like requiring mandatory air bags. Flipping the switch isn't murder. Not flipping the switch when you have the opportunity is murder.

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#68

Post by whamlastxmas »

Maybe we can worry about self driving car deaths once they surpass train related deaths. So far they're safer than multi-million dollar machines operated by highly trained professionals, directed by other highly trained professionals, and are limited to only moving two directions on a set rail.

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#69

Post by uola »

branchan wrote: post #64
uola wrote: post #59

Earlier quoted context omitted.

Computer software can also be updated, yet almost everything is broken. Rushing self-driving cars is not pragmatic, it's keeping the lack of safety culture that driving already has. Whatever lives you save in the short run is likely to be lost many times over on the margin of the standard we expect from self-driving cars. Pragmatic safety would be things like speed and distance enforcement. That's how self-driving is…

Can you elaborate on 'everything is broken'?

There's a supreme mismatch between the state of software (and infrastructure) and how we use it today. If nothing else people should be concerned about this because it makes the Internet de facto not open, since most people can't afford the resources (like security) to run many service.

You would think that sending something to a machine on the Internet would imply that it would actually reach the recipient and that no one else is listening. Not only is that not true, it was essentially never intended to be true. We work very hard to try and make it so, but will probably never reach a point were it will be equal to what it could have been if that was the intention from the beginning.

https://medium.com/message/everything-is-broken-81e5f33a24e1 http://swiftonsecurity.tumblr.com/post/98675308034/a-story-a...

Re: Tesla Autopilot Crash: Why We Should Worry About a Single Death

#70

Post by cmotzakut6 »

greglindahl wrote: post #61
banku_brougham wrote: post #56

The responses in this thread all seem to hold the assumption that these flotillas of self driving cars will have a low failure rate. The Tesla crash makes it easy to imagine a 100 car freeway pile-ups caused by AI drivers discovering novel ways to crash. Driving headlong into the side of a trailer because the color looked like the sky? The possibilities are endless, and the subsequent media attention and reactionary…

Are you saying there hasn't already been a 100-car freeway pileup caused by (now-standard) cruise control? Or that human-driven cars don't drive into the side of a trailer because the driver didn't see it?

>Or that human-driven cars don't drive into the side of a trailer because the driver didn't see it?

I won't claim that that never happens, but I doubt it's very common. If you were to take 1000 random car crash deaths I do not think a very large proportion of them are caused by people running head-on into a sideways tractor trailer that they had a long distance to notice and react to.

Tesla says this is the only fatality in 130 million miles of Autopilot driving, and that in the US there is a fatality every 94 million miles of driving overall. But it's known that fatalities are more common in risky driving scenarios where Autopilot is not used, so this is not an apples-to-apples comparison (https://www.progressive.com/newsroom/article/2002/may/fivemi..., http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1449863/). If the appropriate adjustment is 40% or more, then the Tesla is riskier than human driving in the "non-risky" scenarios. The fact that the accident here was an uncommon failure mode for humans, and the Tesla was at fault, should give a hint as to which direction the numbers need to be adjusted.

Post reply on HN