Why Self Driving Cars Must Be Programmed to Kill • Hacker News Board

Re: Why Self Driving Cars Must Be Programmed to Kill

#11

Post by cpncrunch » Mon, Oct 26, 2015, 2:15 PM UTC

sokoloff wrote: post #9

cpncrunch wrote: post #5

Earlier quoted context omitted.

At 30mph? Or are we assuming 10 pedestrians are somehow running out onto a major highway? I'm struggling to see how this is a valid scenario. If you can think of one, please share...

Two kids run out into the street chasing a ball. Kill the kids or the occupant? Even one kid runs out into the street. Hit the kid straight ahead or risk the occupant(s) by crashing the car or turning the car into an unknown area? Car is about to be struck by a red light/stop light runner from the left. Swerve off the road to prevent it (into an area of "unknown contents, possibly including pedestrians") or just take…

>Two kids run out into the street chasing a ball. Kill the kids or the occupant?

As I mentioned in my previous post, how exactly is that going to kill the driver at 30mph? There isn't going to be a continuous brick wall at the side of the road, otherwise where the heck did the kids come from?!

I think the article is a little overhyped. Certainly there are cases where the car will need to take evasive action, but it's unlikely it will have to decide who to kill.

Re: Why Self Driving Cars Must Be Programmed to Kill

#12

Post by sokoloff » Mon, Oct 26, 2015, 10:55 PM UTC

cpncrunch wrote: post #11

sokoloff wrote: post #9

Earlier quoted context omitted.

Two kids run out into the street chasing a ball. Kill the kids or the occupant? Even one kid runs out into the street. Hit the kid straight ahead or risk the occupant(s) by crashing the car or turning the car into an unknown area? Car is about to be struck by a red light/stop light runner from the left. Swerve off the road to prevent it (into an area of "unknown contents, possibly including pedestrians") or just take…

>Two kids run out into the street chasing a ball. Kill the kids or the occupant? As I mentioned in my previous post, how exactly is that going to kill the driver at 30mph? There isn't going to be a continuous brick wall at the side of the road, otherwise where the heck did the kids come from?! I think the article is a little overhyped. Certainly there are cases where the car will need to take evasive action, but it's…

Agreed that even a continuous brick wall would be relatively safe for an approximately tangent collision at 30 mph.

The concern would come from a break in a guardrail or a series of concrete columns/bollards with a pedestrian crossing, where the car could strike an abutment. (Think of "Jersey barriers" lined up in a construction zone through which a zebra crossing lies. That's an easy abutment scenario and striking a line of Jersey barriers end-on at 30mph provides a deceleration approximately equal to a 30 mph head-on collision with an opposite direction car of equal mass also traveling at 30 mph. That's going to be a massive shunt.)

I agree there's an overhyping in the title, "programmed to kill", but the decision of "cause a near-certain harm to a pedestrian" vs "cause the car to leave the known roadway and suffer/inflict unknown and unknowable harm" is a very legitimate concern.

(I worked as an intern for Daimler on a completely autonomous bus. Our only fallback was "mash the red E-stop button [shutting off the computer servos] and manually take over", so we literally never let the vehicle exceed what the human safety driver was comfortable with, but eventually that's going to go away for fully autonomous vehicles.)

Re: Why Self Driving Cars Must Be Programmed to Kill

#13

Post by mcv » Tue, Oct 27, 2015, 10:37 AM UTC

sokoloff wrote: post #10

mcv wrote: post #8

Earlier quoted context omitted.

In areas where this is likely to happen, cars don't go that fast and should have plenty of time to stop. If pedestrians are actively trying to commit suicide and the only way to prevent that is to kill the driver or an innocent bystander, then hitting the suicidal pedestrians is preferable. More generally, I think the car should try to stay on the road and it its lane whenever possible, and only leave that space when…

The question the article raises is not "should an autonomous vehicle ever use its brakes to avoid an accident?" but rather "in a situation where brakes are insufficient and the controller of the vehicle needs to make a decision about how to weigh the risks to various people, how should those decisions be made?"

That doesn't change the fact that this is a situation that should not occur. When it happens, someone fucked up. Maybe pedestrians are throwing themselves into oncoming traffic, or the car has already made some pretty terrible mistakes.