Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves • Hacker News Board

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#41

Post by mikestew » Wed, Jan 06, 2016, 8:27 PM UTC

mikeash wrote: post #33

mikestew wrote: post #19

Earlier quoted context omitted.

Consider the failure of the car's master cylinder. I realize it's just one example to make a point, but automotive master cylinders don't completely fail all at once since about fifty years ago. What causes a master cylinder to fail is a failure of the seals. But because modern master cylinders have "dual channels" (for lack of a better description), half of it might fail, but it's so unlikely for both sides to fail…

I believe the only steer-by-wire model on the roads right now is some Infiniti. Even that has a clutch which engages the steering mechanically if the car loses power. Electric cars potentially have additional braking redundancy. Even if the brakes failed entirely, they could drive the motor in reverse to stop the car. I don't think any of them actually do this, since total brake failure doesn't happen very often, but…

Yeah, I pretty much whiffed on that one; see reply to sibling comment.

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#42

Post by protomyth » Wed, Jan 06, 2016, 8:35 PM UTC

pc86 wrote: post #37

protomyth wrote: post #26

Earlier quoted context omitted.

Pedestrians on the sidewalk want the car not run them over to avoid killing the Customer.

Fair point, but if the alternative is hitting something that has an as high or higher likelihood of killing the occupants, I'm not sure what the alternative is (don't say "avoid the situation in the first place," that's obvious). Nobody's going to buy an automated vehicle that might intentionally kill them to save a pedestrian.

I think the alternative is pretty much a no go. If some poor pedestrian that was on a side gets hit by a car to preserve the owner then the lawsuits will be huge. Killing bystanders is not going to be acceptable.

Let's look at a picture[1] from an article[2] that made it to HN[3]. If any program chose A as an acceptable solution, then they should be sued into oblivion. Killing the bystander will kill the driverless car.

1) http://www.technologyreview.com/sites/default/files/styles/v...

2) http://www.technologyreview.com/view/542626/why-self-driving...

3) https://news.ycombinator.com/item?id=10439106

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#43

Post by true_religion » Wed, Jan 06, 2016, 8:54 PM UTC

protomyth wrote: post #42

pc86 wrote: post #37

Earlier quoted context omitted.

Fair point, but if the alternative is hitting something that has an as high or higher likelihood of killing the occupants, I'm not sure what the alternative is (don't say "avoid the situation in the first place," that's obvious). Nobody's going to buy an automated vehicle that might intentionally kill them to save a pedestrian.

I think the alternative is pretty much a no go. If some poor pedestrian that was on a side gets hit by a car to preserve the owner then the lawsuits will be huge. Killing bystanders is not going to be acceptable. Let's look at a picture[1] from an article[2] that made it to HN[3]. If any program chose A as an acceptable solution, then they should be sued into oblivion. Killing the bystander will kill the driverless c…

I'd say these aren't simple systems.

The "kill the pedestrian, not me" example is brought up... but what is a real life situation wherein the only two choices in retrospective was 'die' or kill a pedestrian?

Most of the time, you'll avoid killing a pedestrian and it'll result in non-critical injuries to you due to airbags and other safety features. Or it'll result in a dodge or just car damage, since the other cars are more prepared to react to you than a hobbled pedestrian carrying groceries is.

I think often times the car will 'choose' to risk injury you rather than absolutely a pedestrian.

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#44

Post by true_religion » Wed, Jan 06, 2016, 9:04 PM UTC

ghaff wrote: post #27

intopieces wrote: post #22

Earlier quoted context omitted.

When autonomous vehicles encounter an obstacle, they brake. Moreover, why in this problem, that appears in absolutely every thread about self-driving cars, is there a family in a crosswalk with no awareness whatsoever of a speeding vehicle? Are they deaf and blind without assistance? This is not a joke. The car will be aware of the presence of the crosswalk. It will slow long before arriving at it. So the deaf and bl…

I'll concede that there are some scenarios where choices may be required: e.g. Child darts in front of car. Slam on brakes and hope for best or swerve off road (which probably isn't a cliff) or into traffic? And perhaps there is a different calculus if the car isn't carrying a passenger. But, for the most part, these trolley arguments are pretty uninteresting from a technical, as opposed to a philosophical, perspecti…

> e.g. Child darts in front of car. Slam on brakes and hope for best or swerve off road (which probably isn't a cliff) or into traffic? And perhaps there is a different calculus if the car isn't carrying a passenger.

I think its fair to say that the car might have an engine that decides to preserve human life at the cost of human injury.

To avoid killing the child, it may serve into traffic if and only if doing so has a high likelihood of survival for all passengers.

If it can't determine that; it'll take the conservative approach and break to mitigate the damage of the impact to the child.

Regardless the decision engine of the car isn't perfect. I'd have a hard time seeing it predict serving left to cause a 4 car pile up, instead of right to cause a 5 car pile up. How could it know? At some point, it'll just short-circuit its decisions because time is passing, and its critical to do something rather than simply slow down and maintain course.

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#45

Post by cam_l » Thu, Jan 07, 2016, 1:04 AM UTC

protomyth wrote: post #42

pc86 wrote: post #37

Earlier quoted context omitted.

Fair point, but if the alternative is hitting something that has an as high or higher likelihood of killing the occupants, I'm not sure what the alternative is (don't say "avoid the situation in the first place," that's obvious). Nobody's going to buy an automated vehicle that might intentionally kill them to save a pedestrian.

I think the alternative is pretty much a no go. If some poor pedestrian that was on a side gets hit by a car to preserve the owner then the lawsuits will be huge. Killing bystanders is not going to be acceptable. Let's look at a picture[1] from an article[2] that made it to HN[3]. If any program chose A as an acceptable solution, then they should be sued into oblivion. Killing the bystander will kill the driverless c…

People drive SUV's, which have a much higher likelihood than ordinary cars of killing the occupants of other vehicles or pedestrians. No lawsuits. Just for perspective, the lawsuit you are talking about is the family of one dead pedestrian involved in a freak accident suing a car company. I mean, how long could a car company hide the effects of proprietary closed software engineered specifically for flouting the law.

..also from your picture and as I have learned from TWD, if you see a herd of zombies on the road your self driving car should probably be programmed to drive straight through them.

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#46

Post by protomyth » Thu, Jan 07, 2016, 3:10 AM UTC

cam_l wrote: post #45

protomyth wrote: post #42

Earlier quoted context omitted.

I think the alternative is pretty much a no go. If some poor pedestrian that was on a side gets hit by a car to preserve the owner then the lawsuits will be huge. Killing bystanders is not going to be acceptable. Let's look at a picture[1] from an article[2] that made it to HN[3]. If any program chose A as an acceptable solution, then they should be sued into oblivion. Killing the bystander will kill the driverless c…

People drive SUV's, which have a much higher likelihood than ordinary cars of killing the occupants of other vehicles or pedestrians. No lawsuits. Just for perspective, the lawsuit you are talking about is the family of one dead pedestrian involved in a freak accident suing a car company. I mean, how long could a car company hide the effects of proprietary closed software engineered specifically for flouting the law.…

That's not equivalent. We are talking about a vehicle that is programmed from the manufacture to do something. The SUV driver is responsible for his/her actions. When we are talking about a self driving car, we now have machines that decide what to do and killing the bystander tends not to be a sympathetic item in from of the jury.

As to the second part... I'm not sure how the software will judge the depth of the hoard to see if avoidance is a better answer than just pushing through. Also, a couple of zombies could really mess up a Tesla versus a SUV or Jeep. Zombie Apocalypse mode would be a much hard problem.

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#47

Post by cam_l » Thu, Jan 07, 2016, 11:19 PM UTC

protomyth wrote: post #46

cam_l wrote: post #45

Earlier quoted context omitted.

People drive SUV's, which have a much higher likelihood than ordinary cars of killing the occupants of other vehicles or pedestrians. No lawsuits. Just for perspective, the lawsuit you are talking about is the family of one dead pedestrian involved in a freak accident suing a car company. I mean, how long could a car company hide the effects of proprietary closed software engineered specifically for flouting the law.…

That's not equivalent. We are talking about a vehicle that is programmed from the manufacture to do something. The SUV driver is responsible for his/her actions. When we are talking about a self driving car, we now have machines that decide what to do and killing the bystander tends not to be a sympathetic item in from of the jury. As to the second part... I'm not sure how the software will judge the depth of the hoa…

"The SUV driver is responsible for his/her actions."

Hey, that is what ford said about the Bronco II.

Anyway, I agree with you in principle that if presented with the clear evidence in court it would be as you say. My contention is it would take many years and a whistle-blower before anything like that would ever come to light. And there is much greater likelihood of deaths occurring accidentally due to the code, than by design.

Depth of herd is an issue, to be sure, and this even assumes the car could tell the difference between zombies and humans.. perhaps we need a waze for herd avoidance.

Re: Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

#48

Post by protomyth » Sat, Jan 09, 2016, 7:15 PM UTC

cam_l wrote: post #47

protomyth wrote: post #46

Earlier quoted context omitted.

That's not equivalent. We are talking about a vehicle that is programmed from the manufacture to do something. The SUV driver is responsible for his/her actions. When we are talking about a self driving car, we now have machines that decide what to do and killing the bystander tends not to be a sympathetic item in from of the jury. As to the second part... I'm not sure how the software will judge the depth of the hoa…

"The SUV driver is responsible for his/her actions." Hey, that is what ford said about the Bronco II. Anyway, I agree with you in principle that if presented with the clear evidence in court it would be as you say. My contention is it would take many years and a whistle-blower before anything like that would ever come to light. And there is much greater likelihood of deaths occurring accidentally due to the code, tha…

> Hey, that is what ford said about the Bronco II.

Programming is treated a bit different from a tire specs

I get the feeling a lot of programmers are going to be put on witness stands pretty early in the roll out. Lawyers already like to interview Sys Admins.

> Depth of herd is an issue, to be sure, and this even assumes the car could tell the difference between zombies and humans.. perhaps we need a waze for herd avoidance.

Classically, an IR sensor should tell the car the difference between zombies and humans. Herd avoidance is probably a better tactic because of resource depletion.