Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autopilot: an open source driving agent

github.com

Post reply on HN
Print view View original

21–30 of 129 posts

Re: Autopilot: an open source driving agent

#21

Post by antisthenes »

pmoriarty wrote: post #14

I read an interesting ethical dilemma related to self-driving cars recently. Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver. What choice should the car be programmed to make? Who should decide what that choice is? Who is ethically responsible for that choice? In a normal car that is driven by a hum…

Other threads related to autopilot (mainly Tesla), with great posts that explained in detail why this problem is unlikely to arise in reality and that the resultant action undertaken by the system would be the same in both scenarios (e.g. apply brakes as hard as possible).

IIRC part of the explanation was that a human is unlikely to make a qualified choice in a situation like this anyway, so programming a decision matrix based on the utility of the target into the crash avoidance mechanism is moot. Something like 99.999% of the safety benefit would be achieved by a faster brake response time.

Re: Autopilot: an open source driving agent

#22

Post by maxerickson »

pmoriarty wrote: post #14

I read an interesting ethical dilemma related to self-driving cars recently. Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver. What choice should the car be programmed to make? Who should decide what that choice is? Who is ethically responsible for that choice? In a normal car that is driven by a hum…

If it can actually decide, it should probably just decide at random.

Then we can adjust speed limits and move dumb walls and so on to reduce the deaths attributed to random selection by an autodrive system. Kind of like we do with deaths attributed to poor driving.

(I say if in the first paragraph because you need at least 4 conditions to be true: the information to make the choice is available, the vehicle is sophisticated enough to make the choice, there is enough time to make a maneuver and there is not enough time to make a maneuver that saves everybody)

Re: Autopilot: an open source driving agent

#23

Post by jws »

pmoriarty wrote: post #14

I read an interesting ethical dilemma related to self-driving cars recently. Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver. What choice should the car be programmed to make? Who should decide what that choice is? Who is ethically responsible for that choice? In a normal car that is driven by a hum…

Not to nitpick your dilemma, there will always be a dilemma, but at any speed where children in the road are likely ((Ok, I used a "head on collision" number because I couldn't find a wall number, but the physics are probably pretty close. Numbers from a New Zealand Ministry of Transport PDF from 2012: http://www.transport.govt.nz/assets/Import/Documents/_versio... )

Re: Autopilot: an open source driving agent

#25

Post by maxerickson »

shostack wrote: post #17
pmoriarty wrote: post #14

I read an interesting ethical dilemma related to self-driving cars recently. Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver. What choice should the car be programmed to make? Who should decide what that choice is? Who is ethically responsible for that choice? In a normal car that is driven by a hum…

To further complicate it...would this be standardized? Would a cheap economy car opt to kill the driver, but a Mercedes try to preserve the driver at all cost? How much say would insurance companies that insure the automakers have? What about the government? Also, what level of information about how this logic works is necessary to share with a potential buyer to judge them sufficiently informed? I know I'd sure as h…

And what if the child's parents have paid Mercedes to avoid hitting their kid? You think you are safe in your golden Mercedes, but not when the kid's got Platinum Protection on his phone!

Re: Autopilot: an open source driving agent

#26

Post by skuhn »

antisthenes wrote: post #21
pmoriarty wrote: post #14

I read an interesting ethical dilemma related to self-driving cars recently. Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver. What choice should the car be programmed to make? Who should decide what that choice is? Who is ethically responsible for that choice? In a normal car that is driven by a hum…

Other threads related to autopilot (mainly Tesla), with great posts that explained in detail why this problem is unlikely to arise in reality and that the resultant action undertaken by the system would be the same in both scenarios (e.g. apply brakes as hard as possible). IIRC part of the explanation was that a human is unlikely to make a qualified choice in a situation like this anyway, so programming a decision ma…

It's also pointless to worry about because the proposed dilemma greatly overestimates the ability of computer vision systems to determine what an object actually is.

These systems are looking for obstacles and road contours. They don't know that the obstruction in the road is a human baby, let alone the composition of an adjacent wall.

Re: Autopilot: an open source driving agent

#27

Post by hasbroslasher »

Matthias247 wrote: post #7

Really, a safety critical hard realtime system with control loops on top of Python, a general purpose OS and a CAN-USB adapter? There are reasons why we have realtime operating systems, deterministic bus systems and dedicated CPUs for these kinds of applications. That might be an interesting research, prototyping or simulation platform. But nothing you want to have on a real public road.

I'm not really following. If it works then why not use a control loop on top of Python? We are engineers, after all.

Re: Autopilot: an open source driving agent

#28

Post by skuhn »

jws wrote: post #23
pmoriarty wrote: post #14

I read an interesting ethical dilemma related to self-driving cars recently. Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver. What choice should the car be programmed to make? Who should decide what that choice is? Who is ethically responsible for that choice? In a normal car that is driven by a hum…

Not to nitpick your dilemma, there will always be a dilemma, but at any speed where children in the road are likely ( (Ok, I used a "head on collision" number because I couldn't find a wall number, but the physics are probably pretty close. Numbers from a New Zealand Ministry of Transport PDF from 2012: http://www.transport.govt.nz/assets/Import/Documents/_versio... )

What if behind that wall are a ton more babies? Now we'll have to put x-ray machines in self-driving cars too.

Re: Autopilot: an open source driving agent

#29

Post by mikeash »

throwaway729 wrote: post #20
56245623456 wrote: post #12

Earlier quoted context omitted.

It explicitly states: "THIS IS ALPHA QUALITY SOFTWARE FOR RESEARCH PURPOSES ONLY. THIS IS NOT A PRODUCT. YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS. NO WARRANTY EXPRESSED OR IMPLIED."

And yet comma ai were definitely testing their products on public roads, products which were presumably running this code.

This would certainly explain why they folded so quickly once the NHTSA started asking questions.

Re: Autopilot: an open source driving agent

#30

Post by breck »

Matthias247 wrote: post #7

Really, a safety critical hard realtime system with control loops on top of Python, a general purpose OS and a CAN-USB adapter? There are reasons why we have realtime operating systems, deterministic bus systems and dedicated CPUs for these kinds of applications. That might be an interesting research, prototyping or simulation platform. But nothing you want to have on a real public road.

Why not?

We entrust humans with operating cars right now. Humans in a sense are running a general purpose OS.

Post reply on HN