Will your self-driving car be programmed to kill you? • Hacker News Board

Re: Will your self-driving car be programmed to kill you?

#11

Post by ajuc » Tue, Jun 16, 2015, 1:35 PM UTC

tyho wrote: post #8

angdis wrote: post #5

hmm... perhaps "the default" should be programmed maximize human life as a headcount. But maybe they could sell a pricey "upgrade option" that would cause the computer to favor preserving the life of the driver as top priority. The proceeds from these upgrade options could then go into pay-outs for the families of people killed by drivers with this option enabled?

/s, right?

If not officialy I'm sure there will be unofficial path to do this.

Re: Will your self-driving car be programmed to kill you?

#12

Post by angdis » Tue, Jun 16, 2015, 1:38 PM UTC

Malic wrote: post #9

thomasfl wrote: post #2

Sounds like a plot for a sci-fi novel, but things get serious when software can kill you.

Software flaws have been know to kill, so "killing software" has been with us for awhile. For more details, check out Peter G Neumann's "Computer-Related Risks". Automatic doors crushing people, radiation equipment (for the treatment of cancer) going way out of spec, and many more.

Yes, there are interesting lessons in that, but we're entering new territory when software systems will be explicitly tasked to make life-or-death "decisions" without the active control of a human. The correct answer might be ethically foggy even for humans. This is definitely problematic.

Re: Will your self-driving car be programmed to kill you?

#13

Post by ajuc » Tue, Jun 16, 2015, 1:38 PM UTC

agumonkey wrote: post #3

A superior intelligence planning to kill is not superior. I'd love a computer to answer 'lol, boring' when asked to harm humanity.

There are situations where you can't not kill, you just can choose who to kill.

Opposite of "planning to kill" is "killing randomly".

Re: Will your self-driving car be programmed to kill you?

#14

Post by imgabe » Tue, Jun 16, 2015, 1:40 PM UTC

It's an interesting thought experiment, but I think a real situation with such a stark binary choice is so improbable that it's barely worth considering.

Cars are not on trolley tracks and computers can brake or swerve harder and faster than any human could.

Re: Will your self-driving car be programmed to kill you?

#15

Post by andreasklinger » Tue, Jun 16, 2015, 1:40 PM UTC

pestaa wrote: post #7

agumonkey wrote: post #3

A superior intelligence planning to kill is not superior. I'd love a computer to answer 'lol, boring' when asked to harm humanity.

So when humans mount horses and go into the woods to hunt as entertainment, they are not superior either?

op meant superior to us.

we are relative superior to foxes - who hunt for necessity, superior to us might not hunt at all

i agree btw

i am not worried that a superior dreadful AI will come to kill humanity, but that some little half-assed AI in charge of some minor side activity will not be capable of understand context beyond it's own complexity.

Re: Will your self-driving car be programmed to kill you?

#16

Post by jlebrech » Tue, Jun 16, 2015, 1:40 PM UTC

don't program it to swerve in any direction, brakes and tires should do the trick.

Re: Will your self-driving car be programmed to kill you?

#17

Post by kailuowang » Tue, Jun 16, 2015, 1:44 PM UTC

pestaa wrote: post #7

agumonkey wrote: post #3

A superior intelligence planning to kill is not superior. I'd love a computer to answer 'lol, boring' when asked to harm humanity.

So when humans mount horses and go into the woods to hunt as entertainment, they are not superior either?

Computer may have the same intelligence as human, but their fundamental inner drive will be completely different. Human like any other animal has a deep genetically programmed drive: survive and reproduce. Hunting as entertainment, for example, is actually practicing for survival - all hunting animals do that. This deep drive is the result of evolution. AI on the other hand has a completely different evolution process - that is, us. We will ultimately decide that deep drive for AI.

Re: Will your self-driving car be programmed to kill you?

#18

Post by jfoster » Tue, Jun 16, 2015, 1:44 PM UTC

If people become aware enough of this, the answer is almost certainly that cars will choose the life of their passengers in any situation, unless manufacturers will agree not to compete on this.

Re: Will your self-driving car be programmed to kill you?

#19

Post by rjbwork » Tue, Jun 16, 2015, 1:45 PM UTC

angdis wrote: post #5

hmm... perhaps "the default" should be programmed maximize human life as a headcount. But maybe they could sell a pricey "upgrade option" that would cause the computer to favor preserving the life of the driver as top priority. The proceeds from these upgrade options could then go into pay-outs for the families of people killed by drivers with this option enabled?

Oh, how very Swiftian. Brilliant.

Re: Will your self-driving car be programmed to kill you?

#20

Post by andreasklinger » Tue, Jun 16, 2015, 1:46 PM UTC

ajuc wrote: post #13

agumonkey wrote: post #3

A superior intelligence planning to kill is not superior. I'd love a computer to answer 'lol, boring' when asked to harm humanity.

There are situations where you can't not kill, you just can choose who to kill. Opposite of "planning to kill" is "killing randomly".

you meant "killing accidentally" hopefully? :)