Tesla confirms “Autopilot” crash in Montana • Hacker News Board

Re: Tesla confirms “Autopilot” crash in Montana

#161

Post by kirrent » Wed, Jul 13, 2016, 6:25 AM UTC

ethanbond wrote: post #69

yladiz wrote: post #56

Earlier quoted context omitted.

It doesn't matter what they advocate for, it matters the perception of the technology. Just because Tesla doesn't explicitly say that it's a perfect system and says that the driver has to be aware at all times doesn't mean that actually happens, and with a lenient time frame without hands on the wheel it's not surprising that drivers would essentially give control to the car.

What I'm saying is I don't think it has anything to do with calling it autopilot, and even less to do with it not meeting the technical definition of autopilot. People are using it in this way because they're able to, and they'll continue to do so until they're not able to. I think we agree here actually. The name isn't the problem really the issue.

I couldn't agree more. Autopilots of any kind have always required supervision. Tesla doesn't call its car self driving and there are many warnings that it isn't. Ultimately, the system would be called Boris and people would still use it irresponsibly.

Re: Tesla confirms “Autopilot” crash in Montana

#162

Post by Retra » Wed, Jul 13, 2016, 6:25 AM UTC

studentrob wrote: post #155

Retra wrote: post #131

Earlier quoted context omitted.

>untrained consumers Licensed drivers. Drivers who have a license that indicates that they will take responsibility for the actions of any vehicle they control and understand how to operate any features which may put themselves or others at risk.

> Drivers who have a license that indicates that they will take responsibility The fact that customers are licensed does not mean the seller is free from any and all regulation. Gun owners are licensed, and guns are required to have safety switches. Cars have a long history of being regulated [1] Basically, anything that can contribute to deaths is going to monitored closely by consumer protection bureaus, and will p…

I didn't mean to imply otherwise. Tesla probably could do a much better job informing it's customers what they are getting into, because they certainly aren't in a position to figure it out themselves. And they definitely can make their software better.

What I'm saying is, if your car injures someone while you sit in the driver seat, and you could have easily taken an easily foreseeable action to prevent it, then you are at least partially responsible.

Re: Tesla confirms “Autopilot” crash in Montana

#163

Post by NamTaf » Wed, Jul 13, 2016, 6:25 AM UTC

This is going to keep happening until Tesla change their approach from a move-fast-and-break-things startup philosophy to sound engineering one that puts safety in design front-and-centre.

Traditional real-world enigneers have been drilled in this for longer than living memory. When the software guys decide to bring their software appraoches to the real world, this is what happens.

From 12 days ago: https://news.ycombinator.com/item?id=12013713

Re: Tesla confirms “Autopilot” crash in Montana

#164

Post by laksjd » Wed, Jul 13, 2016, 6:26 AM UTC

wyldfire wrote: post #140

cperciva wrote: post #110

Earlier quoted context omitted.

The Pennsylvania Turnpike crash is disturbing, because it's the case the "autopilot" is supposed to handle - divided limited-access highway under good conditions. The vehicle hit a guard rail on the side of the road. That may have been an system failure. Too soon to tell. This is the one I'm very interested to hear more about. From what I've heard, Tesla's self-driving is very aggressive in swerving to avoid obstacle…

> or in fact a legitimate reason to swerve? Reasoning about what's legitimate can be a really fine line. Oncoming car, raccoon, fawn, human, doe, coyote, elk: some of these obstacles might actually be net-safer to not swerve around [perhaps depending on speed]. I'd be really curious if any of the autopilots have a value system that tries to minimize damage/loss of life. And how does it value the exterior obstacle aga…

Such a value system would be a nightmarish thing so I hope that it doesn't exist yet.

Just imagine your car calculating that it should let you die in order to save a group of school children... except that the school children were in fact a bunch of sheep that the system mis-identified.

The danger of AI isn't in paperclip optimizers (at least for now), it's in humans over-relying on machine learning systems.

Re: Tesla confirms “Autopilot” crash in Montana

#165

Post by venomsnake » Wed, Jul 13, 2016, 6:27 AM UTC

snuxoll wrote: post #6

I really think Tesla needs to make autopilot's "hands-on" requirement less lenient, the driver had their hands off the wheel for a full two minutes and the car didn't bark at him? I think (maybe I'm wrong and the other non-fatal one did) this is the first incident where autopilot was involved with an inattentive driver, and probably the one that is going to put the screws to Tesla, unfortunately.

The problem is that you not only need a "hands on", but a "brains on" on the wheel. Which will turn autopilot driving in more unpleasant experience than regular driving.

Re: Tesla confirms “Autopilot” crash in Montana

#166

Post by Dylan16807 » Wed, Jul 13, 2016, 6:28 AM UTC

base698 wrote: post #124

alogray wrote: post #102

Earlier quoted context omitted.

Has this feature decreased safety? Tesla's first fatality came after 130 million autopilot miles whereas traditions in the US, we have one fatality every 90 million miles. If that continues to hold, this seems to be significantly safer with it's own set of caveats. Not altogether different than getting stuck in a burning car due to your seatbelt.

I actually worked out the math on this--Tesla is lying with statistics when they made this comment they did about 1 in 130 million miles. Here's what I did: 1 in 90 million rate: 1.1 x 10^8 Probability of No accident in one mile: 1 - 1.1 x 108 = ~9.9999999 x 10^8 Probability of No accident and driving 130 million miles: (9.99999999 x 10 ^ 8) ^ 130e6 = 0.24 Probability AT LEAST ONE ACCIDENT in 130 million miles = 1 -…

In other words, the exact rate is very uncertain when there's only one crash data point. They might be lucky (or unlucky). Well that's obvious, and I wouldn't call it lying at all.

Re: Tesla confirms “Autopilot” crash in Montana

#167

Post by studentrob » Wed, Jul 13, 2016, 6:29 AM UTC

hop wrote: post #141

I think lawsuits and government intervention will kill self driving cars for the next decade or two. Even if autonoumous are 10x better, thats still hundreds of deaths per year that will be attributed to a half dozen autonomous car maker's software.

Tesla's not the only one at this. Google still has a good program that nobody seems upset about.

Re: Tesla confirms “Autopilot” crash in Montana

#168

Post by dayaz36 » Wed, Jul 13, 2016, 6:29 AM UTC

meddlepal wrote: post #111

Tesla needs (or is going to be forced) to issue a recall soon. Putting "beta" technology in a car isn't OK when it puts drivers and others on the road in danger. This is especially problematic because people have a preconceived notion of what "auto pilot" means and just because Tesla says it's not ready for prime time so therefore you should "keep your hands on the wheel" doesn't make it OK to put into production.

Tesla doesn't do recalls. They have OTA software updates

Re: Tesla confirms “Autopilot” crash in Montana

#169

Post by kbenson » Wed, Jul 13, 2016, 6:31 AM UTC

greendestiny wrote: post #103

cpncrunch wrote: post #92

Earlier quoted context omitted.

Actually, that's precisely how (real) autopilots work. If the autopilot can't reliably continue flying (e.g. in extreme turbulence, severe icing, or failure of some part of the system), it will immediately disengage with a loud warning sound, and the pilot has to take immediate control.

If planes had to fly within centimetres of hard barriers there would be no autopilot.

If planes had to fly within centimeters of hard barriers there would be no pilots.

Re: Tesla confirms “Autopilot” crash in Montana

#170

Post by [deleted] » Wed, Jul 13, 2016, 6:31 AM UTC

studentrob wrote: post #81

lucb1e wrote: post #66

What I don't understand is why Tesla doesn't whitelist roads where this feature can be used, since the manual sets very specific conditions. Especially given that the feature is currently enabled, they must know which roads work with high confidence values. Or assume they wanted to do the right thing from the start, and they don't have this dataset. It's not as if we don't have OpenStreetMap (or as if they couldn't b…

> What I don't understand is why Tesla doesn't whitelist roads where this feature can be used, since the manual sets very specific conditions I think they want the data of drivers attempting to drive with autopilot on these roads. Tesla does, I've read, restrict how fast autopilot can drive on unapproved roads, so it appears they already have them mapped out. They're using customers as guinea pigs for their tech.

[deleted]