Tesla crash in September showed similarities to fatal Mountain View accident • Hacker News Board

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#161

Post by codebolt » Thu, Apr 05, 2018, 7:29 AM UTC

sho_hn wrote: post #148

icc97 wrote: post #111

Earlier quoted context omitted.

Airline pilot training is designed to handle this, as well as they have a co-pilot who prevents them from being distracted and is able to take over if they are distracted. Tesla is just handing it out to anyone who can afford to buy a Model S/X (and now Model 3) with the absolute minimum of warnings that they can get away with.

I would not be surprised if Musk coldly responded that this is a known problem with no solution, but autopilot still lowers the death rate on average. In other words, for the proponents it's a trade-off. And also the idea that an automatic driver is easier to improve over time (skewing the trade-off better and better) than human drivers. Which of course will snowball when reliance on automation keeps making drivers w…

> And also the idea that an automatic driver is easier to improve over time (skewing the trade-off better and better) than human drivers. Which of course will snowball when reliance on automation keeps making drivers worse than better, and at some point you're past the point of no return.

I really hope it does get to the point were all drivers are automatic and interconnected. Cars could cooperate to a much higher extent and traffic could potentially be much more efficient and safe.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#162

Post by elgenie » Thu, Apr 05, 2018, 7:30 AM UTC

danso wrote: post #82

Currently, the Tesla autopilot page has this as its top headline: > Full Self-Driving Hardware on All Cars Followed by this: > All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver. http://web.archive.org/web/20180323054727/https://www.tesla.... Getting past the arguments of whether T…

Their claim is that all the sensing capability is there for the car equivalent of the proverbial sufficiently smart compiler to achieve full self-driving capability. But said sufficiently smart software running sufficiently smart algorithms capable of sufficiently speedy processing of the senser data doesn't exist at the moment.

The claim is falsifiable if and only if one can prove the minimum sensor capability required for full self-driving exceeds what they have. But even proving the falseness at a later date doesn't prove malice or fraud unless Tesla knows right now that their claim is not true.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#163

Post by Animats » Thu, Apr 05, 2018, 7:33 AM UTC

Come to think of it, whatever happened to Tesla's self-driving effort? They claimed to be shipping Model 3 cars with all the hardware needed for self driving (not including a LIDAR). They produced videos. Most of this was 1-2 years ago. But they never got to deployment, or even to the point of demoing it for car magazine writers.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#164

Post by Tempest1981 » Thu, Apr 05, 2018, 7:34 AM UTC

Animats wrote: post #158

mirimir wrote: post #42

Earlier quoted context omitted.

Indeed. His Tesla was clearly following the clearest left-side white line. Which unfortunately pointed the car right at the gore point. But what amazes me is that Tesla autopilot apparently won't swerve or stop when it sees an unmoving object straight ahead. Whether it's a semi, a fire truck, or a gore point. But I gather that trying to detect that stuff generates too many false positives. And that overall, it would…

But what amazes me is that Tesla autopilot apparently won't swerve or stop when it sees an unmoving object straight ahead. It won't even brake unless the thing in front looks like the back end of a car or truck. Even then, if it's stationary, it may still get hit. It won't swerve, ever; it has no logic for that. Four times that we know of, a Tesla has plowed into a vehicle partly blocking a lane. Putting something on…

Will it brake for pedestrians?

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#165

Post by icc97 » Thu, Apr 05, 2018, 7:36 AM UTC

gambiting wrote: post #157

sho_hn wrote: post #148

Earlier quoted context omitted.

I would not be surprised if Musk coldly responded that this is a known problem with no solution, but autopilot still lowers the death rate on average. In other words, for the proponents it's a trade-off. And also the idea that an automatic driver is easier to improve over time (skewing the trade-off better and better) than human drivers. Which of course will snowball when reliance on automation keeps making drivers w…

I don't think that's the correct answer. Flight autopilots have lowered accident rates tenfold, but every time there is a crash while on autopilot Boeing/Airbus will ground every single plane of that type and won't allow them to fly until the problem is found and fixed. If I know that my car's autopilot is statistically less likely to kill me than I am to kill myself, I would still much much rather drive myself.

If Tesla were to respond in a similar vein - that is turn off auto-pilot each time there was a fatal crash until the cause was fully investigated and fixed then I'd feel a lot more comfortable.

From the videos in the article it's clear that auto-pilot should be disabled when there is bright low-light sunshine. Tesla should be prepared to tell customers there's certain times of the day when the reliability of the software is not high enough and turn it off.

These are all 'beta' testers after all so they shouldn't complain too much.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#166

Post by lozenge » Thu, Apr 05, 2018, 7:36 AM UTC

Animats wrote: post #163

Come to think of it, whatever happened to Tesla's self-driving effort? They claimed to be shipping Model 3 cars with all the hardware needed for self driving (not including a LIDAR). They produced videos. Most of this was 1-2 years ago. But they never got to deployment, or even to the point of demoing it for car magazine writers.

They had a falling out with their software supplier Mobileye which set them back a year or more.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#167

Post by carlmr » Thu, Apr 05, 2018, 7:39 AM UTC

Shank wrote: post #89

danso wrote: post #82

Currently, the Tesla autopilot page has this as its top headline: > Full Self-Driving Hardware on All Cars Followed by this: > All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver. http://web.archive.org/web/20180323054727/https://www.tesla.... Getting past the arguments of whether T…

> Getting past the arguments of whether Tesla should be calling this "Autopilot", is its claims about hardware true? Tesla actually hasn't called that Autopilot -- they refer to it as "full self driving" and not in the same category. But...is it true? It's vaporware right now, so we can't really say. Obviously, in a purely hypothetical landscape, even with cameras you should be able to emulate what humans can do and…

>Obviously, in a purely hypothetical landscape, even with cameras you should be able to emulate what humans can do and have 100% attentiveness. That is to say, humans don't have lidar and radar, yet humans do drive.

Which camera has the same dynamic range as the human eye?

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#168

Post by buvanshak » Thu, Apr 05, 2018, 7:50 AM UTC

sho_hn wrote: post #148

icc97 wrote: post #111

Earlier quoted context omitted.

Airline pilot training is designed to handle this, as well as they have a co-pilot who prevents them from being distracted and is able to take over if they are distracted. Tesla is just handing it out to anyone who can afford to buy a Model S/X (and now Model 3) with the absolute minimum of warnings that they can get away with.

I would not be surprised if Musk coldly responded that this is a known problem with no solution, but autopilot still lowers the death rate on average. In other words, for the proponents it's a trade-off. And also the idea that an automatic driver is easier to improve over time (skewing the trade-off better and better) than human drivers. Which of course will snowball when reliance on automation keeps making drivers w…

> but autopilot still lowers the death rate on average.

I think the question of using self driving vehicles comes down to this. Do you want to be part of a statistics that you can control, or do you want to be part of one that you cannot?

When we go in trains or planes, we are already being part of statistics that we cannot control. But those things also are extremely reliable.

So there seems to be a threshold, where someone should opt for being part of a statistics that you cannot control.

The people who are pushing SDV's, by some sleight of hand, seem to hide this aspect, and have successfully showcased raw, (projected) statistics that implicitly assumes the rate of progress with SDV's, also assume that SDV ll continue to progress until they reach that capability.....

>And also the idea that an automatic driver is easier to improve over time (skewing the trade-off better and better) than human drivers.

But there is always are risk of catastrophic regressions [1] with each update, right?

[1] https://en.wikipedia.org/wiki/Catastrophic_interference

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#169

Post by alkonaut » Thu, Apr 05, 2018, 7:55 AM UTC

njarboe wrote: post #53

ChuckMcM wrote: post #17

I have now talked with two people who have autopilot in their model S's and both said the problem with autopilot is that it is "too good". Specifically it works completely correctly like 80 - 90% of the time and as a result, if you use it a lot, you start to expect it to be working and not fail. If it beeps then to tell you to take over, you have to mentally get back into the situational awareness of the road and the…

And in the current climate of "all new things must be super extra safe to be used, even if it displaces much worse technology", I think machine driving cars will have a hard time being accepted. Imagine the autopilot is 100 times safer than human drivers. Full implementation would mean about a fatality a day in the US. It does not seem this would be acceptable by the public even though 30,000 annual human car fatalit…

> And in the current climate of "all new things must be super extra safe to be used, even if it displaces much worse technology", I think machine driving cars will have a hard time being accepted.

Exactly this. People will not accept being hurt in accidents involving AI, but will accept being in accidents caused by human error. There is no way this will change, and car manufacturers should also have realized this by now.

As far as I can see, the only viable solution to the "not quite an autopilot" problem that tesla (and maybe others) has created for themselves is this: Just make the car sound an alarm and slow down as soon as the driver takes his eyes off the road or his hands off the wheel. The first car that doesn't do this should be one that is so good it doesn't have a driver fallback requirement at all - which I think is two decades out.

Re: Tesla crash in September showed similarities to fatal Mountain View accident

#170

Post by omarforgotpwd » Thu, Apr 05, 2018, 7:57 AM UTC

danso wrote: post #97

omarforgotpwd wrote: post #80

Earlier quoted context omitted.

Please. You should see me when the car isn't on AutoPilot. I speed excessively, weave through LA traffic switching lanes frequently and generally exhibit unsafe driving behavior. I can't help it, it's like I get bored or something. I'm a decent driver and have never gotten in a crash but have gotten a lot of tickets. When I use AutoPilot, the car automatically maintains a reasonable distance at a fixed speed. You nee…

The 3 Autopilot deaths so far involved 3 relatively young men, both in relatively elite professions: former Navy SEAL and an Apple engineer -- the third was a son of a Chinese business owner [0]. They fit the profile of men fairly confident about driving and tech, perhaps too confident. 3 fatalities over 320 million miles for Teslas equipped with Autopilot hardware is not much better than the 1.16/100 million miles f…

I know, it definitely gives me pause to see people dying and is definitely a reminder to be safe. I remember when I was using AutoPilot on the 101 in the bay area this weekend, I noticed I was in the far left lane and decided to switch to a middle lane. So it's not that I don't think AutoPilot could kill me, as a matter of fact there have been several times when AutoPilot was definitely about to kill me but I took control in time to course correct. I think I've used the system enough to get a sense of what it can and can't do and feel that I'm less likely to get in a collision due to not paying attention while AutoPilot is on than I am to get into a collision because I was driving.

Also, in response to the statistics you cite: I wouldn't expect the statistics to be that far off the average because AutoPilot is limited in it's possible use cases at the moment. Even in cars equipped with AutoPilot people are always still driving the car for at least some part of the trip. Therefore I wouldn't expect the impact of AutoPilot to lead to a significant deviation from the average. Plus, I'd speculate that the crash rate per mile is probably higher on a Tesla then an average car -- I'm thinking of like a Honda Civic. Faster cars probably get in more crashes, right? Maybe not, who knows. Regardless it should be possible to control for this and assess the effect of AutoPilot on per mile crash rates by comparing rates for Teslas with and without AutoPilot. This is somewhat complicated by the fact that even Teslas without AutoPilot have automatic collision avoidance, but should shed some light on whether AutoPilot is making people crash more or less.