Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

techcrunch.com

Post reply on HN
Print view View original

91–100 of 495 posts

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#91

Post by SteveNuts »

LeoPanthera wrote: post #65
saagarjha wrote: post #19

The problem with Tesla's "Autopilot" is that it's marketed heavily as just that: autopilot. It's good enough to work that way 99% of the time, too, which is a recipe for disaster: things that work 99% of the time but have the potential to fail catastrophically often get glossed over by humans because it's hard to keep paying attention to something that rarely fails.

I'm not a pilot, but I have had flying lessons. Airplane autopilots are extremely simple. They can basically fly you in a straight line, at a fixed altitude. Tesla's autopilot is actually way more advanced than those found in airplanes.

I think you're forgetting (more than a few) important functions that an airliner autopilot can do. Off the top of my head there's auto-throttle, auto-trim, and of course, if the runway is equipped, land the plane.

Also, if the pilots input a heading change, the plane will execute a 1G turn using all control surfaces so it's almost imperceptible to the passengers.

I think you're oversimplifying airliner autopilot.. by a lot.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#92

Post by estebank »

rockinghigh wrote: post #35
Jonanin wrote: post #24

Earlier quoted context omitted.

That's fair enough, but the problem I have is the way this event is being portrayed in news and social media. The implication everywhere is that autopilot just unexpectedly swerved into a wall and there was nothing the driver could have done. It actually was expected, and any reasonable driver could have easily prevented it, because it had almost happened so many times in the past.

> any reasonable driver could have easily prevented it, Not really. The crash attenuator was missing because someone else crashed into it. That highway exit is ridiculous. As for faults, there is obviously fault on both sides. The driver should have been more alert and the "autonomous system" should have detected the obstacle and slowed down the vehicle before hitting the guard.

Both sides? Shouldn't there be some responsibility for the organization tasked with making sure roads have dividers that people don't crash against multiple times in a month?

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#93

Post by mc32 »

LeoPanthera wrote: post #89
rayiner wrote: post #87

I hear people say “self driving cars don’t have to be perfect, they just need to be safer than humans.” Here is a good example of why that might be harder to achieve than expected. Apparently the car had gotten confused at that exact location repeatedly before. That’s what self driving cars are going to do—if there is something “weird” it’s likely that every self driving car (at least, every one from the same manufac…

To play devil's advocate: Assuming self-driving cars are connected to each other, either directly or via the internet, if one car crashes, or even just makes a minor mistake, the knowledge of how to avoid that can be transmitted to every other car on the planet.

Just as good information can be propagated, so can bad. So this had better be a robust algorithm and not one which can either be induced into something or simply learn “wrong”.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#94

Post by rjdagost »

deckar01 wrote: post #72
gooseus wrote: post #34

It seems like a (relatively) trivial addition to the Autopilot system would be to allow the drive to tell the car when it has made a mistake and if Autopilot(s) consistently make mistakes in the same area then Autopilot should give a specific warning, or force a disengagement, when it detects that it is approaching that area. I can't imagine that this isn't already a thing or that someone at Tesla hasn't come up with…

Tesla engineers talked about this in great depth during the recent streaming event. Manual disengagement of Autopilot is the mechanism used to trigger an upload of the sensor data. It is automatically uploaded to Tesla servers and used to train the driving model.

A Twitter user who rooted his Tesla claims that autopilot disengagement reports are very small (https://twitter.com/greentheonly/status/1096322810694287361

Not the kind of information that you use to train computer vision algorithms. I found his claims to be an interesting read.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#95

Post by rayiner »

bagels wrote: post #86
rayiner wrote: post #83

Earlier quoted context omitted.

A commercial airliner autopilot (the kind people think of when you say autopilot) can perform evasive maneuvers and land the plane.

What are they evading?

Other airplanes.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#96

Post by areoform »

I would like to add over here that Tesla does indeed do sensor fusion in their cars. Their Autopilot combines radar and ultrasound with vision to decide where to drive. Commentators bringing up LIDAR are jumping the gun by assuming that this scenario isn't something that these sensors wouldn't have detected in either combination or individually to spot the anomaly. The problem over here is likely to be software due to a bug in their code than a simple lack of additional sensors (LIDAR). And this touches on deeper issues that could have profound ramifications for the autonomous driving industry and the broader industry in general.

At least, in my eyes, the big problem with the autonomous car industry isn't the sensor suites they are deploying (or not), but the over-reliance on neural networks. They are black boxes with failure modes that can't be adequately mapped. See: http://www.evolvingai.org/fooling

What if the neural net or the system used to detect obstacles didn't see it because the precise configuration of the data fooled it? And if that's the case then what's next? How do we decide when it is okay for safety-critical systems to be opaque? How do we deal with autonomous driving if the conclusion comes out to be a "no" for this case? How should broader society deal with a yes? And who decides all of this in the first place?

Possibilities like this scare me far more than the lack of LIDAR because replicating a bug like this would be next to impossible. We don't know what we don't know, and we can't explore and understand the system to suss out what we don't know.

Edit: Fleshed out the idea with more questions.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#97

Post by natch »

rayiner wrote: post #83
LeoPanthera wrote: post #65

Earlier quoted context omitted.

I'm not a pilot, but I have had flying lessons. Airplane autopilots are extremely simple. They can basically fly you in a straight line, at a fixed altitude. Tesla's autopilot is actually way more advanced than those found in airplanes.

A commercial airliner autopilot (the kind people think of when you say autopilot) can perform evasive maneuvers and land the plane.

With a recent software update, Tesla cars now perform evasive maneuvers when necessary even when autopilot is turned off.

There is a growing body of videos from dashcams (a feature enabled by another software update) on youtube showing Teslas avoiding collisions that would have been caused by other drivers being negligent.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#98

Post by wbl »

salawat wrote: post #61
protomyth wrote: post #51

Earlier quoted context omitted.

That really isn't going to play well with a judge or jury. I can see the poor programmer up on the stand not being able to say what it meant or how the car made its decisions.

That is, unfortunately, the problem with neural nets in general. They are amazing logical constructs, but there is a fundamental opaqueness to them that absent sufficient neural network mass to convincingly simulate a human, we can't apply the same methods of formal verification and behavioral inference we could for other more specific machine implementations. No one can explain just why certain weights work at certa…

Liability is about incentives. Strict liability moves the externalities of unsafe cars onto the manufacturer.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#99

Post by valine »

LeoPanthera wrote: post #89
rayiner wrote: post #87

I hear people say “self driving cars don’t have to be perfect, they just need to be safer than humans.” Here is a good example of why that might be harder to achieve than expected. Apparently the car had gotten confused at that exact location repeatedly before. That’s what self driving cars are going to do—if there is something “weird” it’s likely that every self driving car (at least, every one from the same manufac…

To play devil's advocate: Assuming self-driving cars are connected to each other, either directly or via the internet, if one car crashes, or even just makes a minor mistake, the knowledge of how to avoid that can be transmitted to every other car on the planet.

Seems unlikely that the knowledge of how to avoid making a mistake could be compiled without human analysis. The technology to do that autonomously doesn’t exist. It would almost require artificial general intelligence, and if we had that self driving would be trivial.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#100

Post by tntn »

CoolGuySteve wrote: post #73
barrkel wrote: post #21

I think it's deeply irresponsible to create a device which can drive in 90 or 95% of scenarios. It's just a way to fool humans into killing themselves.

If the system is even a few percent better than humans then it would save thousands of lives when widely deployed. Musk has made this argument many times. Basically don't let perfect be the enemy of better. But if the vendor is legally liable in each case then this software is untenable.

Humans can be 30+% better than "humans" by 1. not using a phone and 2. not being drunk or stoned.

A few percent better than "humans" is quite bad relative to a semi-responsible human driver.

Post reply on HN