Live data from Hacker News

Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

techcrunch.com

61–70 of 495 posts

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#61
post #46

Earlier quoted context omitted.

Likely they will get a gigantic sea of trained deep learning weights that noone understands.

That really isn't going to play well with a judge or jury. I can see the poor programmer up on the stand not being able to say what it meant or how the car made its decisions.

That is, unfortunately, the problem with neural nets in general.

They are amazing logical constructs, but there is a fundamental opaqueness to them that absent sufficient neural network mass to convincingly simulate a human, we can't apply the same methods of formal verification and behavioral inference we could for other more specific machine implementations.

No one can explain just why certain weights work at certain situations and not others. They just do.

Whether someone in the justice system is comfortable effectively legislating from the bench by creating precedent holding companies liable for NN based behavior that there are no hard and fast ways for them to proof test against in the first place is another question however.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#62

Earlier quoted context omitted.

The docs say to keep your hands on the wheel and stay alert at all times. Never mind Tesla CEO on national television taking hands off the wheel and not watching the road.

I'm fully aware of their recommendation. But if Tesla is aware that certain parts of the road are more likely to cause incidents and they haven't communicated that. Well sounds to me like that would be criminally negligent.

Surely there's some sort of "autopilot does not work on all roads" disclaimer, right? That's why you're supposed to keep your hands on the wheel and pay attention.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#63

If the Tesla had a proximity sensor that slammed on the breaks in the last 2-3 meters, would he have survived? I ask because it's a common luxury car feature and would be easy for Tesla to implement. IIRC this crash and the semitruck crash had no evidence of the autopilot hitting the breaks according to the NTSB. Even if his velocity was only reduced by a fraction, the amount of power involved in the collision would…

> If the Tesla had a proximity sensor that slammed on the breaks in the last 2-3 meters, would he have survived? I ask because it's a common luxury car feature and would be easy for Tesla to implement.

My car (a Nissan) has this feature, and it comes with a warning that it works best with obstacles that are moving. (ie. other cars)

Although it does work with stationary objects, it warns that if you're moving over ~35mph, it may not detect stationary obstacles, as it has trouble differentiating between them and objects on the side of the road.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#64

If the Tesla had a proximity sensor that slammed on the breaks in the last 2-3 meters, would he have survived? I ask because it's a common luxury car feature and would be easy for Tesla to implement. IIRC this crash and the semitruck crash had no evidence of the autopilot hitting the breaks according to the NTSB. Even if his velocity was only reduced by a fraction, the amount of power involved in the collision would…

they do have that, but as you might imagine, the emergency brakes are only there to mitigate damage in case of a crash, not to prevent them

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#65

The problem with Tesla's "Autopilot" is that it's marketed heavily as just that: autopilot. It's good enough to work that way 99% of the time, too, which is a recipe for disaster: things that work 99% of the time but have the potential to fail catastrophically often get glossed over by humans because it's hard to keep paying attention to something that rarely fails.

I'm not a pilot, but I have had flying lessons. Airplane autopilots are extremely simple. They can basically fly you in a straight line, at a fixed altitude. Tesla's autopilot is actually way more advanced than those found in airplanes.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#66

If the Tesla had a proximity sensor that slammed on the breaks in the last 2-3 meters, would he have survived? I ask because it's a common luxury car feature and would be easy for Tesla to implement. IIRC this crash and the semitruck crash had no evidence of the autopilot hitting the breaks according to the NTSB. Even if his velocity was only reduced by a fraction, the amount of power involved in the collision would…

they do have that, but as you might imagine, the emergency brakes are only there to mitigate damage in case of a crash, not to prevent them

Yes but in this case it didn't trigger.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#67

The problem with Tesla's "Autopilot" is that it's marketed heavily as just that: autopilot. It's good enough to work that way 99% of the time, too, which is a recipe for disaster: things that work 99% of the time but have the potential to fail catastrophically often get glossed over by humans because it's hard to keep paying attention to something that rarely fails.

This is exactly why Waymo and Cruise and such believe autonomy Level 3 or so is utterly dangerous and have intentionally skipped it entirely.

Do you think Tesla's insistence that LIDAR is useless has a part in this case? To me it's telling that everyone in the ASD game _except_ Tesla is using LIDAR.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#68
post #25

While I'm not quite sure about Tesla's responsibility, I do think CA DOT has its part in this tragic accident. Had the attenuator been replaced right after the previous accident, it could have saved the driver's life. Usually I don't complain much about the gov, but just look at the construction mess they've created on 101, it's been like that for more than 4 years!

So don't blame the company whose claiming that their car can "autopilot" under all conditions. Instead blame the road. Interesting take.

Except the company never claimed that.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#69
post #11
post #2

I think it's unacceptable for automation to produce a worse result than a human in the same situation, with the same information. i.e. it's not acceptable for automation to fail danger, it must fail safe, including even if all it can do is give up (disconnects, warning tone, hands control over to the human driver). I think it's reasonable, in the narrow case where primary control is asserted and competency is claimed…

This sounds good, but is impractical. First, a human who is paying attention makes better decision than a human who is surprised by being given control during a crisis. Therefore if automation sees a crisis, it's standard has to be that it is worse than a surprised human, and not how it compares to an alert driver. Second, the analysis that concludes that we're now in a specific situation where a human would do bette…

This is called the Halting Problem. We don't waste time trying to make machines that do that because it's theoretically undecidable.

Furthermore, you're then throwing more neural network (that has to be trained improvably across the dataset which we have no proof is the full dataset required to fully enumerate the problem space), to successfully detect the problem areas in the data going into the driving NN for edge cases. Repeat ad absurdum or until most of your self-driving car's energy is consumed trying to solve the Silicon equivalent of Plato's Cave.

Or... You just teach people to drive the blessed car, which is what the absolute best a theoretical NN implementation would converge to anyway.

The only benefit to a theoretically perfect NN which simulates the average driver being the car doesn't run on a processing system that is adversely effected through the recreational and voluntary (by the processing hardware) application of ethanol to the processing matrix.

Well... Thermally speaking, it could end up causing serious problems if someone were dumb enough to build the capability to do that into the car for some daft reason, but we're talking existential failure of computing hardware rather than gradual degradation of functionality like you'd see with a brain, but still I think everyone gets the point and I can stop torturing this poor analogy.

Re: Tesla sued in wrongful death lawsuit that alleges Autopilot caused crash

#70
post #34

It seems like a (relatively) trivial addition to the Autopilot system would be to allow the drive to tell the car when it has made a mistake and if Autopilot(s) consistently make mistakes in the same area then Autopilot should give a specific warning, or force a disengagement, when it detects that it is approaching that area. I can't imagine that this isn't already a thing or that someone at Tesla hasn't come up with…

You are not missing anything. They have that and gave details in their last Autonomy Day event.
Post reply on HN