Tesla’s Autopilot found partly to blame for 2018 crash on the 405 • Hacker News Board

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#81

Post by Animats » Thu, Sep 05, 2019, 3:28 AM UTC

matt-attack wrote: post #37

Animats wrote: post #11

Figure 7 depicts the movement of the Tesla and the two lead vehicles in the last 15 seconds before the crash. In the left panel—covering 15 to 8 seconds before impact—the Tesla is accelerating from 9 to 18 mph while following a lead vehicle (red car) at 30 to 46 feet.In the middle panel—covering 7 to 4 seconds before impact—the Tesla is traveling at a constant 21 mph while following a lead vehicle (green car) at a di…

It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding . What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost. I really think folks don’t realize how c…

While "human understanding" would be nice, detecting big solid obstacles is a solved problem. It just costs money. Until somebody orders enough LIDAR units to get the cost down.

Tesla's "autopilot" collisions are all very similar. They're all "hit big stationary obstacle that wasn't the rear end of a car". So far, a fire truck, a street sweeper, a stalled car at a freeway median, a crossing semitrailer, and two freeway barriers. This is a well-defined defect.

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#82

Post by dahfizz » Thu, Sep 05, 2019, 3:29 AM UTC

mixmastamyk wrote: post #61

matt-attack wrote: post #37

Earlier quoted context omitted.

It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding . What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost. I really think folks don’t realize how c…

Sounds like dumb policy to me. Why wouldn’t it stop for a boulder or fallen tree?

How often are bolders in the road?

As software engineers, we can understand the hidden costs in complexity and bugs when implementing features that don't need to be implemented. I could imagine that false positives for bolder avoidance causing more harm than false negatives.

Obviously it should be smart enough to detect any and every obstruction, but I'd rather the 99% use case get ironed out first.

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#83

Post by smt88 » Thu, Sep 05, 2019, 3:29 AM UTC

schiffern wrote: post #49

smt88 wrote: post #18

Earlier quoted context omitted.

The null experiment is the human driver of that particular Tesla, not an average of all drivers

So if the crash didn't involve "the human driver of that particular Tesla" but instead was an identical incident with a different Tesla driver, we wouldn't be having this conversation? Because that seems highly improbable. This particular driver isn't what makes the story special or newsworthy. The nature of the incident is. Our choice of statistical cohort should reflect this data selection effect, ie "all Autopilot…

I disagree because we're comparing what this individual driver would have done without a feature advertised as Autopilot.

I'm a partner in an auto insurance company, and most of our no-accident records will stay that way, statistically speaking.

If this driver had a one in a billion miles chance of crashing and Autopilot (with an inattentve driver, as they all tend to be) had a one in a million miles chance, then Autopilot is decreasing safety.

At a certain level of safety, Autopilot would make everyone safer, but it's definitely not there yet.

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#84

Post by Gustomaximus » Thu, Sep 05, 2019, 3:34 AM UTC

jmpman wrote: post #80

Gustomaximus wrote: post #69

Earlier quoted context omitted.

We need to look at driverless a different way and I suspect this is why it's going to be hard to accept driverless for longer than we need to. Computers look at the world in a different way. So they are going to have accidents that seem very obvious to humans making people go "look how bad driverless is because a human would never...". Many people will take that as a reason to avoid driverless tech. Yet driverless wi…

And the lawyers will become very wealthy putting these companies into bankruptcy... as they should.

Why should they?

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#85

Post by elevenbits » Thu, Sep 05, 2019, 3:36 AM UTC

Gustomaximus wrote: post #69

matt-attack wrote: post #37

Earlier quoted context omitted.

It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding . What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost. I really think folks don’t realize how c…

We need to look at driverless a different way and I suspect this is why it's going to be hard to accept driverless for longer than we need to. Computers look at the world in a different way. So they are going to have accidents that seem very obvious to humans making people go "look how bad driverless is because a human would never...". Many people will take that as a reason to avoid driverless tech. Yet driverless wi…

I can't speak for others, but while this is true in aggregate, I'm not personally willing to stake my life on a system that can, on a 1-in-a-million event and with no warning, accelerate into a fatal collision. An otherwise 100% accident-free driving record matters not when I'm dead.

The failure modes of these systems need to be much more gentle: when I'm confronted with an uncertainty while driving, I slow down and pay more attention. I don't speed up until I'm confident that a higher speed is reasonable. This is far far more sophisticated than simply "can I see something in front of me" -- I'm evaluating the probability that I need to stop based on the events I can see happening. If I see a car in front of me slow down and swerve out of the way of something, that means something much more than "obstacle detected" -> "obstacle removed".

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#86

Post by seanmcdirmid » Thu, Sep 05, 2019, 3:37 AM UTC

dreamcompiler wrote: post #9

Fire truck driver here: People crash into parked fire trucks and ambulances and cop cars all the damn time . That's why we park the Big Red Truck at an angle behind the accident -- so that when the 2-ton car crashes into the 30-ton truck, the car will bounce off and the people behind the truck (us) won't be injured. I don't disagree that Tesla's software is partly to blame here, but the null experiment also has a lou…

On Beijing’s third ring road one time, I saw a lady being stopped by a cop for some reason (rare, cops never pull over anybody there), she got into her car and promptly rammed into the police car directly in front of her (on accident I assume). I think she was having a bad day or something.

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#87

Post by macintux » Thu, Sep 05, 2019, 3:40 AM UTC

rictic wrote: post #50

macintux wrote: post #15

Earlier quoted context omitted.

Statistics makes for a tough sell to the survivors. “Yes, this autonomous vehicle killed your son who just happened to be in the wrong place at the wrong time, but someone else’s son would have died in an unrelated accident somewhere else were it not for autonomy.”

Ambulances save some number of lives. Ambulances are involved in some number of fatal accidents. Should ambulances be banned? Subsidized? I can imagine different ways of analyzing the question, but asking "what would you say to the families of those who die" is a nonstarter. There's nothing you can say that will be satisfying, or ease their pain. Statistics may be cold, but they're the only tool that actually lets us…

On the other hand, people know who’s been saved by an ambulance. We’ll never know who was saved by the existence autonomous vehicles.

I’m just pointing out there’s a PR problem that may prove tough to manage.

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#88

Post by seanmcdirmid » Thu, Sep 05, 2019, 3:41 AM UTC

gbrown wrote: post #71

bg24 wrote: post #65

Earlier quoted context omitted.

I was going at 30-40mph in the right lane on a 4-lane all empty highway, when a gentleman flies at 60-70mph and hits us right behind, goes 360 turn with us and we hit the wall. Everyone survived without any medical incident (other than that I do panic and respectfully avoid those fliers now-a-days). Relevant because: - I hope my sprinter (size of mini truck, with upfittings) had an autopilot in the front/rear to spee…

The other driver was clearly 100% totally at fault, but I'm just curious: why were you going 30-40 on a four lane highway? Where I live that's at least 25 below the speed limit. Here it's mostly tractors and other farm equipment driving at those speeds.

Maybe going up a hill? They did mention they were driving a truck, it’s easy to have vastly different climbing speeds in that case.

Also, I would often go less than 20 down a pass during the winter because I didn’t have snow tires or chains, and was passed by plenty of locals doing 60+ because they were equipped for it.

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#89

Post by akira2501 » Thu, Sep 05, 2019, 3:41 AM UTC

NeedMoreTea wrote: post #55

matt-attack wrote: post #37

Earlier quoted context omitted.

It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding . What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost. I really think folks don’t realize how c…

Except humans crash into stationary vehicles in lane all the damn time. Most highway pile ups are a simple crash lots of folks drive into the back of. Why's it hard? A vehicle coming from a side road has a lot of movement within your field of vision. When driving on a highway we're well used to the fact and expectation that stuff isn't stopped. When the vehicle ahead emergency brakes or stops you have the growth of s…

> Why's it hard?

That humans get distracted and often make mistakes is no surprise, it should also be obvious that the amount of risky behavior is correlated to the amount of _perceived_ risk in any given situation, which is probably why so many are "a simple crash."

It is, however, somewhat scary that a machine that has nothing else to do other than to drive the car as safely as possible still makes these "dumb" mistakes.

Re: Tesla’s Autopilot found partly to blame for 2018 crash on the 405

#90

Post by kevingadd » Thu, Sep 05, 2019, 3:43 AM UTC

toomuchtodo wrote: post #72

organsnyder wrote: post #42

Earlier quoted context omitted.

I've noticed the same thing driving my Chrysler Pacifica, which has lane keeping and adaptive cruise (with FCW). But it's not called "autopilot", and it deactivates if it detects that my hands are off the wheel.

Model S and X owner here. Autopilot will nag you after so many seconds (depending on vehicle speed and autopilot path planning confidence) if it doesn’t detect steering wheel torque, and if you ignore the nags, it brings the vehicle to a stop safely with the hazards on. Car and Driver tested AEB in several vehicles (Tesla, Toyota, Subaru) and they are all pretty equally bad at it. Figuring out if matter in front of y…

How do you reconcile this with the claims in the accident report that the hands-off alert intervals are measured in minutes for the accident scenario? Quoted here with link to full source:

https://twitter.com/vogon/status/1169358625220882432

Visual indicator after 2 minutes of no detected driver-applied torque to the steering wheel (apparently because they only detect torque - i.e. turning/steering motions - and not pressure?)