Normalized crash data shows Autopilot is much less safe than Tesla claims
1–10 of 462 posts
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#2Underlying study is here. [1] Full disclosure, I co-wrote the piece for The Daily Beast [2] that originally suggested that Tesla's methodology was seriously flawed. I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims.
1: https://engrxiv.org/preprint/view/1973/3986
2: https://www.thedailybeast.com/how-tesla-and-elon-musk-exagge...
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#3Wow, adjusted for driver age the difference vanishes.
That's quite an interesting observation, and it shows the importance of proper statistics. It also raises questions about companies producing their own stats, it's often only when someone takes the time to dig into it that we discover this kind of thing.
I am not holding my breath waiting for the common consumer to understand statistical critical thinking though. I sometimes catch myself forgetting the due diligence even after many years of forcing myself to think about stats.
Perhaps some sort of ombudsman could do this, pointing out when stats are lying to people.
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#4Underlying study is here. [1] Full disclosure, I co-wrote the piece for The Daily Beast [2] that originally suggested that Tesla's methodology was seriously flawed. I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims. 1: https://engrxiv.org/preprint/view/1973/3986 2: https://www.thedailybeast.com/how-tesla-and-elon-musk-exagge...
The study says this:
> An estimated 15% of injury crashes and 24% of property damage-only crashes are never reported to police (M. Davis and Company, Inc., 2015), while9% of injury crashes and 24% of property damage-only crashes are reported but not logged (Blincoe et al., 2015). Even with robust data, establishing the statistical significance of automated vehicle safety can be expensive. Kalra and Paddock (2016)demonstrated that establishing that an AV has a fatal crash rate equivalent to the national average with 95% confidence would require driving a fleet of 100 vehicles continuously for 12.5 years.
I presume there's an adjustment for this in the figures you compare to, but I am having trouble finding it, though that may be my own failure so I wanted to ask if you could help me find how that's adjusted for?
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#5Wow, adjusted for driver age the difference vanishes. That's quite an interesting observation, and it shows the importance of proper statistics. It also raises questions about companies producing their own stats, it's often only when someone takes the time to dig into it that we discover this kind of thing. I am not holding my breath waiting for the common consumer to understand statistical critical thinking though.…
If you have taken a course in statistics you'll know its always possible to make the numbers looks favorable to your cause. It would take several unrelated blind third parties to do this correctly and no public company will ever let that happen, especially Tesla.
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#6EDIT: After reading the full paper and not just the twitter graphs, some of what I said is wrong. That is, they also controlled for age, and when controlling for age and road type, autpilot was just worse.
On the one hand I had the same suspicion about the unadjusted data, on the other hand the fact that autopilot isn't worse is pretty promising!
However, average human rates include people driving drunk, tired, elders, teenagers etc.
So at 43 when I am driving sober, I'm probably safer not using autopilot. When I was 19 probably safer to use autopilot.
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#7Wow, adjusted for driver age the difference vanishes. That's quite an interesting observation, and it shows the importance of proper statistics. It also raises questions about companies producing their own stats, it's often only when someone takes the time to dig into it that we discover this kind of thing. I am not holding my breath waiting for the common consumer to understand statistical critical thinking though.…
You could easily have a Simpsons paradox, e.g., it's safer overall, but for any single age group its neutral or even less safe.
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#8Underlying study is here. [1] Full disclosure, I co-wrote the piece for The Daily Beast [2] that originally suggested that Tesla's methodology was seriously flawed. I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims. 1: https://engrxiv.org/preprint/view/1973/3986 2: https://www.thedailybeast.com/how-tesla-and-elon-musk-exagge...
> I just thought it was odd that the company was playing quite so fast and loose with their safety claims.
That's because it's marketing, not science.
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#9I'm not too surprised by this and bet a lot of HN folks suspected that this was the case, but I'm really glad to see this come out and hope it gets a lot of visibility.
Controlling for confounding variables is a complicated and often subtle process that many people don't have an intuitive grasp of, so Elon's BS claims have probably been very convincing for many.
Re: Normalized crash data shows Autopilot is much less safe than Tesla claims
#10Are there stats that even show that any self driving system is safer than the median driver?
Basically accidents involve two people. Said accidents have responsibility divided between both individuals.
A hypothetically perfect driver could still be involved in fatal accidents by virtue of the other driver.
The worse drivers obviously will cause these accidents.
The questions are three fold:
1. Are self driving vehicles safer than the median driver?
2. What’s the ratio between at fault and not at fault for any accidents the median driver gets in?
3. What’s the same ratio for self driving cars?
Normalizes for location, of course.