Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Normalized crash data shows Autopilot is much less safe than Tesla claims

twitter.com

Post reply on HN
Print view View original

131–140 of 462 posts

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#131

Post by mdoms »

andrepd wrote: post #51
gnicholas wrote: post #2

Underlying study is here. [1] Full disclosure, I co-wrote the piece for The Daily Beast [2] that originally suggested that Tesla's methodology was seriously flawed. I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims. 1: https://engrxiv.org/preprint/view/1973/3986 2: https://www.thedailybeast.com/how-tesla-and-elon-musk-exagge...

>I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims. If you're still surprised by this you've not been paying attention to how Telsa operates.

He never said he was surprised.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#132

Post by w-j-w »

jjulius wrote: post #94
paxys wrote: post #22

I have felt since day 1 that "autopilot is better than the average driver!" is the most misleading fact/statistic in the industry. The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. To be worth it, the car…

I don't disagree with your broader statement about autopilot's safety relative to other drivers, but... >The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. ... what ? Tesla owners absolutely can be habitua…

No post body was provided.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#133

Post by spywaregorilla »

jcranberry wrote: post #96
spywaregorilla wrote: post #76

Earlier quoted context omitted.

If it were the case that autopilot was benefiting from dumping all of the difficult parts on the humans, you would expect to see much higher than average rates of accidents per mile on the human driven parts, because they're only getting the hard parts. This tends not to be true, so I'm doubting it's a real effect.

Could you provide a source? That sounds fairly interesting.

Just some reasoning really. Statistics and proper normalization are hard.

Tesla tends to say that autopilot crashes occur 1/2 as often as non autopilot crashes. That's likely not normalized to road conditions. But if you assume that Tesla is secretly just putting all the hard miles on the humans, then that would imply humans are driving many more hard miles and should have higher accident rates. The autopilots meanwhile must be performing worse on the easy miles and racking up additional accidents that wouldn't have otherwise happened.

If you combine those two, the overall rate of accidents should be higher than average, but it's actually lower by a fair margin. Again, normalization is hard.

Ideally you would be able to compare human drivers of another comparable car brand to the human drivers of Tesla to confirm the Tesla drivers don't seem to be being judged on unreasonably difficult conditions.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#134

Post by JoeAltmaier »

spywaregorilla wrote: post #80
JoeAltmaier wrote: post #17

Is it eggregiously worse than normal human pilots? If not, then that's an accomplishment in itself.

It's marginally better

Right! A few years ago, claiming that a robot car would drive as safely as a human driver would have been an incredible claim. Just saying.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#135

Post by gameswithgo »

justapassenger wrote: post #25
gameswithgo wrote: post #6

EDIT: After reading the full paper and not just the twitter graphs, some of what I said is wrong. That is, they also controlled for age, and when controlling for age and road type, autpilot was just worse. On the one hand I had the same suspicion about the unadjusted data, on the other hand the fact that autopilot isn't worse is pretty promising! However, average human rates include people driving drunk, tired, elder…

> So at 43 when I am driving sober, I'm probably safer not using autopilot. When I was 19 probably safer to use autopilot. That’s very dangerous thinking. Autopilot isn’t self driving and will actively try to kill you, from time to time, like all driver assist systems. 19 year old are much more likely to abuse autopilot and drive distracted/drunk. Only reason those systems aren’t crashing left and right is because dr…

It is only dangerous thinking with the most uncharitable interpretation of what I wrote.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#136

Post by sudosysgen »

jjulius wrote: post #94
paxys wrote: post #22

I have felt since day 1 that "autopilot is better than the average driver!" is the most misleading fact/statistic in the industry. The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. To be worth it, the car…

I don't disagree with your broader statement about autopilot's safety relative to other drivers, but... >The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. ... what ? Tesla owners absolutely can be habitua…

Because they are buying an expensive, new car, meaning higher income, meaning that those things are less likely : https://academic.oup.com/alcalc/article/46/6/721/129644

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#137

Post by gameswithgo »

lbrito wrote: post #41
gameswithgo wrote: post #6

EDIT: After reading the full paper and not just the twitter graphs, some of what I said is wrong. That is, they also controlled for age, and when controlling for age and road type, autpilot was just worse. On the one hand I had the same suspicion about the unadjusted data, on the other hand the fact that autopilot isn't worse is pretty promising! However, average human rates include people driving drunk, tired, elder…

>So at 43 when I am driving sober, I'm probably safer not using autopilot. When I was 19 probably safer to use autopilot. That might work out for you, but if teens think like that _now_ and avoid years of driving experience, they can't expect to be any better at driving at 43 than they are as teenagers, which kind of kills the argument.

The skills are pretty easy to acquire, it is the maturity that is hard.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#138

Post by contravariant »

gnicholas wrote: post #2

Underlying study is here. [1] Full disclosure, I co-wrote the piece for The Daily Beast [2] that originally suggested that Tesla's methodology was seriously flawed. I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims. 1: https://engrxiv.org/preprint/view/1973/3986 2: https://www.thedailybeast.com/how-tesla-and-elon-musk-exagge...

So to summarize:

- Freeways account for 93% of distance travelled with Autopilot

- Freeways account for 28% of the distance travelled by regular drivers.

- "vehicles on non-freeways crashed 2.01 times more often per mile"

a quick back of the envelop calculation suggests merely driving on the freeway lowers accidents per mile by ~40%, which accounts for most of the difference between Autopilot and regular drivers.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#139

Post by hnburnsy »

From the paper:

(1) Statistics obtained from Hardmanet al.(2019) (2) Statistics obtained from Blanco et al.(2016) (3) Statistics obtained from Transportation Research Board of the National Academy of Sciences (2013)

I don't understand how such chronologically disparate sets of data could produce reliable statistics. The Telsa demographic data (1) was from a "2018 demographic survey of 424 Tesla owners"

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#140

Post by spywaregorilla »

dboreham wrote: post #120
spywaregorilla wrote: post #61

I find this headline to be pretty misleading. * It doesn't find anything to suggest Autopilot is less safe than claimed. It finds reasons to believe Autopilot's improvement on safety relative to humans is less than claimed * Under both the road adjusted model; and the road+age adjusted model, Autopilot outperforms the average driver on average. Not by much, but its there. * Everyone here seems to be assuming that the…

Perhaps the downvotes are because most people would be apprehensive about putting their life in the hands of an "average" safe driver that's built from software? I probably wouldn't allow any other average safe human to drive me, if the alternative of me driving myself is available.

You never take ubers on the grounds of safety? Because that is, on average, an average human.

Either way, I'm not too impressed. I wouldn't purchase Autopilot personally for similar safety reasons. But let's not twist the stats to suggest they're actually much worse than they are.

Post reply on HN