Normalized crash data shows Autopilot is much less safe than Tesla claims • Hacker News Board

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#141

Post by buran77 » Wed, Feb 02, 2022, 8:57 PM UTC

spywaregorilla wrote: post #97

mrtksn wrote: post #82

Earlier quoted context omitted.

Humans don’t disengage and let the machines handle the situation but machines do it all the time. Do the machines disengage at tricky situations or straight lines?

The statistics are there to read however, and suggest your reasoning is not correct. Case A: Non Tesla Human drives 999 easy miles and 1 hard mile Case B: Autopilot drives 999 easy miles, and human drives 1 hard mile If the effect size of the hard mile is so large that its skewing the statistics, you would expect the Telsa human driver to have a horrendous per mile accident rate relative to the non tesla human driver…

Student drivers (with an instructor in the car, and secondary controls) have very few accidents, if any, not because of their safety record but because every mistake is saved by the instructor. A car can be considered as driving "safer than a human" when it can match the average human in any conditions the average human drives. Everything else is just squinting at the data and choosing an interpretation that fits your personal opinion.

A Tesla can relatively safely cover traffic like divided or controlled access highways. That's a very narrow slice of all possible driving situations and not one responsible for most accidents.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#142

Post by bagels » Wed, Feb 02, 2022, 8:57 PM UTC

M2Ys4U wrote: post #74

endisneigh wrote: post #10

Are there stats that even show that any self driving system is safer than the median driver? Basically accidents involve two people. Said accidents have responsibility divided between both individuals. A hypothetically perfect driver could still be involved in fatal accidents by virtue of the other driver. The worse drivers obviously will cause these accidents. The questions are three fold: 1. Are self driving vehicl…

>Basically accidents involve two people. Not always, no. It's perfectly possible to crash a car with nobody else around.

Or for one car to crash in to many cars.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#143

Post by gnicholas » Wed, Feb 02, 2022, 8:58 PM UTC

tacLog wrote: post #36

gnicholas wrote: post #15

Earlier quoted context omitted.

> the fact that autopilot isn't worse is pretty promising! Presumably drivers take over when it makes mistakes, which tilts the stats a bit. > When I was 19 probably safer to use autopilot. As a parent, I wonder about what safety tech I would let my kids use when learning to drive. I want to make sure they fully learn how to drive, which makes me think I shouldn't let them use too much semi-autonomous tech. At the sa…

> At the same time, it seems foolish to tell them not to use blind spot monitors, or to even expect them to "check their blind spot" the old-fashioned way if their cars have monitors. I have never owned a car with blind spot monitoring. Do people really just rely on them completely? Even in dense urban environments with bikes around? Or is it more of a judgement call? Different perspectives on this would be nice. I h…

I've never owned a car with BSM but have rented for extended periods of time. I would expect that after a year or two of using the system while also actively checking my blind spot the conventional way, I would become comfortable with understanding its reliability and limitations (if any) and would adjust my behavior.

Some Hondas pipe through a camera feed from the side of the vehicle when you turn on your blinker, which wouldn't have the same potential weakness for cyclists that you mention with the binary sensor-based systems.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#144

Post by spywaregorilla » Wed, Feb 02, 2022, 9:00 PM UTC

mrtksn wrote: post #118

spywaregorilla wrote: post #97

Earlier quoted context omitted.

The statistics are there to read however, and suggest your reasoning is not correct. Case A: Non Tesla Human drives 999 easy miles and 1 hard mile Case B: Autopilot drives 999 easy miles, and human drives 1 hard mile If the effect size of the hard mile is so large that its skewing the statistics, you would expect the Telsa human driver to have a horrendous per mile accident rate relative to the non tesla human driver…

stats for easy miles and hard miles? link please? Humans don’t crash at every tricky situation but Tesla claims that humans are horrible drivers and their cars gives the control to the humans when when something happens.

Literally any distribution of hard miles and easy miles will produce the same outcome in this thought experiment, to varying effects, if your premise of Tesla hiding the AI's incompetence with passing off to humans when driving is challenging is accurate.

If your premise is to be believed as a significant effect, you must also accept that this outcome should be visible in the data

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#145

Post by ActorNightly » Wed, Feb 02, 2022, 9:01 PM UTC

> Crash rates can be adjusted to account for differences in environment and demographics in different data sets. A sample dataset with acrash rateris exposed to some variableiat a different proportion p than the comparison dataset. In the case study, for example, vehicles running Autopilot were driven on freeways (i) 93% of the time, resulting in pi= 0.93. In the SHRP 2 NDS, only 28% ofvehicle mileage was recorded on freeways, i.e pi= 0.28. In the SHRP 2 NDS data, vehicles on non-freeways crashed 2.01 times more often per mile than vehicles on freeways. The observed Autopilot crash rate can be adjusted to reflect national driving ratesto reflect the crash rate that might be observed if 28% of Autopilot mileage was on freeways and 72% were on non-freeways, assuming that the 2.01 ratio holds for Autopilot.

So if humans are 2 times more likely to crash on non-highways than highways, the argument presented here is that the added danger of non-freeways affects autopilot in the same way, degrading its performance. Seems like a far reaching assumption.

If there is no data on autopilot performance on rural roads, then you cannot make a claim either way.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#146

Post by smnrchrds » Wed, Feb 02, 2022, 9:01 PM UTC

gnicholas wrote: post #23

yrral wrote: post #12

How about the predominately male ownership of teslas causing higher accident rates. I see the paper adjusts for age and road type but missing gender is a pretty big slip-up? I didn't read the paper fully but grepped for "gender" and "sex" and couldn't find anything.

Good question! According to one study in LA, [1] men cause 60% of accidents. I do wonder, however, the extent to which this gap fades with age. Given that most Tesla owners are not super young, it might not make as much of a difference as the 60% number (assuming it is nationally representative) would indicate. 1: https://xtown.la/2019/12/12/women-drivers-maybe-we-need-more...

Keep in mind that men drive 16,500 miles per year on average, while women drive 10,100 miles. There is a noticeable gender gap in "accidents per year", but not in "accidents per mile". The paper compares accidents per mile, so there should be little gender effect in the results.

https://www.fhwa.dot.gov/ohim/onh00/bar8.htm

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#147

Post by paxys » Wed, Feb 02, 2022, 9:02 PM UTC

jjulius wrote: post #94

paxys wrote: post #22

I have felt since day 1 that "autopilot is better than the average driver!" is the most misleading fact/statistic in the industry. The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. To be worth it, the car…

I don't disagree with your broader statement about autopilot's safety relative to other drivers, but... >The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. ... what ? Tesla owners absolutely can be habitua…

For your specific question, actuarial tables for car accidents/deaths are available publicly, and the profile for the typical Tesla owner (middle aged, high income, driving expensive sedan/SUV) is considered much safer than average.

Moreover it is true in general that most people driving aren't drunk or high, aren't reckless etc., and so most Tesla drivers aren't either. The number of accidents isn't uniformly distributed among the public.

More broadly though, I never said that zero Tesla owners (or any other specific car drivers) do any of these things, but that using autopilot with an above average safety record can still mean that the personal safety of an individual or group of individuals goes down. For the system to be a net benefit it has to be pushed to the below average drivers.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#148

Post by spywaregorilla » Wed, Feb 02, 2022, 9:03 PM UTC

buran77 wrote: post #141

spywaregorilla wrote: post #97

Earlier quoted context omitted.

The statistics are there to read however, and suggest your reasoning is not correct. Case A: Non Tesla Human drives 999 easy miles and 1 hard mile Case B: Autopilot drives 999 easy miles, and human drives 1 hard mile If the effect size of the hard mile is so large that its skewing the statistics, you would expect the Telsa human driver to have a horrendous per mile accident rate relative to the non tesla human driver…

Student drivers (with an instructor in the car, and secondary controls) have very few accidents, if any, not because of their safety record but because every mistake is saved by the instructor. A car can be considered as driving "safer than a human" when it can match the average human in any conditions the average human drives. Everything else is just squinting at the data and choosing an interpretation that fits you…

So as a thought experiment, if Autopilot + Human intervention reduced the rate of accidents by 50% vs. just humans after normalization; can we consider autopilot to be adding value?

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#149

Post by jahewson » Wed, Feb 02, 2022, 9:03 PM UTC

jjulius wrote: post #94

paxys wrote: post #22

I have felt since day 1 that "autopilot is better than the average driver!" is the most misleading fact/statistic in the industry. The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. To be worth it, the car…

I don't disagree with your broader statement about autopilot's safety relative to other drivers, but... >The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. ... what ? Tesla owners absolutely can be habitua…

I read it that way at first too but I think the reasoning is actually correct. The population of Tesla drivers will of course resemble the overall population of drivers but most drivers are not sleepy/raging/drunks and by extension “most people buying Teslas aren't part of” that group either.

To use an analogy: the average person has 1.9 legs but a product which results in its user having 1.95 legs is not an improvement for most people.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#150

Post by vanattab » Wed, Feb 02, 2022, 9:04 PM UTC

TedDoesntTalk wrote: post #75

Natsu wrote: post #4

Earlier quoted context omitted.

The study says this: > An estimated 15% of injury crashes and 24% of property damage-only crashes are never reported to police (M. Davis and Company, Inc., 2015), while9% of injury crashes and 24% of property damage-only crashes are reported but not logged (Blincoe et al., 2015). Even with robust data, establishing the statistical significance of automated vehicle safety can be expensive. Kalra and Paddock (2016)demo…

Why would injury crashes go unreported?

I got hit a few weeks ago while walking my dog (dog was fine). I thought I had just bruised my tailbone so I waved the driver on after a short chat and never reported it. In hindsight I wish I had as it seems like my tailbone was fractured and not just bruised but it definitely happens.