Live data from Hacker News

Normalized crash data shows Autopilot is much less safe than Tesla claims

twitter.com

111–120 of 462 posts

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#111

How is it that Tesla (and other companies developing self-driving capabilities) are allowed to deploy their products in the real world? We make people take a test to get a drivers license -- why don't we have requirements for self-driving systems? We require that cars pass routine mechanical inspections -- why isn't the software controlling these cars regulated?

Because in the context of regulations, they don't claim it to be self-driving.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#112
post #29
post #18

Earlier quoted context omitted.

> It also raises questions about companies producing their own stats Companies saying "trust us, we have the data to prove it" while keeping that data secret should be ignored. Especially if their CEO has a long history of blatant dishonesty! I'm most annoyed that people gave any weight to Tesla's safety statistics claims in the first place.

> Companies saying "trust us, we have the data to prove it" while keeping that data secret should be ignored. While true, I do not think this is a great heuristic. How would the average person know whether the "data" is available? Most people don't even know what "data" really is. They have never heard of CSVs or SQL. We need a better signal for consumers to know what they can and can't trust.

A good journalist will include that context in a story. I know I read at least one article that included these safety claims while also noting that the data was private and unverifiable.

I think it's a good default to assume any self-beneficial corporate message is false, absent evidence to the contrary.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#113
post #94
post #22

I have felt since day 1 that "autopilot is better than the average driver!" is the most misleading fact/statistic in the industry. The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. To be worth it, the car…

I don't disagree with your broader statement about autopilot's safety relative to other drivers, but... >The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. ... what ? Tesla owners absolutely can be habitua…

Yeah, seems like a typo a GPT-3 would do.

The claim seems to be that Teslas do not drink before driving, not Tesla owners, right?

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#114
post #47
post #22

I have felt since day 1 that "autopilot is better than the average driver!" is the most misleading fact/statistic in the industry. The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. To be worth it, the car…

> several other risky groups that most people buying Teslas aren't part of. This is a fallacy. Ask any demographic you like if they're road ragers, drunk drivers, etc... and they'll deny it. Everyone is sure that rare and terrible outcomes will never happen to them because of their own behavior, including the people to whom it happens! It's probably true that middle aged premium car owners are safer on average, but y…

I just think a lot of Tesla drivers see themselves as elite drivers or something when we all know that's not true if you step back away from the situation.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#115
post #77
post #44

I was just watching this video of a Tesla trying and miserably failing to drive itself through the streets of Boston: https://twitter.com/TaylorOgan/status/1488555256162172928 A number of these near-crashes seem to be pretty odd errors (a couple of times it just decided to drive on the wrong side of the street?) -- albeit ones that one _could_ see drivers unfamiliar with the area making. If a human driver started dri…

To be fair Boston city roads are like horse carriage routes that were converted to streets and will probably one of the last places FSD optimizes for.

Good point, Boston is a great real-world fuzzer. If they spent their time making it work there they'd have less issues elsewhere

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#116

How is it that Tesla (and other companies developing self-driving capabilities) are allowed to deploy their products in the real world? We make people take a test to get a drivers license -- why don't we have requirements for self-driving systems? We require that cars pass routine mechanical inspections -- why isn't the software controlling these cars regulated?

Regulators are asleep at the wheel.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#117
post #36

Earlier quoted context omitted.

> the fact that autopilot isn't worse is pretty promising! Presumably drivers take over when it makes mistakes, which tilts the stats a bit. > When I was 19 probably safer to use autopilot. As a parent, I wonder about what safety tech I would let my kids use when learning to drive. I want to make sure they fully learn how to drive, which makes me think I shouldn't let them use too much semi-autonomous tech. At the sa…

> At the same time, it seems foolish to tell them not to use blind spot monitors, or to even expect them to "check their blind spot" the old-fashioned way if their cars have monitors. I have never owned a car with blind spot monitoring. Do people really just rely on them completely? Even in dense urban environments with bikes around? Or is it more of a judgement call? Different perspectives on this would be nice. I h…

It's practially a trope how in sci-fi movies with flying futuristic cars, it's a demonstration of extreme competence for a character to demand "manual controls" to pilot the vehicle...like every normal driver does today. I expect that will become more and more real-world. Even today, most drivers don't know how to drive a manual transmission vehicle, much less how to double-clutch one without synchros or how to stop on ice without ABS. Those are disappearing from public roads and from the capabilities of average drivers, I expect that driving without aids will follow in the same way.

As part of my job, I end up renting vehicles pretty frequently, they're often newer models than my daily and have blind spot monitoring, backup cameras, radar cruise, vision-based lanekeeping assist, etc. It's frightening how in the course of a week you can get used to having to put less effort into centering yourself in the lane, trusting that cruise control will just keep a comfortable distance from the vehicle ahead of you. Driving becomes a lot less stressful. When I get back home and climb into my old manual-everything beater, though, it's quite an adjustment.

Regarding blind spot checks, yes, the Audi I recently drove had alerts for that, the blinking yellow light in my peripheral vision when cars were passing me was a nice reinforcement, but I'm too conditioned to do head checks to skip those. Likewise, reverse cameras - I've driven many work vans, pickups with headache racks, Jeeps with scratched up plastic, pulled trailers and RVs, etc where the windshield-mounted rearview mirror is useless; lots of pros get used to backing up using the side mirrors only. However, I asked my sister in law (who is extremely competent at most things) to drive my truck for an errand and she asked for help backing it out of the driveway - her car has always had a backup camera, which is honestly lots easier and she was completely uncomfortable using the side mirrors.

It's not hard to imagine that someone who only drives with assistive tools would adapt to become dependent on them; I'd argue it's more unusual to expect that they wouldn't!

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#118
post #82

Earlier quoted context omitted.

Humans don’t disengage and let the machines handle the situation but machines do it all the time. Do the machines disengage at tricky situations or straight lines?

The statistics are there to read however, and suggest your reasoning is not correct. Case A: Non Tesla Human drives 999 easy miles and 1 hard mile Case B: Autopilot drives 999 easy miles, and human drives 1 hard mile If the effect size of the hard mile is so large that its skewing the statistics, you would expect the Telsa human driver to have a horrendous per mile accident rate relative to the non tesla human driver…

stats for easy miles and hard miles? link please?

Humans don’t crash at every tricky situation but Tesla claims that humans are horrible drivers and their cars gives the control to the humans when when something happens.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#119

Wow, adjusted for driver age the difference vanishes. That's quite an interesting observation, and it shows the importance of proper statistics. It also raises questions about companies producing their own stats, it's often only when someone takes the time to dig into it that we discover this kind of thing. I am not holding my breath waiting for the common consumer to understand statistical critical thinking though.…

Driver age isn't the factor that explains the difference, accident rate adjusted for age shifts up a similar amount for both Autopilot and regular drivers.

What truly makes a difference is the kind of road they're driving on. The accompanying paper provides a figure showing Autopilot is essentially only used on freeways which are inherently safer. Adjusting for that seems to explain most of the difference.

Though personally I'd prefer to have just compared the accident rate on freeways for both types rather than this weird weighting method they try to use.

Re: Normalized crash data shows Autopilot is much less safe than Tesla claims

#120

I find this headline to be pretty misleading. * It doesn't find anything to suggest Autopilot is less safe than claimed. It finds reasons to believe Autopilot's improvement on safety relative to humans is less than claimed * Under both the road adjusted model; and the road+age adjusted model, Autopilot outperforms the average driver on average. Not by much, but its there. * Everyone here seems to be assuming that the…

Perhaps the downvotes are because most people would be apprehensive about putting their life in the hands of an "average" safe driver that's built from software? I probably wouldn't allow any other average safe human to drive me, if the alternative of me driving myself is available.
Post reply on HN