Live data from Hacker News

US probes Tesla's Full Self-Driving software after fatal crash

reuters.com

281–290 of 1001 posts

Re: US probes Tesla's Full Self-Driving software after fatal crash

#281

Earlier quoted context omitted.

My dream is of a future where humans are banned from driving without special licenses.

So.........like right now you mean? You need a special licence to drive on a public road right now.

Geez, clearly they mean like a CDL

Re: US probes Tesla's Full Self-Driving software after fatal crash

#282

I don't understand why this debate/probing is not just data driven. Driving is all big data. https://www.tesla.com/VehicleSafetyReport This report does not include fatalities, which seems to be the key point in question. Unless the above report has some bias or is false, Teslas in autopilot appear 10 times safer than the US average. Is there public data on deaths reported by Tesla? And otherwise, if the stats say it…

> Unless the above report has some bias or is false

Welcome to Tesla.

The report measures accidents in FSD mode. Qualifiers to FSD mode: the conditions, weather, road, location, traffic all have to meet a certain quality threshold before the system will be enabled (or not disable itself). Compare Sunnyvale on a clear spring day to Pittsburgh December nights.

There's no qualifier to the "comparison": all drivers, all conditions, all weather, all roads, all location, all traffic.

It's not remotely comparable, and Tesla's data people are not that stupid, so it's willfully misleading.

> This report does not include fatalities

It also doesn't consider any incident where there was not airbag deployment to be an accident. Sounds potentially reasonable until you consider:

- first gen airbag systems were primitive: collision exceeds threshold, deploy. Currently, vehicle safety systems consider duration of impact, speeds, G-forces, amount of intrusion, angle of collision, and a multitude of other factors before deciding what, if any, systems to fire (seatbelt tensioners, airbags, etc.) So hit something at 30mph with the right variables? Tesla: "this is not an accident".

- Tesla also does not consider "incident was so catastrophic that airbags COULD NOT deploy*" to be an accident, because "airbags didn't deploy". This umbrella could also include egregious, "systems failed to deploy for any reason up to and including poor assembly line quality control", as also not an accident and also "not counted".

> Is there public data on deaths reported by Tesla?

They do not.

They also refuse to give the public much of any data beyond these carefully curated numbers. Hell, NHTSA/NTSB also mostly have to drag heavily redacted data kicking and screaming out of Tesla's hands.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#283
post #224

> The collision happened because the sun was in the Tesla driver's eyes, so the Tesla driver was not charged, said Raul Garcia, public information officer for the department. Am I missing something or is this the gross miscarriage of justice that it sounds like? The driver could afford a $40k vehicle but not $20 polarized shades from Amazon? Negligence is negligence.

Not to mention that when you can't see, you slow down? Does the self-driving system do that sufficiently in low visibility? Clearly not if it hit a pedestrian with enough force to kill them. The article mentions that Tesla's only use cameras in their system and Musk believes they are enough, because humans only use their eyes. Well firstly, don't you want self-driving systems to be better than humans? Secondly, human…

Unfortunately there is also an AI training problem embedded in this. As Mobileye says, there are a lot of driver decisions that are common, but wrong. The famous example is rolling stops, but also failing to slow down for conditions is really common.

It wouldn't shock me if they don't have nearly enough training samples of people slowing appropriately for visibility with eyes, much less slowing for the somewhat different limitations of cameras.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#284

Lots of people are asking how good the self driving has to be before we tolerate it. I got a one month free trial of FSD and turned it off after two weeks. Quite simply: it's dangerous. - It failed with a cryptic system error while driving - It started making a left turn far too early that would have scraped the left side of the car on a sign. I had to manually intervene. - In my opinion, the default setting accelera…

The thing that doesn't make sense is the numbers. If it is dangerous in your anecdotes, why don't the reported numbers show more accidents when FSD is on? When I did the trial on my Tesla, I also noted these kinds of things and felt like I had to take control. But at the end of the day, only the numbers matter.

What numbers? Who’s measuring? What are they measuring?

Re: US probes Tesla's Full Self-Driving software after fatal crash

#286
post #229

Earlier quoted context omitted.

> Pedestrian' in this context seems pretty misleading What's misleading? The full quote: "A red Tesla Model Y then hit the 4Runner and one of the people who exited from it . A 71-year-old woman from Mesa, Arizona, was pronounced dead at the scene." If you exit a vehicle, and are on foot, you are a pedestrian. I wouldn't expect FSD's object recognition system to treat a human who has just exited a car differently than…

The description has me wondering if this was definitely a case where FSD was being used. There have been other cases in the past where drivers had an accident and claimed they were using autopilot when they actually were not. I don't know for sure, but I would think that the car could detect a collision. I also don't know for sure, but I would think that FSD would stop once a collision has been detected.

> FSD would stop once a collision has been detected.

Fun fact, at least until very recently, if not even to this moment, AEB (emergency braking) is not a part of FSD.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#287
post #224

Earlier quoted context omitted.

Not to mention that when you can't see, you slow down? Does the self-driving system do that sufficiently in low visibility? Clearly not if it hit a pedestrian with enough force to kill them. The article mentions that Tesla's only use cameras in their system and Musk believes they are enough, because humans only use their eyes. Well firstly, don't you want self-driving systems to be better than humans? Secondly, human…

The Tesla knows when it's cameras and blinded by sun and act accordingly or tells the human to take over.

Expect when it doesn't actually do that, I guess? Like when this pedestrian was killed?

Re: US probes Tesla's Full Self-Driving software after fatal crash

#288

The interesting question is how good self-driving has to be before people tolerate it. It's clear that having half the casualty rate per distance traveled of the median human driver isn't acceptable. How about a quarter? Or a tenth? Accidents caused by human drivers are one of the largest causes of injury and death, but they're not newsworthy the way an accident involving automated driving is. It's all too easy to se…

Many people don't (and shouldn't) take the "half the casualty rate" at face value. My biggest concern is that Waymo and Tesla are juking the stats to make self-driving cars seem safer than they really are. I believe this is largely an unintentional consequence of bad actuary science coming from bad qualitative statistics; the worst kind of lying with numbers is lying to yourself.

The biggest gap in these studies: I have yet to see a comparison with human drivers that filters out DUIs, reckless speeding, or mechanical failures. Without doing this it is simply not a fair comparison, because:

1) Self-driving cars won't end drunk driving unless it's made mandatory by outlawing manual driving or ignition is tied to a breathalyzer. Many people will continue to make the dumb decision to drive themselves home because they are drunk and driving is fun. This needs regulation, not technology. And DUIs need to be filtered from the crash statistics when comparing with Waymo.

2) A self-driving car which speeds and runs red lights might well be more dangerous than a similar human, but the data says nothing about this since Waymo is currently on their best behavior. Yet Tesla's own behavior and customers prove that there is demand for reckless self-driving cars, and manufacturers will meet the demand unless the law steps in. Imagine a Waymo competitor that promises Uber-level ETAs for people in a hurry. Technology could in theory solve this but in practice the market could make things worse for several decades until the next research breakthrough. Human accidents coming from distraction are a fair comparison to Waymo, but speeding or aggressiveness should be filtered out. The difficulty of doing so is one of the many reasons I am so skeptical of these stats.

3) Mechanical failures are a hornets' nest of ML edge cases that might work in the lab but fail miserably on the road. Currently it's not a big deal because the cars are shiny and new. Eventually we'll have self-driving clunkers owned by drivers who don't want to pay for the maintenance.

And that's not even mentioning that Waymos are not self-driving, they rely on close remote oversight to guide AI through the many billions of common-sense problems that computets will not able to solve for at least the next decade, probably much longer. True self-driving cars will continue to make inexplicably stupid decisions: these machines are still much dumber than lizards. Stories like "the Tesla slammed into an overturned tractor trailer because the AI wasn't trained on overturned trucks" are a huge problem and society will not let Tesla try to launder it away with statistics.

Self-driving cars might end up saving lives. But would they save more lives than adding mandatory breathalyzers and GPS-based speed limits? And if market competition overtakes business ethics, would they cost more lives than they save? The stats say very little about this.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#289

Earlier quoted context omitted.

My dream is of a future where humans are banned from driving without special licenses.

So.........like right now you mean? You need a special licence to drive on a public road right now.

The problem is it’s obviously too easy to get one and keep one, based on some of the drivers I see on the road.

Re: US probes Tesla's Full Self-Driving software after fatal crash

#290

Lots of people are asking how good the self driving has to be before we tolerate it. I got a one month free trial of FSD and turned it off after two weeks. Quite simply: it's dangerous. - It failed with a cryptic system error while driving - It started making a left turn far too early that would have scraped the left side of the car on a sign. I had to manually intervene. - In my opinion, the default setting accelera…

It's not just about relative safety compared to all human driving.

We all know that some humans are sometimes terrible drivers!

We also know what that looks like: Driving too fast or slow relative to surroundings. Quickly turning every once in a while to stay in their lane. Aggressively weaving through traffic. Going through an intersection without spending the time to actually look for pedestrians. The list goes on..

Bad human driving can be seen. Bad automated driving is invisible. Do you think the people who were about to be hit by a Tesla even realized that was the case? I sincerely doubt it.

Post reply on HN