Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

fuelarc.com

Post reply on HN
Print view View original

421–430 of 449 posts

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#421

Post by ModernMech »

josephcsible wrote: post #347
ModernMech wrote: post #267

Earlier quoted context omitted.

This. People arguing over base rates of motorcycle accidents as if Tesla didn't get fooled by a loony tunes wall. If Waymo had killed 5 motorcyclists in SF we would know. But they operate there without an incident for years. Meanwhile, Tesla just after releasing autopilot to the world a man is decapitated because the system is deficient. Then it happened again in 2016 under eerily similar circumstance. Then we observ…

> as if Tesla didn't get fooled by a loony tunes wall Important distinction: FSD didn't get fooled by a looney tunes wall. Legacy Autopilot did.

That's not an important distinction, because the reason it was fooled is the sensor suite is deficient, which is the same whether it's FSD or Autopilot. Other less sophisticated systems are not fooled because they have sufficient sensors. FSD being a more sophisticated system doesn't negate the fact their sensor suite is fundamentally not up to the task.

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#422

Post by Retric »

bumby wrote: post #416
Retric wrote: post #313

Earlier quoted context omitted.

The same basic questions arise for things like a tire blowout. At minimum cars need to handle being on a curve when LiDAR or vision etc cuts off. Stopping on a freeway or railroad is a real risk even if people can get out, but now you’re doing risk mitigation calculations not just trying to handle normal driving. “What if someone is asleep in a level 5 car?” is a question worth considering, but we’re a long way self…

We absolutely should expect safety critical software to mitigate risk. The idea of “welp, the software is confused so just toss it back to a human” is a cop-out in risk mitigation. At the most generous, it constitutes an “administrative” control, which is the least preferred option, tantamount to giving directions in the users manual. The general control hierarchy is: remove the hazard, engineering control, PPE, admi…

I agreed, but thing about risk mitigation is there’s tradeoffs.

A self driving Taxi with a tire pressure sensor error vs total break failure are simply wildly different situations and each should get different responses. Further, designing for a possibility sleeping but valid human driver is very different than designing for a likely empty vehicle.

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#423

Post by Retric »

throw0101d wrote: post #225
Retric wrote: post #146

Earlier quoted context omitted.

Cars unlike aircraft don’t need to move forward to maintain safety. If the lidar normally has very good uptime but happens to break, you need to safely come to a complete stop and that’s basically it. Trusting vision systems for 30 seconds or even 30 minutes over the lifetime of a car is very different than trusting them for 30,000 hours. So for edge cases sure, add a “this is an emergency drive to a hospital without…

> So for edge cases sure, add a “this is an emergency drive to a hospital without LiDAR” mode but you don’t need to handle normal driving without them. As a comparison for planes, there are not "this is an emergency, please land yourself" buttons for smaller aircraft: * https://www.garmin.com/en-US/blog/aviation/five-ways-garmin-... * https://www.garmin.com/en-US/legal/aluse/

Interesting: “Autoland is designed to be used in emergency situations only and should not be used in nonemergency situations where the pilot is fully capable of landing the aircraft.” https://www.garmin.com/en-US/legal/aluse/

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#424

Post by bumby »

Retric wrote: post #422
bumby wrote: post #416

Earlier quoted context omitted.

We absolutely should expect safety critical software to mitigate risk. The idea of “welp, the software is confused so just toss it back to a human” is a cop-out in risk mitigation. At the most generous, it constitutes an “administrative” control, which is the least preferred option, tantamount to giving directions in the users manual. The general control hierarchy is: remove the hazard, engineering control, PPE, admi…

I agreed, but thing about risk mitigation is there’s tradeoffs. A self driving Taxi with a tire pressure sensor error vs total break failure are simply wildly different situations and each should get different responses. Further, designing for a possibility sleeping but valid human driver is very different than designing for a likely empty vehicle.

I don’t think anyone who work in risk mitigation would conflate those scenarios. Risk = probability x consequence. What you’re pointing to is the difference in consequence. It’s already acknowledged in mature risk mitigation programs. Eg, FMEAs list fault consequence, severity, likelihood, and detectability and don’t assume all faults are the same. The tradeoff is an engineering/business decision to assign appropriate mitigations to land in an acceptable risk posture.

If you aren’t mitigating it with the appropriate controls, you aren’t managing risk. My point is just passing the buck to the human is not an appropriate control in many critical scenarios.

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#425

Post by Ajedi32 »

Hikikomori wrote: post #224
dagw wrote: post #23

Earlier quoted context omitted.

Without knowing how many FSD miles Teslas have done compared to other brands, it's hard to judge. It could just be that Tesla owners are far more likely to trust and use their cars FSD capabilities, and thus end up in more FSD accidents. Other brands might have so bad FSD that people simply not trust it and basically never use it, and thus never get into an FSD accident.

How good can it be if it rear ends motorcycles?

It could be 100x better than human drivers and still rear end motorcycles occasionally. Without better statistical information this article tells you almost nothing.

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#426

Post by Ajedi32 »

bink wrote: post #263
Hobadee wrote: post #123

Earlier quoted context omitted.

The problem is there isn't enough data here. Killing 5 motorcycles isn't great, but if human driver killed say 100 in the same sample, 5 is actually pretty good. Of course if human drivers kill 1 or 2 in the same sample, 5 is really bad.

The difference to me, as a motorcyclist, is that to prevent humans from crashing cars into motorcycles is complicated. There are a multitude of reasons someone may not be paying attention or may not see another vehicle. This problem with Tesla has an easily implemented solution that could be deployed now but isn't. We don't have such a solution for non-Tesla crashes into motorcycles. How many motorcyclists deaths are…

Installing LIDAR isn't an "easily implemented solution", assuming that's what you mean. Otherwise why not mandate that all cars have LIDAR, self-driving or not?

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#427

Post by Retric »

bumby wrote: post #424
Retric wrote: post #422

Earlier quoted context omitted.

I agreed, but thing about risk mitigation is there’s tradeoffs. A self driving Taxi with a tire pressure sensor error vs total break failure are simply wildly different situations and each should get different responses. Further, designing for a possibility sleeping but valid human driver is very different than designing for a likely empty vehicle.

I don’t think anyone who work in risk mitigation would conflate those scenarios. Risk = probability x consequence. What you’re pointing to is the difference in consequence. It’s already acknowledged in mature risk mitigation programs. Eg, FMEAs list fault consequence, severity, likelihood, and detectability and don’t assume all faults are the same. The tradeoff is an engineering/business decision to assign appropriat…

> What you’re pointing to is the difference in consequence.

No. A break failure doesn’t guarantee a specific negative outcome, it dramatically raises the probability of various negative consequences.

My point is risk mitigation is about lowering risk but there may be no reasonably safe options. A car stopped on a freeway is still a high risk situation, but it beats traveling at 70 MPH without working cameras.

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#428

Post by bumby »

Retric wrote: post #427
bumby wrote: post #424

Earlier quoted context omitted.

I don’t think anyone who work in risk mitigation would conflate those scenarios. Risk = probability x consequence. What you’re pointing to is the difference in consequence. It’s already acknowledged in mature risk mitigation programs. Eg, FMEAs list fault consequence, severity, likelihood, and detectability and don’t assume all faults are the same. The tradeoff is an engineering/business decision to assign appropriat…

> What you’re pointing to is the difference in consequence. No. A break failure doesn’t guarantee a specific negative outcome, it dramatically raises the probability of various negative consequences. My point is risk mitigation is about lowering risk but there may be no reasonably safe options. A car stopped on a freeway is still a high risk situation, but it beats traveling at 70 MPH without working cameras.

I'm sorry but I think you’re mixing things up in the traditional sense of risk management. The probability is covered separately in risk management. A brake failure doesn't guarantee a particular consequence, but it does bound it and general practice is to assign the worst consequence. To use it on the Max scenario, MCAS can fault and not cause a plane to crash, but it still should be characterized as a "catastrophic" fault because it can cause loss of life in some scenarios. The probability is determined separately. Then post-mitigation consequence/likelihood/detectability are assessed. Take a look at how FMEAs are conducted if you still disagree; it’s fairly standardized in other safety critical domains.

I agree there may be cases where there are no reasonably safe options. That means your engineered system (especially in a public-facing product) is not ready for production because you haven't met a reasonable risk threshold.

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#429

Post by josephcsible »

ModernMech wrote: post #421
josephcsible wrote: post #347

Earlier quoted context omitted.

> as if Tesla didn't get fooled by a loony tunes wall Important distinction: FSD didn't get fooled by a looney tunes wall. Legacy Autopilot did.

That's not an important distinction, because the reason it was fooled is the sensor suite is deficient, which is the same whether it's FSD or Autopilot. Other less sophisticated systems are not fooled because they have sufficient sensors. FSD being a more sophisticated system doesn't negate the fact their sensor suite is fundamentally not up to the task.

> the reason it was fooled is the sensor suite is deficient, which is the same whether it's FSD or Autopilot.

But other people have tried to reproduce the experiment with FSD, and it wasn't fooled.

Re: Self-Driving Teslas Are Fatally Rear-Ending Motorcyclists More Than Any Other

#430

Post by Retric »

bumby wrote: post #428
Retric wrote: post #427

Earlier quoted context omitted.

> What you’re pointing to is the difference in consequence. No. A break failure doesn’t guarantee a specific negative outcome, it dramatically raises the probability of various negative consequences. My point is risk mitigation is about lowering risk but there may be no reasonably safe options. A car stopped on a freeway is still a high risk situation, but it beats traveling at 70 MPH without working cameras.

I'm sorry but I think you’re mixing things up in the traditional sense of risk management. The probability is covered separately in risk management. A brake failure doesn't guarantee a particular consequence, but it does bound it and general practice is to assign the worst consequence. To use it on the Max scenario, MCAS can fault and not cause a plane to crash, but it still should be characterized as a "catastrophic…

FMEA/FMECA are only the first step in a system reliability study, it isn’t an analysis of mitigation strategies. Fault tree analysis (etc) concerns itself with things like how long an aircraft can fly with an engine failure. Doing that involves an assessment of various risks.

> I agree there may be cases where there are no reasonably safe options. That means your engineered system (especially in a public-facing product) is not ready for production because you haven't met a reasonable risk threshold.

Individual failures should never result in such scenarios, but ditching in the ocean may be the best option after a major fuel leak, loss of all engine power, etc.

Post reply on HN