Tesla crash victim had complained about auto-pilot in same location • Hacker News Board

Re: Tesla crash victim had complained about auto-pilot in same location

#211

Post by brfox » Sat, Mar 31, 2018, 3:44 AM UTC

greglindahl wrote: post #180

oldgradstudent wrote: post #130

Earlier quoted context omitted.

They also provide numbers without any context. Is a crash every 85,000 times in that intersection means the Autopilot is safe or is it a death trap? There are over 200,000 cars a day driving at that stretch of highway, if a car crashes there once a week, then the Tesla Autopilot is an order of magnitude more dangerous than a human driver.

How is Tesla supposed to know, yet, if Autopilot actually made the maneuver? They're saying what they know. Edit: in a posting today, Tesla has said more: https://www.tesla.com/blog/update-last-week’s-accident but at the time of the earlier blog posting, Tesla says they did not know yet.

From their blog: "In the US, there is one automotive fatality every 86 million miles across all vehicles from all manufacturers. For Tesla, there is one fatality, including known pedestrian fatalities, every 320 million miles in vehicles equipped with Autopilot hardware. If you are driving a Tesla equipped with Autopilot hardware, you are 3.7 times less likely to be involved in a fatal accident."

Notice it says one fatality per "320 million miles in vehicles equipped with Autopilot hardware." But, how many of those miles (or what percent of the time) does a Tesla driver use Autopilot? Also, maybe those good stats are due to the structural safety features and not the autopilot. It may be more fair to compare to other cars with excellent crash protection but no autopilot.

I think it would be a great service to the world to improve driving safety, but maybe we need to really start looking at the stats and get some more hubris as we transition to full autopilot. For example, require that drivers keep their hands on the wheel and eyes on the road. When more cars have autopilot, then mesh behavior between vehicles, and e-road features, then maybe we'll be more ready for driver-less cars.

Re: Tesla crash victim had complained about auto-pilot in same location

#212

Post by anonytrary » Sat, Mar 31, 2018, 3:44 AM UTC

nemothekid wrote: post #201

anonytrary wrote: post #189

Earlier quoted context omitted.

Then let's add some perspective. You must be referring this[0] paper. If the average person puts on 1,000 miles per month[1], then that means they'd have to deal with disengagement (a mishap) at least twice a year, which is not acceptable for fully autonomous driving. I'm going to define a "fully autonomous vehicle" as "a vehicle which should not ever require me to sit in the front seat and control it under any conce…

> I don't even want the probability of encountering a mishap to be once per lifetime, This doesn't seem reasonable - Waymo's report doesn't dive in depth enough about each disengagement to warrant this sort of extreme reliability. If "2 disengagements" per year, were at most fenderbenders - something I'd wager humans do way more than twice per year - that would be a very different story than if those 2 disengagements…

> This report covers disengagements following the California DMV definition, which means "a deactivation of the autonomous mode when a failure of the autonomous technology is detected or when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle.”

So, you're right, there's no clear distinction, but I would further argue that it doesn't matter. Even if only 1/1000 disengagements are fatal, my conclusion remains the same. I think we're splitting hairs at this point, though.

Even if not fatal, I highly doubt a significant fraction of such events (as defined above) would allow me take a nap upon departure and wake up at my destination, so it would still be unacceptable to me. I guess we have to agree on what an acceptable end-game is for fully autonomous vehicles. If you think "waking up on the shoulder exchanging insurance" is acceptable, then that would indeed change the numbers (but by how much? Two, maybe four orders of magnitude?).

Humans get into fender-benders all the time, but surely we'd strive to eradicate this inefficiency in the automated driver. I think this is still an active area of debate; some assembly-line work can be made more efficient with machines, but we've seen humans out-perform machines in other types of work. I think driving tends to utilize more reactive, intuitive "System 1" thinking[0], so I imagine that humans will be vastly better than machines at driving for a very long time.

[0] https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow

Re: Tesla crash victim had complained about auto-pilot in same location

#213

Post by firebones » Sat, Mar 31, 2018, 3:44 AM UTC

ethbro wrote: post #197

ineedasername wrote: post #108

Earlier quoted context omitted.

Exactly. Same lane, same time of day, same time of year (sunset/sunrise changes in lighting conditions) same weather patterns, same level of traffic. The combinatorics involved expand the possibilities to the point that some scenariors may have had little or no coverage from a dataset with 85k observations.

There was a big note in the GTC keynote this year about Nvidia using virtual environments to debug their autopilot algorithms. Think "car" being driven in a high fidelity video game. Advantage being that if you can provide representative, simulated input then you can increase the training miles by orders of magnitude in the same amount of time, limited by computation rather than physical mileage.

This is true, to the extent that the selection of input parameters to simulate provides coverage over the domain of all possible input parameters. At some point you're back to testing the failure of imagination of the simulation/test creators.

Re: Tesla crash victim had complained about auto-pilot in same location

#214

Post by theossuary » Sat, Mar 31, 2018, 3:45 AM UTC

heartbreak wrote: post #100

JohnTClark wrote: post #34

Earlier quoted context omitted.

Maybe Tesla told him that the issue was fixed and he can use it again. Why do people trust auto pilot at such high speeds? It is still new technology. Let the autopilot park your car, if it makes a mistake the only bad thing that can happen is just a few scratches and in 5-10 years use autopilot on low traffic streets and then go to the highway. You can't expect new technology to be safe.

> You can't expect new technology to be safe. Maybe I’m old fashioned, but I expect all of the technology in my mass-produced car to be safe, where safe is defined as “will not steer my vehicle into a barrier at freeway speeds.”

Great, bet your life on it then.

I don't understand most people's perspective on this. It's like walking out in a crosswalk without looking; sure if you get hit the driver may go to jail, but you'll still be dead (or seriously injured).

Use your own brain, you can't rely on others to keep you alive. At the end of the day you're the one with whose got the most to lose, always.

Re: Tesla crash victim had complained about auto-pilot in same location

#215

Post by throwaway2048 » Sat, Mar 31, 2018, 3:46 AM UTC

ethbro wrote: post #197

ineedasername wrote: post #108

Earlier quoted context omitted.

Exactly. Same lane, same time of day, same time of year (sunset/sunrise changes in lighting conditions) same weather patterns, same level of traffic. The combinatorics involved expand the possibilities to the point that some scenariors may have had little or no coverage from a dataset with 85k observations.

There was a big note in the GTC keynote this year about Nvidia using virtual environments to debug their autopilot algorithms. Think "car" being driven in a high fidelity video game. Advantage being that if you can provide representative, simulated input then you can increase the training miles by orders of magnitude in the same amount of time, limited by computation rather than physical mileage.

The problem is the simulation by definition can only simulate things that are accounted for. Any number of completely arbitrary out of the blue things can happen in real life.

Re: Tesla crash victim had complained about auto-pilot in same location

#216

Post by tim333 » Sat, Mar 31, 2018, 3:46 AM UTC

hagope wrote: post #170

Cshelton wrote: post #99

Earlier quoted context omitted.

In Texas, I've seen crushed barriers remain collapsed for weeks on end. Either that or they are just hit again right after being replaced. Which tells me, it's a poorly designed road and causes confusion for drivers. Which in fact may be what this Tesla crash turns out to be.

Exactly, I drive past this barrier every day...the problem is the left two lanes on 101 are carpool/EV lane so Tesla drivers just zoom down it...at this particular exit, the left carpool lane leads to an HOV flyover exit which puts you on 85. If you are not paying attention (ie on autopilot flying past traffic), you will end up on a completely different highway! I see people very frequently swerve out of the flyover…

photo, I think https://imgur.com/a/iMY1x (copied from elsewhere on this thread)

Re: Tesla crash victim had complained about auto-pilot in same location

#217

Post by modeless » Sat, Mar 31, 2018, 3:47 AM UTC

_ph_ wrote: post #49

One thing to remember in all the discussions about the accident is, that there is no information available whether the autopilot was active at the time of the accident.

Tesla has just announced that autopilot was on during the crash. https://www.tesla.com/blog/update-last-week’s-accident

It sounds like they are saying the driver had 5 seconds to notice the problem and react. It is scary to think that when you are on autopilot you may be 5 seconds away from death at any time. Better not take those eyes off the road!

Re: Tesla crash victim had complained about auto-pilot in same location

#218

Post by throwaway2048 » Sat, Mar 31, 2018, 3:48 AM UTC

tensor wrote: post #176

buvanshak wrote: post #71

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Catastrophic_interference >Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to completely and abruptly forget previously learned information upon learning new information.

These self driving systems are not likely learning in realtime. When the algorithms are updated, best practice is to validate their performance on a large regression set. So something like this should be caught during validation, assuming your test data is good.

No set of test data can exist that covers every real world situation.

Re: Tesla crash victim had complained about auto-pilot in same location

#219

Post by tempestn » Sat, Mar 31, 2018, 3:49 AM UTC

todd8 wrote: post #173

gowld wrote: post #125

Earlier quoted context omitted.

Whoa whoa whoa, "auto-pilot" is NOT self-driving. Tesla goes to great pains to make auto-pilot sound like self-driving while still stringently claiming that it requires constant supervision, like an airplane auto-pilot.

Yes, but my friends that own Tesla's actually believe that self-driving cars are almost here. When I say that it's at least ten years away and likely much longer because it will take big breakthroughs (strong AI, lot's of knowledge about the world, etc.) they look at me like I'm crazy. I really hope it happens in my lifetime, but I remember talking about Greenblatt's chess program (see [1]) when I was at MIT in the 6…

Tesla's current autopilots are pretty far off, but I think Waymo and possibly others are much closer.

Re: Tesla crash victim had complained about auto-pilot in same location

#220

Post by firebones » Sat, Mar 31, 2018, 3:54 AM UTC

albertop wrote: post #167

dazc wrote: post #54

Earlier quoted context omitted.

'I was so excited about auto-pilot and dreamed of getting in my car and sleeping while it made cross-country trips. So much for that, that seems way far out.' My only experience of automated driving is the feature where you can set a maximum speed. This is useful in the UK since average speed cameras are commonplace on the motorway network nowadays and it is easy for your concentration to lapse and drift over the cur…

If you just need to follow speed limits, Tesla can do it perfectly fine. It knows speed limits for most roads. It would fail if the speed limit is due to construction or speed trap :-)

Great point. What's the over-under on a road-worker death when autopilot plows through a confusingly-marked construction zone at 70MPH?