NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction • Hacker News Board

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#311

Post by bdnsgt » Fri, Jan 20, 2017, 3:35 PM UTC

agumonkey wrote: post #237

uncoder0 wrote: post #211

After looking at the report it looks like Tesla ran into the same issue we did in the 2007 DARPA Urban Challenge. The trailer was higher than the front facing sensors. We and most other teams had all assumed 'Ground Based Obstacles' meant that any obstacles on the test track would make contact with the ground in the lane of travel. DARPA decided to put a railroad bar across the street and expected cars to back up and…

I don't understand why the paradigm isn't about full volumetric prediction .. is it because people were too tired of solving all other issues (surely possible at the time of DARPA challenge, but Tesla had time and money since); or hardware limits (like scanning a larger area would impede signal quality or make processing too heavy ..)

I'm not sure what you mean, precisely. The grand goal IS complete autonomous function, but it's wise to approach this with caution. Legal and moral grey area abound, new technology brings new technophobia, definite "completeness" would require rigorous proof and we're JUST getting autonomous vehicles into the wild. But the kicker is that if you actually care about the safety of the public, then you have to care about whether or not the public will adopt the new technology that makes them safer-- and to do that you have to introduce it slowly and respond to criticism.

Other companies are being even more cautious; Ford isn't putting autonomy on the road until 2021. In the meanwhile Tesla can sell products as Level 2, and perhaps should do so until roads are as safe as commercial aircraft.

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#312

Post by feedjoelpie » Fri, Jan 20, 2017, 3:40 PM UTC

gambiting wrote: post #288

blazespin wrote: post #246

Earlier quoted context omitted.

The only thing that matters is "NHTSA notes that crash rates involving Tesla cars have dropped by almost 40 percent since the wide introduction of Autopilot" Even if autopilot was faulty in some way, if more people were living than dying - does it matter?

Well, yeah. Imagine there's heart surgery which has only 20% success rate. So every time 100 people go in for that surgery, only 20 survive. Now, someone makes a robot that has 100% success rate, but every 100 operations, it has a glitch in the sofware, where it stabs and kills the patient instead. It has been calculated that on average, using the robot only kills 1-2 out of 100 people going in for surgery, compared…

This analogy doesn't work because you don't do manual heart surgery on yourself. Whether or not a robot does the surgery, the hospital's butt is on the line.

The biggest difference between an autopilot accident and a traditional accident is fault. Most accidents are driver error. Even if it drives accidents, overall, down (which it will), changing the responsibility for safe travel from mostly the individual to mostly the manufacturer has huge legal repercussions. Not to mention many people's discomfort with no longer being in control of their own safety, even if study after study shows that they shouldn't be.

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#313

Post by agumonkey » Fri, Jan 20, 2017, 4:00 PM UTC

bdnsgt wrote: post #311

agumonkey wrote: post #237

Earlier quoted context omitted.

I don't understand why the paradigm isn't about full volumetric prediction .. is it because people were too tired of solving all other issues (surely possible at the time of DARPA challenge, but Tesla had time and money since); or hardware limits (like scanning a larger area would impede signal quality or make processing too heavy ..)

I'm not sure what you mean, precisely. The grand goal IS complete autonomous function, but it's wise to approach this with caution. Legal and moral grey area abound, new technology brings new technophobia, definite "completeness" would require rigorous proof and we're JUST getting autonomous vehicles into the wild. But the kicker is that if you actually care about the safety of the public, then you have to care about…

I know and I don't care. If they want public approval they can run demonstration, and probably 392584 other marketing techniques to generate social interest. Crappy solutions suck; period.

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#314

Post by Coincoin » Fri, Jan 20, 2017, 4:31 PM UTC

DonHopkins wrote: post #109

snewman wrote: post #2

Tesla comes off extremely well in this report. For one thing, the 40% statistic cited in the headline appears to be well supported by the NHTSA report (section 5.4) and actually manages to frame the incident in a very positive light: ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either in…

The car should immediately pull over and shut down if it detects any activity on the driver's Twitter or Facebook feeds: Social Media Strike Out!

Are you telling me my car should have pulled over on the highway and potentially caused an accident when my wife used my phone last week?

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#315

Post by uncoder0 » Fri, Jan 20, 2017, 4:34 PM UTC

mjevans wrote: post #212

uncoder0 wrote: post #211

After looking at the report it looks like Tesla ran into the same issue we did in the 2007 DARPA Urban Challenge. The trailer was higher than the front facing sensors. We and most other teams had all assumed 'Ground Based Obstacles' meant that any obstacles on the test track would make contact with the ground in the lane of travel. DARPA decided to put a railroad bar across the street and expected cars to back up and…

Predictions like that only work out if the lesson is sufficiently broadcast. Clearly, since this is still newsworthy here, that is not the case. (However this back channel is helping.)

Agreed, this is a fantastic idea. All the DARPA documents from all three autonomous car related Grand Challenges should be availible. Not sure if they are all released on their website if not they're FOIA'able since most everything DARPA does is public.

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#316

Post by mikeash » Fri, Jan 20, 2017, 4:36 PM UTC

kalleboo wrote: post #287

mikeash wrote: post #197

Earlier quoted context omitted.

It doesn't require continuous engagement. You can drive with hands off the wheel for several minutes at a time in many situations.

Didn't they make the check more aggressive after the latest accident? Or am I just mis-remembering speculation.

They've been tweaking it pretty regularly since the beginning. The initial release had no timed events at all, and only asked the driver for hands on the wheel when the system's confidence dropped below a certain level. Then they added some timers as well (I think before this crash) and have been tweaking the situations and durations. Right now, I believe at highway speeds it's either 1 or 3 minutes depending on whether you're following a car or not. (The system has more confidence when it has another car to track, so it's more lenient there.)

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#317

Post by cr0sh » Fri, Jan 20, 2017, 5:12 PM UTC

krschultz wrote: post #117

dkonofalski wrote: post #8

I don't really know why this is surprising. Computers are already better than humans at most tasks that involve a limited set of behaviors and they have infinitely better response time than humans (and continue to get better). How could anyone think that a report like this was going to end up any differently?

The history of self driving cars shows that it is not an easy task. We've been working on this for over 15 years. I was on a self driving car team during 2008-2010. We won multiple intercollegiate competitions. At that time I was certain it would be 20+ years before we had something that could drive on the roads. The progress has been spectacular.

> We've been working on this for over 15 years.

Over 25 years, actually - CMU's ALVINN (1989) was one of the first full-sized vehicle implementations:

http://repository.cmu.edu/cgi/viewcontent.cgi?article=2874&c...

It was preceded by an E-MAXX based RC car system called DAVE (DARPA Autonomous Vehicle).

CMU also worked on an investigated other systems, going back to 1984 or so:

http://www.cs.cmu.edu/afs/cs/project/alv/www/

They had an interesting system called the Terragator, from at least 1983; I can't find a paper about this system, though I know they exist - there is a video, though:

https://www.youtube.com/watch?v=beGRM9ZmWlI

One might consider the Stanford Cart as an earlier example (I believed it used computer vision techniques - and was really slow at computing the "next move"):

https://web.stanford.edu/~learnest/cart.htm

(side note: As part of SAIL, the Cart was instrumental in paving the way for Thrun's Stanley - which won the DARPA Grand Challenge, and ultimately led to Google's efforts in self-driving cars)

ALVINN was unique in that it used a (simple compared to today's CNNs and deep learning) neural network (and a WARP systolic array):

https://en.wikipedia.org/wiki/WARP_(systolic_array)

...coupled to a camera to digitize and model/learn the road features for driving, while "watching" the human driver (early form of "behavioral cloning" - but it wasn't termed as such in the various papers you can find about ALVINN).

Interestingly, during the ML Class in 2011 (which I participated in), one of the students - after we were shown several times how ALVINN worked - decided to replicate the system using "modern hardware":

http://blog.davidsingleton.org/nnrccar/

We've really come a long way...

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#318

Post by ghaff » Fri, Jan 20, 2017, 5:20 PM UTC

Piskvorrr wrote: post #309

gambiting wrote: post #289

Earlier quoted context omitted.

That's also a solved problem - Mercedes cars have "attention assist" which can figure out how attentive you are by measuring how you drive. If you are literally falling asleep at the wheel, the system will get very annoying and insist that you stop for some rest. Of course, whatever algorithm they are using won't work if the car is driving itself, you can't judge the reaction time if all the driver is doing is holdin…

One of the approaches used on long-distance buses is "track head position and angle." If you're "looking" into your lap or into the ceiling while driving, that's a fairly reliable signal that something is wrong, never mind how you're holding the steering wheel.

The technology is almost certainly there to very aggressively monitor "eyes on the road" and quickly blare alarms if they aren't. It's probably fair to say that would be a very unpopular auto feature.

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#319

Post by mikeash » Fri, Jan 20, 2017, 6:25 PM UTC

yohui wrote: post #227

greglindahl wrote: post #220

Earlier quoted context omitted.

Yes, Tesla cars had AEB since March 2015. Autopilot (with autosteer) came out in October 2015. Presumably Tesla's AEB gets improved a little with every firmware release.

Thanks. But the NHTSA report looks at data starting with "MY 2014" (i.e. calendar year 2013) cars. If AEB did lead to a 40% reduction in crash rates for Tesla cars, as it did for other car models, I suspect that moving the dividing line by five months later wouldn't change the figures much: you would still have a lot more crashes prior to AEB and fewer after.

Tesla model years match the year of production. 2014 Teslas were built in 2014.

Tesla rolled out AEB in March 2015 as a software update for all cars with the appropriate hardware. All Teslas produced starting in mid-September 2014 have the appropriate hardware, so several thousand MY 2014 Teslas have AEB as of early 2015.

Re: NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

#320

Post by mikeash » Fri, Jan 20, 2017, 6:31 PM UTC

ebbv wrote: post #42

cryptoz wrote: post #34

Earlier quoted context omitted.

35,000 people die from car accidents every single year in the USA alone. Other drivers are not "really good".

Personally I believe most people are really terrible drivers. That being said, there were 210 million drivers in the US in 2009. There's probably a lot more today. That's just 0.01% of drivers. That fact makes me have to question just how bad people really are, despite my own personal opinion. Or maybe cars are just really safe. A better statistic might be number of accidents. In 2015 2.44 million people were injured…

The average driver goes about 10 years between a crash, which doesn't seem to be all that good.

Another factor is the extent to which bad driving doesn't result in an accident, either because of infrastructure, other drivers reacting, or just a lack of cars. For example, most red light runners don't cause a crash because stoplights usually have an all-red period to compensate, there often aren't other cars to crash into, and other cars might brake to avoid crashing into the culprit.

Many crashes require two bad drivers to misbehave at about the same time. For example, I couldn't tell you how many times I've had somebody change lanes into me, but it hasn't (yet) resulted in a crash because I've seen it and gotten out of the way before they hit.