Tesla confirms “Autopilot” crash in Montana • Hacker News Board

Re: Tesla confirms “Autopilot” crash in Montana

#431

Post by etendue » Wed, Jul 13, 2016, 3:43 PM UTC

Vraxx wrote: post #405

TetOn wrote: post #363

Earlier quoted context omitted.

>In this incident, the driver was using autopilot in a fashion that it should not be; twisting road at high speed. The driver IS at fault. I disagree on this point. In any system that's supposed to be sophisticated enough to drive the car and but also carries a giant caveat like "except in these common driving situations..." failure is not instantly down to "user error." Such a system should gracefully refuse to take…

It's not lulling anybody into anything! The guy was given multiple warnings by the car that what he was doing was unsafe. The last of which was extremely pertinent to the crash. To quote the article, "As road conditions became increasingly uncertain, the vehicle again alerted the driver to put his hands on the wheel". This is after the initial alerts telling him that he needs to remain vigilant while using this featu…

The Tesla warnings remind me of Charlie and the Chocolate Factory, where one of the brats were about to do something disaterous and Gene Wilder would in a very neutral tone protest "no, stop, you shouldn't do that".

From a risk management standpoint, Tesla's design doesn't strike me as sufficient. Warnings without consequences are quickly ignored.

Re: Tesla confirms “Autopilot” crash in Montana

#432

Post by tdaltonc » Wed, Jul 13, 2016, 3:43 PM UTC

icc97 wrote: post #376

mattlondon wrote: post #330

Earlier quoted context omitted.

Er - that is wrong. 2014 Model S got 5-star/82%[0] (where 100% is best) adult occupant protection on EuroNCAP. For comparison, the smaller, lighter, cheaper 2015 Honda Jazz got 5-star/93%[1] Other cars are significantly better than this [2]. E.g. 2016 Alfa Romeo Giulia 5-star/98%, 2016 Toyota Prius 5-star/92%, 2015 Volvo XC90 5-star/97%, 2014 Porsche Macan 5-star/88%. I could go on, but the Model S is not the safest…

The differences between the Tesla and the Jazz are a bit strange. The main difference in score between the two appears to be the side impact from a pole - which is all 'good' for the Jazz, but it was 'weak' for the Tesla in the body section. But if you look at the two videos of the cars, after the pole crash, Tesla is almost completely intact (29 kph) [1] and the Jazz is half destroyed (32 kph) [2]. The Jazz impact w…

The point of a crash test is not to show that the vehicle is super rigid and impervious to damage. It is to show that a human inside the car is safe. To that end, the car should "crumple." It shouldn't remain 100% rigid, and it should fall apart like tissue paper. It should absorb as much kinetic energy as possible so that the human doesn't have to.

The Jazz is doing the right thing.

Re: Tesla confirms “Autopilot” crash in Montana

#433

Post by FireBeyond » Wed, Jul 13, 2016, 3:45 PM UTC

HNaTTY wrote: post #132

greendestiny wrote: post #121

Earlier quoted context omitted.

Once again Tesla have made the discussion about the driver. They have put an auto-control system out in the wild that steers the car into hard obstacles at high speed. Beeps to warn the driver at some stage in the preceding interval do not make this control systems failure any more acceptable. It still misjudged situation as acceptable to continue and it still drove into obstacles. Those are significant failures and…

I agree; I think it's despicable that Tesla is acting in such a defensive manner. -- "No force was detected on the steering wheel for over two minutes after autosteer was engaged," said Tesla, which added that it can detect even a very small amount of force, such as one hand resting on the wheel. "As road conditions became increasingly uncertain, the vehicle again alerted the driver to put his hands on the wheel," sa…

Exactly! In this case the driver may have been at fault.

What happens next time, when the driver has a stroke?

"Well, he didn't put his hand on the wheel when the car told him to, it's not our fault!"

Like you say, slow it down. Sure, the driver of a non-autonomous vehicle who has a stroke might be in a whole world of trouble, but isn't (semi-) autonomy meant to be a _better_ system?

Re: Tesla confirms “Autopilot” crash in Montana

#434

Post by danso » Wed, Jul 13, 2016, 3:47 PM UTC

eatporktoo wrote: post #413

hanklazard wrote: post #387

The autopilot feature seems like one that should be taken through rigorous testing approved by the appropriate regulatory structure before being allowed out on the roads. The fact that Tesla has been able to simply update firmware to include this experimental new feature on their vehicles as they please feels very "wild west" and as much as I don't tend to prefer regulation, this feels like the proper place for it. B…

The real problem is, how do you actually design a test that will catch these issues. If you have 3 crashes in millions of miles, you are pretty unlikely to find those errors during testing. I wish Tesla was more open about the way the actually test their Autopilot software.

While automated driving is a relatively new field, the science and data of traffic conditions and accidents is quite deep. You don't have to blindly wait until your beta users get into accidents. In the case of the first fatality, how is it possible that Tesla engineers were unaware of a situation in which a lightly-colored vehicle might show up on days in which the sky is bright?

Of course they were aware of that situation. They chose to deprioritize that to reduce the incidence of false positives that came from overhead highway signs. Nothing wrong with attempting to squash false positives, which can lead to deadly situations themselves. But there was nothing preventing Tesla from doing adequate testing to determine what unintended consequences their software modifications would entail. They don't have to wait for someone to get decapitated to realize that there's a trade off in reducing false positives from overhead signs.

Re: Tesla confirms “Autopilot” crash in Montana

#435

Post by tdaltonc » Wed, Jul 13, 2016, 3:48 PM UTC

mattlondon wrote: post #330

dayaz36 wrote: post #239

Earlier quoted context omitted.

You do know Tesla has the highest safety rating of any car ever rated in history right? https://www.teslamotors.com/blog/tesla-model-s-achieves-best...

Er - that is wrong. 2014 Model S got 5-star/82%[0] (where 100% is best) adult occupant protection on EuroNCAP. For comparison, the smaller, lighter, cheaper 2015 Honda Jazz got 5-star/93%[1] Other cars are significantly better than this [2]. E.g. 2016 Alfa Romeo Giulia 5-star/98%, 2016 Toyota Prius 5-star/92%, 2015 Volvo XC90 5-star/97%, 2014 Porsche Macan 5-star/88%. I could go on, but the Model S is not the safest…

Tesla was talking about their NHTSA VSS, your talking about EURO NCAP. I'm not sure how to compare the two.

Re: Tesla confirms “Autopilot” crash in Montana

#436

Post by talmand » Wed, Jul 13, 2016, 3:50 PM UTC

jessaustin wrote: post #406

talmand wrote: post #389

Earlier quoted context omitted.

>> Google admitted fault, took the blame Let's wait until Google's car kills someone and then compare their reaction to that of Tesla's.

Still waiting...

Still waiting to be able to purchase a Google self-driving car as well...

Re: Tesla confirms “Autopilot” crash in Montana

#437

Post by tdaltonc » Wed, Jul 13, 2016, 3:51 PM UTC

wepple wrote: post #402

3pt14159 wrote: post #385

The real issue is whether or not "Autopilot" is safer than a human or not. If the number of deaths for similar road conditions per distance travelled are less then Tesla should be allowed to continue the way that they are doing things. It's obvious to me the way other systems do this (by slowing down if the user doesn't put his or her hands on the wheel frequently) leads to people not using the feature because it is…

instead of aiming for "the same number or slightly less" accidents than a human, why not have humans give oversight to prevent autopilot errors, and autopilot assist where humans might make a mistake? actually try to let both computers and people excel where they do best, and massively decrease road accidents?

That sounds good to. I think 3pt14159's implicit question was, "should Tesla be allowed to continue doing this the way they want?" not "Is tesla take the best of all possible approaches to car automation?"

Re: Tesla confirms “Autopilot” crash in Montana

#438

Post by CaptSpify » Wed, Jul 13, 2016, 3:52 PM UTC

Bromskloss wrote: post #188

CaptSpify wrote: post #34

Earlier quoted context omitted.

And this is why I don't think they should call it "autopilot". The name is misleading, imo

Actually, I don't know what it does.

I'm not clear on it either, but I think it's some kind of cruise-control, with auto-steering. (If I'm wrong, someone please let me know)

Re: Tesla confirms “Autopilot” crash in Montana

#439

Post by vkou » Wed, Jul 13, 2016, 3:56 PM UTC

studentrob wrote: post #83

vkou wrote: post #54

Earlier quoted context omitted.

I hate to play Devil's advocate here, but a traditional car on cruise control will absolutely become a lethal missile if its driver has a stroke. I don't think automation makes that particular use case more dangerous.

> I hate to play Devil's advocate here, but a traditional car on cruise control will absolutely become a lethal missile if its driver has a stroke. You're talking worst case scenario. This is a case where the driver was alive and chose to ignore warnings. > I don't think automation makes that particular use case more dangerous. It's more dangerous when the autopilot driver can easily misuse the system. Add a few requ…

Autopilot absolutely should start slowing the car down if the driver takes his hands off the wheel for more then a few seconds. Anything else is reckless.

I was, however, responding to the specific scenario of 'Driver has a stroke --> Self-driving car becomes an unguided missile' by pointing out that it the exact same scenario as 'Driver has a stroke --> Regular car with cruise control becomes an unguided missile.'

Re: Tesla confirms “Autopilot” crash in Montana

#440

Post by danso » Wed, Jul 13, 2016, 4:00 PM UTC

JoeAltmaier wrote: post #430

danso wrote: post #428

Earlier quoted context omitted.

Do you have links that support the assertion that Toyota was "solely relying on junior folks for the bread-and-butter coding"?

No only my cultural experience with foreign engineers. Anybody over 25 still doing engineering/coding is considered a failure. Their career path includes management early. All the engineers I work with from Japan, Singapore, China are young and inexperienced. And they are only allowed to report success up the chain. Management is fed a constant diet of sweet crap.

I won't argue with your assumptions because they fall in line with my (completely non-experience based) assumptions about that dev culture (and the fact that my Honda onboard app is such shit), but I had thought that something like autopilot would be outsourced to a specialty firm?

http://fortune.com/2016/03/10/toyota-jaybridge-robotics/

> Toyota has brought onboard the entirety of the workforce at Jaybridge Robotics, an artificial intelligence software firm based in Cambridge, Mass.

Obviously, it's extremely possible for experienced engineers to have their work be for naught when working for an incompetent overseer. And my intuition is that automotive engineering requires such a high amount of coupling and integrated behavior that you can't just outsource your AI to an expert firm and then tack it onto the finished car. But those are causes of failures that are separate from the aspect of auto software engineer culture that you mention, which could be incompetent for their own devices while not being the primary driver of failure when it comes to autopilot features.