Tesla confirms “Autopilot” crash in Montana • Hacker News Board

Re: Tesla confirms “Autopilot” crash in Montana

#471

Post by FireBeyond » Wed, Jul 13, 2016, 6:29 PM UTC

kirkbauer wrote: post #467

TetOn wrote: post #411

Earlier quoted context omitted.

I just want to see "the vehicle again alerted the driver to put his hands on the wheel" followed by "and Autopilot then pulled to the side of the road, turned on the hazard lights, and required the driver to take over."

That's stupid. Autopilot doesn't have the ability to determine if it is safe to pull over. It has a fallback mode; eventually it will come to a complete stop in the middle of the road and turn the hazards on. But since this is also dangerous it makes sense to give the driver time to stop it from happening.

It's stupid to come to a gradual slowdown with hazard lights, but it's not stupid to keep blazing away at speed, with an obviously inattentive (or perhaps completely disabled) driver? I'm confused - even you acknowledge it has that fallback, but it's 'stupid' to enact it after a couple of failed warnings? When should that threshold be crossed, then?

Re: Tesla confirms “Autopilot” crash in Montana

#472

Post by oconnore » Wed, Jul 13, 2016, 6:39 PM UTC

Dylan16807 wrote: post #177

cperciva wrote: post #117

Earlier quoted context omitted.

You're both right. That's exactly how aircraft autopilot systems work... and it doesn't work. There have been several plane crashes -- most notably AF447 -- which resulted from autopilot systems disengaging when the (human) pilots had insufficient situational awareness to safely take over control of the aircraft.

The confusion between normal and alternate law on AF447 is not because the autopilot transferred control to the pilots when they weren't ready. The same thing could have happened if the transfer was made with full preparation. It was a signalling and training issue, not evidence that auto->manual transitions are bad.

If the question is "could a human be trained to use this?" then, yes, you can imagine a human reacting appropriately to the AF447 scenario.

But that's the wrong question. The right question is: does the system incorporate the success of human actors into its design? And the answer here is no. If the Airbus system works as designed, the human pilots will tend to be woefully unprepared to resolve failure modes.

Attempting to create an external training regiment to account for that design failure is a band-aid, not a solution.

Re: Tesla confirms “Autopilot” crash in Montana

#473

Post by FireBeyond » Wed, Jul 13, 2016, 6:43 PM UTC

NhanH wrote: post #454

jessaustin wrote: post #404

Earlier quoted context omitted.

Yeah this thread has really squicked me out. Of course dehumanization is an age-old tactic for enabling asshole behaviors of all sorts. I'd like to hope, however, that outside the current commuter-hell driving context, few people would even admit to themselves, let alone broadcast to the world, their considered preference for killing children. That is, when they no longer drive, perhaps people won't be quite so insan…

I don't even drive at all, and yet this thread irks me from the other direction. Dehumanization is an age-old tactic for enabling asshole behaviors, and bringing children into a discussion is an age old tactic for justifying irrational decisions ("Think of the children"). You claimed that any car may be driven slowly enough to avoid killing people on the road with only mere convenient to others. That's patently false…

"You claimed that any car may be driven slowly enough to avoid killing people on the road with only mere convenient to others. That's patently false: firetruck."

As a firefighter I'd like to offer a few thoughts on this:

- fire engines, let alone ladder trucks are big but slow. Older ones have the acceleration of a slug and struggle to hit high speeds

- with very, very few exceptions, the difference of 10 seconds to 2 minutes that your arrival makes is likely to have no measurable difference (do the math, most departments are likely to have policies allowing only 10mph above posted limits at best, and if you're going to a call 3mi away...)

- again, as mentioned, most departments have a policy on how far the speed limit can be exceeded. It might feel faster when it goes by when you're pulled over, but realistically there won't be much difference

- "due caution and regard" - almost all states have laws saying that there's implied liability operating in "emergency mode" - that is, you can disobey road laws as part of emergency operations, but any incident that happens as a result thereof will be implied to be the fault of the emergency vehicle/operator until and unless proven otherwise

If I'm driving an engine, blazing around without the ability to respond to conditions as much as I would in a regular situation/vehicle, then emergency mode or not, I am in the wrong.

Re: Tesla confirms “Autopilot” crash in Montana

#474

Post by FireBeyond » Wed, Jul 13, 2016, 6:46 PM UTC

jessaustin wrote: post #427

pc86 wrote: post #423

Earlier quoted context omitted.

Would I? Of course not. Would my insurance company? Probably.

Let me guess, you haven't done much work in public relations? Much worse for a company than doing some evil reprehensible shit, is doing such shit in a fashion that's an easy, emotional "narrative hook" for the media. A grieving mother clutching a tear-stained auto repair bill itemized with "child's-head-sized dent, one" is the easiest hook imaginable.

Maybe so. But Google for "parents of dead child sent bill for airlift" - if you want to be pedantic, you can even argue that in some of those cases, there may not have been any choice, only an implied consent by law that says that "a reasonable person would have wanted to be flown to hospital".

You'll find examples. And for every waived bill you'll also hear "well, regardless of the tragic outcome, a service was rendered, and we have the right to be paid for our work".

Please note that I am carefully avoiding passing any judgment on the morality of any of the above.

Re: Tesla confirms “Autopilot” crash in Montana

#475

Post by FireBeyond » Wed, Jul 13, 2016, 6:50 PM UTC

gaur wrote: post #128

Animats wrote: post #109

With three accidents on "autopilot" in a short period, the defects in Tesla's design are becoming clear. 1. Tesla's "hands on wheel" enforcement is much weaker than their competitors. BMW, Volvo, and Mercedes have similar systems, but after a few seconds of hands-off-wheel, the vehicle will start to slow. Tesla allows minutes of hands-off time; one customer reports driving 50 miles without touching the wheel. Tesla i…

> Tesla allows minutes of hands-off time; one customer reports driving 50 miles without touching the wheel. How are these numbers compatible with each other? Was the car going at >1000 miles per hour?

Multiple people here have reported that their car has allowed 15 minutes or more without having to touch the wheel. If each warning reset that timer, by design or bug, then you could easily travel 50 miles plus.

Plus, I'm almost certain that Tesla's goal with the system isn't to "have an autonomous system that you have to interact with steadily", hence the 15 minute warning in the first place. They have perverse incentives - to allow it in the first place implies reliability that may or may not be there.

Re: Tesla confirms “Autopilot” crash in Montana

#476

Post by FireBeyond » Wed, Jul 13, 2016, 7:09 PM UTC

the_common_man wrote: post #7

Nobody reads manuals and follows instructions . I feel Tesla is going to learn this lesson that hard way. From the article: "We specifically advise against its use at high speeds on undivided roads," it said. Tesla states clearly in its owner's manual that drivers should stay alert and keep their hands on the wheel to avoid accidents when the Autopilot feature is engaged.

"... and then we post blog entries, and our CEO retweets videos of people using our Autopilot exactly in the manner by which we 'specifically advise'."

Similar with the garage door incident - "Well, no-one said it should be used without full supervision! It needs to be watched!" ... like in the Tesla videos where the owner summons the vehicle as he's walking out the front door, or hits auto-park... and walks away.

Re: Tesla confirms “Autopilot” crash in Montana

#477

Post by abalone » Wed, Jul 13, 2016, 7:10 PM UTC

taf2 wrote: post #320

abalone wrote: post #285

Earlier quoted context omitted.

It bugs me how quick Tesla is to blame their drivers. That autopark crash, for example, was clearly the result of a software design flaw. Tesla pushed an update where to activate autopark all you had to do was double tap the park button and exit the vehicle. That's it. No other positive confirmation. No holding the key fob required. The vehicle would start moving without a driver inside. You'd think someone would hav…

its a bit harder than a double tap... You press and hold. Count to three. Now wait for the car to start blinking and make a shifting noise. Press either the front of the key or back of the key to move forward. Definitely not easy to engage so I can understand tesla view on this one... I do this every morning/night to pull my car out of the tiny garage since I can't get my door open inside the garage

False. Did you watch the video? It proves that you only had to double tap and exit.

Tesla pushed an update that enabled this as a "convenient" shortcut to the more elaborate, safer method you describe. In fact the video author praises it. Seems like a nice handy feature.. At first. Safety-critical design needs more care than this though.

Re: Tesla confirms “Autopilot” crash in Montana

#478

Post by FireBeyond » Wed, Jul 13, 2016, 7:12 PM UTC

lsaferite wrote: post #318

BillinghamJ wrote: post #241

Earlier quoted context omitted.

Then it should gradually and slowly come to a stop.

Stopping in 'a winding canyon with no shoulder' seems like a colossally bad idea. Mind you, trusting the badly named autopilot in this situation seems like an even worse idea. Especially if you consider the recent death that EVER Tesla drive must know about at this point. To then overconfidently trust the AP in an already dangerous environment for a human drive seems crazy.

Right, as you say, the only worse thing than coming to a stop is saying "Screw it, might as well give it a try and keep going!".

Re: Tesla confirms “Autopilot” crash in Montana

#479

Post by Vraxx » Wed, Jul 13, 2016, 7:13 PM UTC

foolfoolz wrote: post #429

TetOn wrote: post #411

Earlier quoted context omitted.

I just want to see "the vehicle again alerted the driver to put his hands on the wheel" followed by "and Autopilot then pulled to the side of the road, turned on the hazard lights, and required the driver to take over."

so far we have been lucky the accidents have only impacted the driver. is it going to take the first accident where an autopilot car kills another innocent driver before people see the problem?

Assuming we stick with the same conditions as in this case, I'd call an innocent life taken due to a driver disregarding all warnings from his system negligence. That also fits with what this driver was ticketed with which was reckless endangerment. We don't even get heads up display warnings about not driving into oncoming traffic, but nobody is going to blame a dumb-car for the actions of its driver. I would say that inaction in this hypothetical case would be the driver's crime.

Re: Tesla confirms “Autopilot” crash in Montana

#480

Post by azernik » Wed, Jul 13, 2016, 7:40 PM UTC

haikuginger wrote: post #372

azernik wrote: post #280

Earlier quoted context omitted.

Not sure exactly to what you're referring, but if it involves hacking your Tesla to re-enable it, a) that will vastly reduce the level of usage, and b) the OTA update could very well completely remove the code from storage, making the job of hacking a much more involved process than just enabling a flag.

Sony is settling a class-action suit with people who bought PS3s while relying on the "Other OS" feature. So, it's not about technical ability to re-enable the Autopilot feature; it's about being liable to their customers who relied on the feature being present when they shelled out $70k+.

Ah. Indeed. I suspect that if they do disable this, though, it'll be my NTSB mandate, so they won't have much of a choice - similar to the VW thing. They might then have to deal with the resulting lawsuits, but I can see many scenarios where they don't have a chance to avoid them. Tesla did, after all, sell a product with an advertised feature that isn't ready for prime-time.