If your design fails one user, blame the user. If it keeps failing, blame the design.
Tesla confirms “Autopilot” crash in Montana
251–260 of 530 posts
Re: Tesla confirms “Autopilot” crash in Montana
#252Nobody reads manuals and follows instructions . I feel Tesla is going to learn this lesson that hard way. From the article: "We specifically advise against its use at high speeds on undivided roads," it said. Tesla states clearly in its owner's manual that drivers should stay alert and keep their hands on the wheel to avoid accidents when the Autopilot feature is engaged.
People have been trained to recognize the discrepancy between a product's by-the-book usage (which is often impractically stringent, as a CYA for the company) and its real world usage. Tesla could easily have the "autopilot" turn off if no hands are detected on the wheel for, say, 10 seconds. Putting a blurb in the instruction manual about keeping your hands on the wheel but then allowing users to use it hands-free i…
I don't know about most people but there are some things (like cars), that I treat as super-reliable machines that will protect me even if I do some dumb things. And most car systems are made like that. They assume that the driver is going to fuck up at some point and they try to handle that. It seems Tesla's auto-drive feature is not like that. It's like when Chrome has a silly bug, I just accept it as it is and try to work my way around it.
Re: Tesla confirms “Autopilot” crash in Montana
#253Earlier quoted context omitted.
> obstacles might actually be net-safer to not swerve around In the UK its illegal to swerve for these things. In the sense that if you subsequently cause an accident for swerving to avoid a cat, you are at fault.
Wait, it's illegal to swerve for a moose (TIL: moose are called elk in Europe, American elk are totally different) in the UK? Hitting a moose is a fast route to the grave.
I guess that they can let the courts set precedence when the UK has it's first moose-on-road-based collision...
Re: Tesla confirms “Autopilot” crash in Montana
#254Earlier quoted context omitted.
Such a value system would be a nightmarish thing so I hope that it doesn't exist yet. Just imagine your car calculating that it should let you die in order to save a group of school children... except that the school children were in fact a bunch of sheep that the system mis-identified. The danger of AI isn't in paperclip optimizers (at least for now), it's in humans over-relying on machine learning systems.
I think such a value system is fine in as much as it preserves the safety of the driver. I'd like my car to make value judgements about what's safer - braking and colliding with an object or swerving to avoid it. I'd rather it brakes and collides with a child than swerve into oncoming traffic, but I'd rather it swerved into oncoming traffic to avoid a truck that would otherwise T-bone me. As long as the system only a…
The moral choice is the opposite direction: the car should prefer to kill the occupant than a bystander. The occupant is the reason why the car is in use; if it fails, the penalty for that failure should be on the occupant, not bystanders.
Re: Tesla confirms “Autopilot” crash in Montana
#255With three accidents on "autopilot" in a short period, the defects in Tesla's design are becoming clear. 1. Tesla's "hands on wheel" enforcement is much weaker than their competitors. BMW, Volvo, and Mercedes have similar systems, but after a few seconds of hands-off-wheel, the vehicle will start to slow. Tesla allows minutes of hands-off time; one customer reports driving 50 miles without touching the wheel. Tesla i…
I agree their sensor suite needs improvement. I hope they are adding Lidars and/or radars before they start shipping large volumes. A recall of large numbers of cars would kill Tesla
Re: Tesla confirms “Autopilot” crash in Montana
#256Earlier quoted context omitted.
There are crop dusters that fly close to the ground and obstacles - do you want to get in a manual one or can I code up a computer controlled one and put you in it? What you really mean is you want to see safety data and a track record. As for the actual planes and auto-pilots, the real point is the kinds of engagements and disengagements that work in the air don't necessarily work on the ground and being so close to…
Close, but not constantly centimeters close. It's a whole different league, and not one I'm comfortable trusting a human with.
Humans write the software that powers computers FYI.
Re: Tesla confirms “Autopilot” crash in Montana
#257With three accidents on "autopilot" in a short period, the defects in Tesla's design are becoming clear. 1. Tesla's "hands on wheel" enforcement is much weaker than their competitors. BMW, Volvo, and Mercedes have similar systems, but after a few seconds of hands-off-wheel, the vehicle will start to slow. Tesla allows minutes of hands-off time; one customer reports driving 50 miles without touching the wheel. Tesla i…
> It takes seconds, not milliseconds, for the driver to recover situational awareness and take over. The really depressing thing is that this is first-year material for one of the most popular tertiary courses around: psychology. It is simply gobsmacking to think that Tesla thinks a human can retarget their attention so quickly. It's the kind of thing you'd give to first-year students and say "what is wrong with this…
> It is simply gobsmacking to think that Tesla thinks a human can retarget their attention so quickly
They don't actually believe it, but they have to make the facts fit the narrative of autonomous driving being right there.
Re: Tesla confirms “Autopilot” crash in Montana
#258Great, more data points.
People down vote this, but it does a great job of highlighting that human deaths and injuries are in fact being treated purely as data points.
Re: Tesla confirms “Autopilot” crash in Montana
#259Earlier quoted context omitted.
> So the transparency itself is mandatory Then why on earth is Tesla being opaque and seemingly passing the buck with their PR?! I don't get it.
The requirement does not seem to be there for driver-assist vehicles. Also, it's a state-level requirement. Looks like it's just in CA for now. The accidents detailed in the Google reports are all from California. And, Tesla may be unwilling to start reporting driver-assist accident rates to the public unless the other driver-assist car companies are forced to do it too. I'll make an addendum to my above comment.
> The accidents detailed in the Google reports are all from California.
You could have opened the first report[1] on that page and notice this is wrong.
> June 6, 2016: A Google prototype autonomous vehicle (Google AV) was traveling southbound on Berkman Dr. in Austin, TX in autonomous mode and was involved in a minor collision north of E 51st St. The other vehicle was approaching the Google AV from behind in an adjacent right turn-only lane. The other vehicle then crossed into the lane occupied by the Google AV and made slight contact with the side of our vehicle. The Google AV sustained a small scrape to the front right fender and the other vehicle had a scrape on its left rear quarter panel. There were no injuries reported at the scene.
[1] https://static.googleusercontent.com/media/www.google.com/en...
Re: Tesla confirms “Autopilot” crash in Montana
#260Earlier quoted context omitted.
> What I don't understand is why Tesla doesn't whitelist roads where this feature can be used, since the manual sets very specific conditions I think they want the data of drivers attempting to drive with autopilot on these roads. Tesla does, I've read, restrict how fast autopilot can drive on unapproved roads, so it appears they already have them mapped out. They're using customers as guinea pigs for their tech.
Why not have Autopilot running and monitoring the sensors to learn the road, but not have it actually take control of the vehicle? Even if the driver is manually in control the Autopilot system could still be learning.
Tesla may have begun by doing that. Recall that last October autopilot was introduced "over the air", so, customers who already had vehicles could use it immediately. Presumably, Tesla was collecting data up until that point.
I imagine Tesla is generating more sales with a usable autopilot. It generated quite a bit of media hype and they are having trouble keeping up with production demands.
So, Tesla chose to introduce this feature to increase sales and get more data on how drivers interact with the system. It seems excessively risky to me. I'm not building a self-driving system myself, nor running a huge company, so I can't say much about what it takes to do that. I can say I would not like to drive on the road next to a vehicle that can unexpectedly make the wheel turn, forcing its driver to be corrective at a moment's notice.