One way to help might be to disable AP following an update for X days or weeks. Let AP run in a disabled simulation mode while you drive your normal routes. If it determines your course of action in any situation breaks the threshold of what it was thinking it should do next, auto-report video feed to Tesla, and prevent AP from being used in a X-mile radius around that area.
Another Tesla on autopilot steers towards a barrier
311–320 of 481 posts
Re: Another Tesla on autopilot steers towards a barrier
#312Does anyone know how software updates are tested for such autonomous driving software ? Is there a possibility for this driver's video to be added to a regression suite and tested, or can they take the data from his car and build a regression test based on it ?
I sure hope they do have a regression suite with thousands of real world sensor-data samples in some normalized format made amenable to unit testing. Assuming such a normalized format is possible to generate, I expect these samples to be open-sourced and standardized. That would give the NTSB a systematized certification procedure: just run the new version of the software on all the known test cases before allowing it to be released.
Still, I suspect that may not be enough - a new weird road condition or construction site can be created at any time. This is why I believe self-driving companies would be better off spending their budget upgrading and certifying specific routes as safe for automation. Certified routes could be subject to special construction protocols and regular road quality audits that ensure automated cars won't run into a non-automatable condition on those routes. It's also easy to verify and confirm that a car works on all certified routes, rather than trying to test the entire North American road network.
Re: Another Tesla on autopilot steers towards a barrier
#313Here is my armchair diagnosis: right before the car veers towards the barrier it drives through a stretch of road where the only visible lane marker is on the left. Then the right lane marker comes into view at about the point where the lane starts to widen out for the lane split. The lines that will become the right and left lane markers of the split left and right lanes respectively are right next to the van in fro…
> It's not possible to engineer an autonomous system that never fails, but it is possible to engineer one in such a way that it never fails to detect that it has failed.
That is an extremely surprising result. How is that possible? Are you really claiming that any control system can be engineered to detect that is has failed in any possible manner? What's an example of actual real-world system like that?
Re: Another Tesla on autopilot steers towards a barrier
#314Here is my armchair diagnosis: right before the car veers towards the barrier it drives through a stretch of road where the only visible lane marker is on the left. Then the right lane marker comes into view at about the point where the lane starts to widen out for the lane split. The lines that will become the right and left lane markers of the split left and right lanes respectively are right next to the van in fro…
In other words an old ambiguous parsing problem. It might be fixable in software. I'm a bit annoyed at Tesla for over reliance over painted lines. They fade, they can be covered, be outdated .. That old fail video of a SDV stuck inside a circle is not funny anymore.
> SDV stuck inside a circle
Off topic: I never understood why this video was discussed that much, especially in order to blame SDVs. It's an example of pointless road markings and a perfectly behaving vehicle. It's like driving into a one way street that turns out to have no exit. The driver can't be blamed.
Re: Another Tesla on autopilot steers towards a barrier
#315Earlier quoted context omitted.
If autonomous systems reach the point where they are measurably safer than human drivers, there's an obvious benefit to using them. Again it's an if, but lots of people drive long distances and don't enjoy it. An autonomous vehicle would relieve them of the driving task. It would also add a transport option for people that are unable to drive long distances. It's likely to reduce the cost of taxi like services, as dr…
No, I'm with you here. Just thinking through the other side of some of these. Lost jobs. Not sure "safety" is a big enough issue here. On a larger scale, I'm significantly concerned that society (thru tech) is becoming increasingly optimized for comfort and safety. That bothers me. It removes so much of what is good about life. I don't want guaranteed life until 100. I don't want comfort all the time; that stunts gro…
Losing truck driving jobs isn't removing what is good about life. Neither is too much safety. The people whose lives are being saved by technology aren't worrying about having too much safety. Only those who are privileged and don't have to worry about anything, realistically.
Re: Another Tesla on autopilot steers towards a barrier
#316Here is my armchair diagnosis: right before the car veers towards the barrier it drives through a stretch of road where the only visible lane marker is on the left. Then the right lane marker comes into view at about the point where the lane starts to widen out for the lane split. The lines that will become the right and left lane markers of the split left and right lanes respectively are right next to the van in fro…
I totally agree with your analysis. What is more worrying is that even following the rest of the traffic would have solved the problem. If you are the only car doing something then there is a good chance you are doing something wrong. Also: the lane markings are confusing but GPS and inertial readings should have clearly indicated that that is not a lane. If two systems give conflicting data some kind of alarm should…
Pretty sure I heard an audible warning in the vid.
Re: Another Tesla on autopilot steers towards a barrier
#317Earlier quoted context omitted.
The average driver will probably never actually experience abs. It's a good failsafe that is rarely employed...if it is, you definitely realize it
I have ABS kick on around once-twice a season. I mean, sure, if you live somewhere it doesn't snow you won't even experience ABS, but if you live in a climate where it snows you definitely will. That being said, it's always been while stopping for a light, not in a case when I had to swerve, so its never prevented a crash for me, but it really is comforting to have it.
Around Houston I was surprised to see many warning about ice on the roads. Aparently even in Texas on a cold night a road especially on bridges can freeze resulting in black ice.
Re: Another Tesla on autopilot steers towards a barrier
#318I wonder if Tesla has a way to visualize how disruptive a change to Autopilot. They've supposedly got all this driving data lying around. Would they be able to apply the updated algorithms to that data and quantify how many decisions would be affected, and by how much? i.e. 30% of past decisions would have been changed with the new algorithm, 5% changed by an angle of over 2 degrees. On a similar note, shouldn't cust…
The problem is you can't simulate the video that happens after your new decision. So this steering update would take you 0.3 degrees to the right. So what? Well those 0.3 degrees might change the angle of the line which influences the car to steer another 0.3 degrees, etc. But without that followup video (which you can't simulate from a 2D recording) you don't know how it would react to the change in environment. The…
Following the branch down is obviously impossible, but simply detecting that there are new branches--perhaps a greater than expected number--could be useful information. For both the engineers and the operaters.
Re: Another Tesla on autopilot steers towards a barrier
#319Earlier quoted context omitted.
A system that can sense when it is in a failure mode is analogous to a “detector” in the Neyman-Pearson sense. A detector that says it is OK when it is actually in a failure condition is said to have a missed detection (MD). OTOH, if you say you’ve failed when you haven’t, that’s a false alarm (FA). In general, there is a trade-off between MD and FA. You can drive MD probability to near-zero, but typically at a cost…
You've done a great job of describing the problem. It is manifestly possible to drive missed detection rates very close to zero without too many false alarms because humans are capable of driving safely.
Ah, yes... you have convinced me - though humans can apply generalized intelligence to the problem, which I imagine is particularly useful in lots of special cases.