Another Tesla on autopilot steers towards a barrier

Yes, they do that, and they rarely get that wrong.

Also, the better ones have inertial backup & flux gate for when GPS is unavailable. Inertial backup could also help to detect the signatures of lane changes and turns.

The scariest bit for me; in the comment's it's claimed that it has only done this for the last two software updates. Currently the worst I have to worry about when getting an OTA update is that it may make my phone less stable. Apparently in the future it may also influence precisely how my car tries to kill me.

> It's not possible to engineer an autonomous system that never fails, but it is possible to engineer one in such a way that it never fails to detect that it has failed.

This is a very good point: just like a human driver should slow down if they can't observe the road ahead well enough, an AI should slow down when it's not confident enough of its surroundings. This is probably very difficult to do, and I'm skeptical about your claim that an AI can be engineered to always detect its own failures. Further, I naively believe that Tesla is already doing a lot to detect conflicting inputs and other kinds of failures. Maybe being careful enough would prevent autopilot from working at all, so compromises have been made?

I probably don't understand AIs well enough to understand how they can be engineered to (almost) always recognize their own failures. But if a simple explanation exists, I'd love to hear it.

The whole problem seems to me that to get a 90% solution for a self driving vehicle is now no longer hard. The remaining 10% is super hard and the degree to which the present day solutions tend to get these wrong is worrisome enough that if the quality doesn't jump up very quickly the whole self driving car thing could end up in an AI winter all of its own making. And that would be a real loss.

This is not a field where 'move fast and break stuff' is a mantra worth anything at all, the stuff that is broken are people's lives and as such you'd expect a far more conservative attitude, akin to what we see in Aerospace and medical equipment software development practices.

But instead barely tested software gets released via over-the-air updates and it's anybody's guess what you'll be running the next time you turn the key.

I agree with you that the software should have been able to detect that something was wrong either way, either it was already halfway in the wrong lane or it was heading to be halfway in the wrong lane, a situation that should not have passed without the car at least alerting the driver.

And from what we have seen in the video it just gave one inferred situation priority over the rest and that particular one happened to be dead wrong about it's interpretation of the model.

I use Autopilot every day. I think it's amazing and borderline life-changing if you are like me and dread the monotony of a daily commute. I don't think most people realize just how mentally fatiguing the act of driving (constantly making all sorts of micro adjustments) is until they use Autopilot for a while and see the difference for themselves. I certainly didn't. I can now drive for many hours and feel just as alert as when I first got into the car. I was never able to do that before Autopilot.

Yes, you still need to pay attention. It is hard for me to believe that any Autopilot user doesn't know this because you learn it by experience almost immediately. People text and drive all the time in manual cars, but for some reason when they do it in a Tesla, we declare that Autopilot lulled them into it.

While I agree that you need to be ready to grab the wheel or hit the brake on short notice, I disagree about what that means. There is a big difference between having to be ready to do those things and actually having to do them every few seconds. This difference wasn't intuitive to me, but in practice I've found it to be extremely mentally liberating and true beyond question.

I've also found that Autopilot makes it easier to take in all of your surroundings and drive defensively against things you otherwise wouldn't see. One thing that has struck me, as I now see more drivers than just the one in front of me, is how many people are distracted while driving. If I glance continually at an arbitrary driver on my way to work, there is a greater likelihood than not that within 10 seconds they'll look at a phone. That is terrifying to me, but it is also good information to have as I drive -- I am now able to drive defensively against drivers around me, not just the one in front of me.

I've also found I'm more able to think or listen to music or podcasts than I was before Autopilot. I could never get much out of technical audiobooks, for instance, while driving manually. But Autopilot has changed that. I hesitate to say this because I worry that I will give the impression that I am focusing less on the road, but I don't think that's what's happening. My mental abilities feel much higher when I am not constantly turning a wheel or adjusting a pedal. I'm listening to music, podcasts, or audiobooks either way -- I just get a lot more out of them with Autopilot. I think it goes back to the lack of mental fatigue.

Whatever you make of my experience, I urge you to try it on a long drive if you ever get an opportunity. I have put over 60k miles on Autopilot, I have taken over on demand hundreds if not thousands of times, and I've never had a close call that was Autopilot's fault.