Another Tesla on autopilot steers towards a barrier • Hacker News Board

Re: Another Tesla on autopilot steers towards a barrier

#361

Post by [deleted] » Fri, Apr 06, 2018, 10:00 PM UTC

microtherion wrote: post #154

ams6110 wrote: post #12

Earlier quoted context omitted.

> Very glad the driver had 100% of their attention on the road at that moment. Yes, and 100% is critical. That required pretty quick and smooth reactions. The car started to follow the right-hand lane, then suddenly re-centered on the barrier. The driver had about a second to see that and correct. That's just the sort of situation where even a slightly inattentive driver might panic and overcorrect, which could cause…

> The driver had about a second to see that and correct. And if there had been a crash, Tesla probably would have said that the driver had an unobstructed view of the barrier for several seconds before impact (conveniently omitting at what point AP started to swerve).

[deleted]

Re: Another Tesla on autopilot steers towards a barrier

#362

Post by pedrocr » Fri, Apr 06, 2018, 10:02 PM UTC

wcunning wrote: post #236

hyperbovine wrote: post #159

Earlier quoted context omitted.

Ahem. That is purely semantics -- ABS makes the car perform more like what you are conditioned to expect (under normal driving conditions). That's the whole success of ABS; it saves lives precisely because it conveys the illusion of you still being 100% under direct control when in fact you would be careening out of control otherwise.

One of the things I've heard mentioned by professional driving instructors, without a good citation, is that many more accidents could be avoided if people trusted ABS more. Specifically, when you're braking hard/panic braking and the ABS kicks it, the shock of the rapid firing against your foot causes you to back off pressure on the brake pedal. I was specifically instructed to stand on it anyway to see what the car…

That's why cars have Brake Assist (BA) now that will break more than you're actually requesting with the pedal if you break suddenly.

https://en.wikipedia.org/wiki/Emergency_brake_assist

Re: Another Tesla on autopilot steers towards a barrier

#363

Post by sul4bh » Fri, Apr 06, 2018, 10:05 PM UTC

  Previous autopilot versions didn't do this. 10.4 and 12 do this 90% of the time. I didn't post it until I got .12 and it did it a few times in case it was a bug in 10.4.

It looks like software updates will soon won't just break your workflow. They might break your spine or your head or take away your life entirely.

Re: Another Tesla on autopilot steers towards a barrier

#364

Post by jacquesm » Fri, Apr 06, 2018, 10:05 PM UTC

volkl48 wrote: post #337

jacquesm wrote: post #167

Earlier quoted context omitted.

The whole problem seems to me that to get a 90% solution for a self driving vehicle is now no longer hard. The remaining 10% is super hard and the degree to which the present day solutions tend to get these wrong is worrisome enough that if the quality doesn't jump up very quickly the whole self driving car thing could end up in an AI winter all of its own making. And that would be a real loss. This is not a field wh…

The saying in those fields is "the rules are written in blood". The Therac-25 (a medical device) is taught in practically every CS Ethics class for having changed software development practices for such things. The cynic in me thinks self-driving will require the same, the rules and conservative practices will only come after a bunch of highly publicized, avoidable failures that leave their own trail of blood.

I wonder how the public would respond if the person killed in that Uber accident a while ago was a kid on a bicycle and how the response will be when/if a Tesla on autopilot plows into a schoolbus (a situation they seem to have problems with is stationary objects partially in a lane).

Re: Another Tesla on autopilot steers towards a barrier

#365

Post by fwip » Fri, Apr 06, 2018, 10:06 PM UTC

Tloewald wrote: post #271

lisper wrote: post #232

Earlier quoted context omitted.

> This seems to me to be a clearly incorrect (and self-contradictory) claim. I guess that means I was able to pull the wool over the eyes of all five members of my thesis committee because none of them thought so. > It entirely depends on your definition of failure. Well, yeah, of course. So? There is some subset of states of the world that you label "failure". The guarantee is not that the system never enters one of…

Arguing that it's possible to build an autonomous that is sometimes wrong but always knows when it's wrong seems like a stretch to me.

You can if you dial the specificity down to 0.

i.e: Always report "I'm wrong."

Re: Another Tesla on autopilot steers towards a barrier

#366

Post by Piskvorrr » Fri, Apr 06, 2018, 10:06 PM UTC

volkl48 wrote: post #337

jacquesm wrote: post #167

Earlier quoted context omitted.

The whole problem seems to me that to get a 90% solution for a self driving vehicle is now no longer hard. The remaining 10% is super hard and the degree to which the present day solutions tend to get these wrong is worrisome enough that if the quality doesn't jump up very quickly the whole self driving car thing could end up in an AI winter all of its own making. And that would be a real loss. This is not a field wh…

The saying in those fields is "the rules are written in blood". The Therac-25 (a medical device) is taught in practically every CS Ethics class for having changed software development practices for such things. The cynic in me thinks self-driving will require the same, the rules and conservative practices will only come after a bunch of highly publicized, avoidable failures that leave their own trail of blood.

If there's anything left to be conservative upon. Current climate of "drive fast and kill people" might wipe out the field via PR.

Re: Another Tesla on autopilot steers towards a barrier

#367

Post by Barrin92 » Fri, Apr 06, 2018, 10:07 PM UTC

tjwds wrote: post #2

This comment from OP (beastpilot) is pretty frightening: > Yep, works for 6 months with zero issues. Then one Friday night you get an update. Everything works that weekend, and on your way to work on Monday. Then, 18 minutes into your commute home, it drives straight at a barrier at 60 MPH. > It's important to remember that it's not like you got the update 5 minutes before this happened. Even worse, you may not know…

This is just about what I expected it to look like when software engineering safety standards are let loose on the physical world.

Re: Another Tesla on autopilot steers towards a barrier

#368

Post by sah2ed » Fri, Apr 06, 2018, 10:08 PM UTC

Cthulhu_ wrote: post #43

fma wrote: post #26

Earlier quoted context omitted.

I have always wondered why each company is developing their own software and algorithms... How will NTSB certify the software? Or will they? I mean, they certify car safety features now through crash tests, roll over tests, etc... Why is Tesla, or anyone manufacturer, allowed to modify vehicle safety features through OTA updates without recertification? I realize it adds more regulation...and I'm sure there can be a…

Patents and the ability to earn money by licensing out technology. That, or hoping they are the best and get more customers through it - I think that's currently the case for Tesla, selling the only commercially available semi-autonomously driving car ATM.

> I think that's currently the case for Tesla, selling the only commercially available semi-autonomously driving car ATM.

"I think that's currently the case for Tesla, selling the only commercially available semi-autonomous [electric] car ATM."

FTFY

Re: Another Tesla on autopilot steers towards a barrier

#369

Post by Piskvorrr » Fri, Apr 06, 2018, 10:12 PM UTC

danso wrote: post #184

jacquesm wrote: post #137

Earlier quoted context omitted.

I totally agree with your analysis. What is more worrying is that even following the rest of the traffic would have solved the problem. If you are the only car doing something then there is a good chance you are doing something wrong. Also: the lane markings are confusing but GPS and inertial readings should have clearly indicated that that is not a lane. If two systems give conflicting data some kind of alarm should…

> What is more worrying is that even following the rest of the traffic would have solved the problem. That's what makes this example bizarre to me. I had thought that AutoPilot's ideal situation was having a moving vehicle in front of it. For example, AP does not have the ability to react to traffic lights, but can kind of hack it by following the pace of the vehicle ahead of it (assuming the vehicle doesn't run a re…

That's a heck of an ass-u-mption. Running the red light as the n-th exponentially increases the probability you'll get into collision.

Re: Another Tesla on autopilot steers towards a barrier

#370

Post by swedishfish » Fri, Apr 06, 2018, 10:14 PM UTC

binaryjay wrote: post #359

lisper wrote: post #122

Here is my armchair diagnosis: right before the car veers towards the barrier it drives through a stretch of road where the only visible lane marker is on the left. Then the right lane marker comes into view at about the point where the lane starts to widen out for the lane split. The lines that will become the right and left lane markers of the split left and right lanes respectively are right next to the van in fro…

>Here is my armchair diagnosis: Odd how your "armchair diagnosis" matches perfectly with the top ranked comment in the reddit thread that was posted 1 day ago. courtlandreOwner 815 points 1 day ago It sees the white line on the left, the white line on the right and thinks its a big lane. Its trying to center the car in the lane.

The comment above is saying the split looks like a diamond lane marker indicating an HOV/electric car lane, so the car is centering itself on the diamond. That's a completely different idea from the Reddit comment about the car centering itself between the right and left white lines. My guess is that both inputs contributed to the car "thinking" it was following a lane.