Live data from Hacker News

Another Tesla on autopilot steers towards a barrier

reddit.com

331–340 of 481 posts

Re: Another Tesla on autopilot steers towards a barrier

#331
post #12
post #2

This comment from OP (beastpilot) is pretty frightening: > Yep, works for 6 months with zero issues. Then one Friday night you get an update. Everything works that weekend, and on your way to work on Monday. Then, 18 minutes into your commute home, it drives straight at a barrier at 60 MPH. > It's important to remember that it's not like you got the update 5 minutes before this happened. Even worse, you may not know…

> Very glad the driver had 100% of their attention on the road at that moment. Yes, and 100% is critical. That required pretty quick and smooth reactions. The car started to follow the right-hand lane, then suddenly re-centered on the barrier. The driver had about a second to see that and correct. That's just the sort of situation where even a slightly inattentive driver might panic and overcorrect, which could cause…

From the OP's comments on Reddit - he said that it happens about 90% of the time on that stretch and that he has more footage of the same exact incident.

He was likely prepared for it, which kinda makes it even scarier in a way. An inattentive driver would have totally botched this.

Re: Another Tesla on autopilot steers towards a barrier

#332
post #36

AV driving seems to be ripe for disruption

Any 16 year old can already "disrupt" the industry just by getting a license and driving. Humans are and will continue to be far more capable, rational, and economical drivers for the foreseeable future.

Sounds similar to tech bros who want to radically "disrupt" carbon sequestration. Here's an actually proven idea: plant a tree.

Re: Another Tesla on autopilot steers towards a barrier

#333
I would argue that these incidents indicate a need for aerospace levels of quality assurance.

With the right tooling this does not need to be expensive.

I hope that incidents like this encourage people to develop open source tooling to support safer software and higher levels of quality.

We can see how lives might depend on it.

Re: Another Tesla on autopilot steers towards a barrier

#334

Earlier quoted context omitted.

Most modern cars are drive-by-wire, entirely relying on computers and the software. And their software is huge blobs of proprietary compiled code. It's guaranteed to have bugs.

Probably not that huge, really. Embedded systems on most vehicles are fairly minimal and reliable, at least compared to modern consumer operating systems and (apparently) Tesla's systems.

Car software is huge.

https://cdn-images-1.medium.com/max/2000/1*cjVl6u_s34Mqjcn68...

Re: Another Tesla on autopilot steers towards a barrier

#335

Earlier quoted context omitted.

Personally I hate cruise control. I prefer to be actively engaged when driving. Either I am driving 100% or I am not at all, there is absolutely no middle ground in my opinion

Cruise control is great where there's long stretches of open road, light traffic (LOS A or B [1]), and little need for frequent braking. The star usecase is to set cruise to be very near the speed limit, such that after acceleration events like overtaking, you coast back to highway speed. It's a low-effort way of ensuring that one will be compliant with speed laws most of the time, yet maintaining a steady pace. I to…

> if it gets disengaged for any reason then you can easily overrun into the car ahead

Are there cars that have ACC and don't have AEB (Automatic Emergency Braking)?

Re: Another Tesla on autopilot steers towards a barrier

#336
post #122

Here is my armchair diagnosis: right before the car veers towards the barrier it drives through a stretch of road where the only visible lane marker is on the left. Then the right lane marker comes into view at about the point where the lane starts to widen out for the lane split. The lines that will become the right and left lane markers of the split left and right lanes respectively are right next to the van in fro…

> This is actually the most serious aspect of all of these crashes: the system does not seem to be aware when it is getting things wrong.

That’s exactly my experience as a driver:

You learn to anticipate that the ‘autopilot’ will disengage or otherwise fail. I have been good enough at this, obviously, but it is sometimes frightening how close you get to a dangerous situation …

Re: Another Tesla on autopilot steers towards a barrier

#337

Earlier quoted context omitted.

When I think of self-driving car technology, I think of all of these inputs (maps, cameras, GPS, etc.) being fed into a world model from which inferences are drawn.[1] Here, the car could have known from cameras and maps that there was a lane split, and should have been able to detect that the line following was giving a conflicting inference compared to those other sources. The question is: does Tesla actually do th…

The whole problem seems to me that to get a 90% solution for a self driving vehicle is now no longer hard. The remaining 10% is super hard and the degree to which the present day solutions tend to get these wrong is worrisome enough that if the quality doesn't jump up very quickly the whole self driving car thing could end up in an AI winter all of its own making. And that would be a real loss. This is not a field wh…

The saying in those fields is "the rules are written in blood". The Therac-25 (a medical device) is taught in practically every CS Ethics class for having changed software development practices for such things.

The cynic in me thinks self-driving will require the same, the rules and conservative practices will only come after a bunch of highly publicized, avoidable failures that leave their own trail of blood.

Re: Another Tesla on autopilot steers towards a barrier

#338
Perfect example of a system doing what it's supposed to, but that's not the right thing. I don't believe it was headed toward the center barrier - I believe it was neatly avoiding it. Here's what I think was happening:

The car did an admirable job trying to stay in that left lane. Each time it saw a red and white diagonal lined barrier, it gracefully edged a comfortable distance away.

The problem is, most humans catch on, "Gee, three scary-looking barriers in a row! Maybe they're trying to tell me something. "Maybe this road is closed."

But I assume the car's system perceived each one as in individual, isolated obstacle to be handled without making inferences. And it did that much well, and when it realized the third or 4th barrier was impassable, it had to cross the other side of the yellow barrier and it did that well, too.

Re: Another Tesla on autopilot steers towards a barrier

#339

Earlier quoted context omitted.

Yes, they do that, and they rarely get that wrong. Also, the better ones have inertial backup & flux gate for when GPS is unavailable. Inertial backup could also help to detect the signatures of lane changes and turns.

Try driving near a place with a lot of roads that are close together, like an airport. You’ll quickly see that snapping doesn’t really work, especially if you deviate from the route the GPS is trying to take you on.

I do this all the time and it works remarkably well. Amsterdam and the roads around it are pretty dense, up to 6 lanes each way in quite a few places with all kinds of flyovers and multi-lane exits.

Re: Another Tesla on autopilot steers towards a barrier

#340
An honest question. Are any of these autonomous driving systems open source or peer reviewed in any way? If not, isn't it really weird that we are talking about regulations even though the underlying technology is not even peer reviewed? How do we know (mathematical proofs, etc.) that a self-driving car manufacturer has done a good enough job if all of that is proprietary?
Post reply on HN