Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Another Tesla on autopilot steers towards a barrier

reddit.com

Post reply on HN
Print view View original

451–460 of 481 posts

Re: Another Tesla on autopilot steers towards a barrier

#451

Post by freerobby »

Piskvorrr wrote: post #430
freerobby wrote: post #296

Earlier quoted context omitted.

Lidar isn't reliable in conditions that are less than ideal (rain, fog, snow, etc). If you build a car that is dependent on real-time lidar observations, it is worthless much of the time in some climates and some of the time in all climates. Most companies are pursuing a Lidar approach because dev_speed(Lidar approach) > dev_speed(camera approach). Tesla is pursuing a camera approach because max(camera approach) > ma…

In such conditions, cameras are not unreliable, they're downright useless. Heck, even people have trouble driving during snowfall. I have a feeling that this is a massive set of scenarios that aren't even edge cases, yet SDVs are completely unprepared for them. ("No snow in coastal California - it doesn't exist at all!")

I think you may be imagining what a human sees in a foggy camera image, rather than what a neural network can see in it. In fact, atmospheric obstructions degrade Lidar's signal much faster than a camera's.

If you check out NVidia's DriveNet demo from a couple years back, you'll see that they already have a vision-based NN that outperforms humans in fog and snow[1]. We can debate whether people should drive in those conditions at all, but today's expectations will be the baseline that SDVs are up against, and cameras are much better suited than Lidar to achieve that baseline.

[1] Start at 7:21 https://www.youtube.com/watch?v=HJ58dbd5g8g

Re: Another Tesla on autopilot steers towards a barrier

#452

Post by mlvljr »

ansible wrote: post #282
jacquesm wrote: post #167

Earlier quoted context omitted.

The whole problem seems to me that to get a 90% solution for a self driving vehicle is now no longer hard. The remaining 10% is super hard and the degree to which the present day solutions tend to get these wrong is worrisome enough that if the quality doesn't jump up very quickly the whole self driving car thing could end up in an AI winter all of its own making. And that would be a real loss. This is not a field wh…

I agree with all you've said. One of my concerns with self-driving systems is that they don't have a good model of the world they are operating in. Yes, sure, they build up 3-D models using multiple sensor types, and react to that accordingly, oftentimes with faster reflexes than a human. However, consider this situation with pedestrians A and B, while driving down a street, approaching an intersection. Pedestrian A,…

A may lose his balance and step back suddenly.

Re: Another Tesla on autopilot steers towards a barrier

#454

Post by andosa »

lisper wrote: post #449
andosa wrote: post #439

Earlier quoted context omitted.

> It's not possible to engineer an autonomous system that never fails, but it is possible to engineer one in such a way that it never fails to detect that it has failed. That sounds highly dubious. Here's a hypothetical scenario: there's a very drunk person on the sidewalk. As a human driver, you know he might act unexpectedly so you slow down and steer to the left. This will help you avoid a deadly collision as the…

An even more extreme example: suppose someone on the sidewalk suddenly whips out a bazooka and shoots it at you. Does your failure to anticipate this contingency count as a failure? "Failure" must be defined with respect to a particular model. If you're driving in the United States, you're probably not worried about bazookas, and being hit by one is not a failure, it's just shit happening, which it sometimes does. (B…

Way to dodge the question. And how did we get from always knowing when you're failed to "just drive very, very slow when", when dealing with situations that human drivers deal with all the time.

Let's not pretend that anticipating potentially dangerous behaviour from subtle clues is some once-in-a-lifetime corner case. People do this all the time when driving -- be it a drunk guy on the sidewalk, a small kid a tad bit too unstable when riding a bike by the roadside, kids playng catch nex to the road and not paying attention, etc etc. Understanding these situation is crucial in self driving if we want to beat the 1 fatality per 100M mile that we have with human drivers. For such scenarios, please explain how the AI can always know when it failed to anticipate a problem that a normal human driver can.

Re: Another Tesla on autopilot steers towards a barrier

#455

Post by lisper »

andosa wrote: post #454
lisper wrote: post #449

Earlier quoted context omitted.

An even more extreme example: suppose someone on the sidewalk suddenly whips out a bazooka and shoots it at you. Does your failure to anticipate this contingency count as a failure? "Failure" must be defined with respect to a particular model. If you're driving in the United States, you're probably not worried about bazookas, and being hit by one is not a failure, it's just shit happening, which it sometimes does. (B…

Way to dodge the question. And how did we get from always knowing when you're failed to "just drive very, very slow when", when dealing with situations that human drivers deal with all the time. Let's not pretend that anticipating potentially dangerous behaviour from subtle clues is some once-in-a-lifetime corner case. People do this all the time when driving -- be it a drunk guy on the sidewalk, a small kid a tad bi…

> how did we get from always knowing when you're failed to "just drive very, very slow when", when dealing with situations that human drivers deal with all the time

You raised this scenario:

> there's a very drunk person on the sidewalk. As a human driver, you know he might act unexpectedly so you slow down...

I was just responding to that.

> Let's not pretend that anticipating potentially dangerous behaviour from subtle clues is some once-in-a-lifetime corner case.

I never said it was. All I said was that "failure must be defined with respect to some model." If you really want to anticipate every contingency then you have to take into account some very unlikely possibilities, like bazookas or (to choose a slightly more plausible example) having someone hiding behind the parked car that you are driving past and jumping out just at the wrong moment.

The kind of "failure" that I'm talking about is not a failure to anticipate all possible contingencies, but a failure to act correctly given your design goals and the information you have at your disposal. Hitting someone who jumps out at you from behind a parked car, or failing to avoid a bazooka attack, may or may not be a failure depending on your design criteria. But the situation in the OP video was not a corner case. Steering into a static barrier at freeway speeds is just very clearly not the right answer under any reasonable design criteria for an autonomous vehicle.

My claim is simply that given a set of design criteria, you cannot in general build a system that never fails according to those criteria, but you can build a system that, if it fails, knows that it has failed. I further claim that this is useful because you can then put a layer on top of this failure-detection mechanism that can recover from point failures, and so increase the overall system reliability. If you really want to know the details, go read the thesis or the paper.

These are not particularly deep or revolutionary claims. If you think they are, then you haven't understood them. These are really just codifications of some engineering common-sense. Back in 1991, applying this common sense to autonomous robots was new. In 2018 it should be standard practice, but apparently it's not.

Re: Another Tesla on autopilot steers towards a barrier

#456

Post by thaumasiotes »

lisper wrote: post #448
gowld wrote: post #431

Earlier quoted context omitted.

What happens if all your sensors fail, including the one that senses the failure of your sensors? What happens if the power source disconnects?

Having all your sensors fail is actually a very easy case. Imagine if all of your sensors failed: suddenly you could not see, hear, feel, smell, or taste... do you think it would be hard to tell that something was wrong?

Having your sensors fail doesn't mean they're not providing data. It means they're not providing accurate data. In humans, we would call this hallucinating, and humans in fact cannot generally tell that they are hallucinating.

Re: Another Tesla on autopilot steers towards a barrier

#457

Post by lisper »

thaumasiotes wrote: post #456
lisper wrote: post #448

Earlier quoted context omitted.

Having all your sensors fail is actually a very easy case. Imagine if all of your sensors failed: suddenly you could not see, hear, feel, smell, or taste... do you think it would be hard to tell that something was wrong?

Having your sensors fail doesn't mean they're not providing data. It means they're not providing accurate data. In humans, we would call this hallucinating, and humans in fact cannot generally tell that they are hallucinating.

> Having your sensors fail doesn't mean they're not providing data.

That is one possible failure mode. But you're right, it's not the only one.

There is an extensive literature on how to detect and correct sensor errors resulting from all kinds of different failure modes.

Re: Another Tesla on autopilot steers towards a barrier

#458

Post by Tloewald »

lisper wrote: post #292
mturmon wrote: post #287

Earlier quoted context omitted.

A system that can sense when it is in a failure mode is analogous to a “detector” in the Neyman-Pearson sense. A detector that says it is OK when it is actually in a failure condition is said to have a missed detection (MD). OTOH, if you say you’ve failed when you haven’t, that’s a false alarm (FA). In general, there is a trade-off between MD and FA. You can drive MD probability to near-zero, but typically at a cost…

You've done a great job of describing the problem. It is manifestly possible to drive missed detection rates very close to zero without too many false alarms because humans are capable of driving safely.

So we can get within epsilon for an undefined delta because humans can do something, although not always.

Right, that whole claim about engineering a system that always knows when it’s not working sounds rock solid to me. After all, we can build a human, right?

Re: Another Tesla on autopilot steers towards a barrier

#459

Post by lisper »

Tloewald wrote: post #458
lisper wrote: post #292

Earlier quoted context omitted.

You've done a great job of describing the problem. It is manifestly possible to drive missed detection rates very close to zero without too many false alarms because humans are capable of driving safely.

So we can get within epsilon for an undefined delta because humans can do something, although not always. Right, that whole claim about engineering a system that always knows when it’s not working sounds rock solid to me. After all, we can build a human, right?

> we can build a human, right?

Not yet. But there's no reason to believe we won't be able to eventually.

Post reply on HN