Live data from Hacker News

Why collision avoidance is harder for an AI-based system

medium.com

31–40 of 145 posts

Re: Why collision avoidance is harder for an AI-based system

#31
post #25
post #13

Earlier quoted context omitted.

They released inward-facing video of the safety driver shortly after the incident. The driver was staring at her phone in her lap with only occasional glances at the road every 5 seconds or so. I would call that pretty negligent.

That certainly doesn’t sound good. Is there footage of the driver at the time of the collision?

Here after about 10 seconds: https://www.ft.com/video/414f079b-a63d-4b0f-8dcf-b14f66e5ab2...

Re: Why collision avoidance is harder for an AI-based system

#32
post #2

> AI is not preprogrammed to monitor a known input from a sensor to take a predefined action. I guess one of my outstanding questions, which reading this only confirmed, is why this is the case? I mean, humans are pretty good examples of intelligence. And yet we still have and use these anti-collision systems. Because, in the end, when wrong decisions are made these systems save lives. Why would AI-driven vehicles no…

Author here :) I cannot answer for Uber, but the grand reason why we cannot put deterministic parts into the AI is the same as for humans - we do not control the internals of it, the complexity is too high. Although we can build systems that are composed of both AI and deterministic collision avoidance (as seen on new cars today). It is what I also suggested in the article.

Re: Why collision avoidance is harder for an AI-based system

#33

Has anyone been able to even remotely explain why the LIDAR system wasn't going nuts? I saw the "it was dark" nonsense, but I assume this vehicle had laser and IR right? The camera footage was released, I'd like to see the lidar representation.

What makes you think it wasn't? You're making some assumptions about which of the many systems failed, I think.

Re: Why collision avoidance is harder for an AI-based system

#34

Has anyone been able to even remotely explain why the LIDAR system wasn't going nuts? I saw the "it was dark" nonsense, but I assume this vehicle had laser and IR right? The camera footage was released, I'd like to see the lidar representation.

Author here :) LIDAR itself is just a sensor, it does not process the data nor does it output a directly usable image like a camera more or less does.

LIDAR in this case is a rotating laser and while it scans, the vehicle moves (imagine moving a paper when a copier scans it). All processing is done later, first to construct an image and then to understand and use it. Part of why I wrote the piece was to explain how things can go wrong even if your LIDAR works fine.

Re: Why collision avoidance is harder for an AI-based system

#36
post #24

I'd like to see a law that says: to obtain certification to operate a self-driving car on a roadway, an organization must agree to: 1) Conform to a standard set of protocols for how sensors provide data to a self-driving software system. 2) Log data in a form that could be submitted to any conforming self-driving software system, to obtain results from that system reporting what the system would do given these inputs…

Perhaps some sort of logging standard that is enforced across all systems? A black box of some sort?

Yes, exactly, but with the additional factors I mentioned.

Re: Why collision avoidance is harder for an AI-based system

#37
Why isn't car driving AI put to the same rigour of testing and approval as that of airplanes? There are far more people on the road than in the sky, and yet, there's formal verification for airplane autopilot code but an accident like this is supposed to be "you couldn't have prevented it either?"

If no one could prevent such a thing, this AI should never drive a car!

Re: Why collision avoidance is harder for an AI-based system

#38
post #13
post #9

Earlier quoted context omitted.

Why? If it is not proven the driver was negligent why fire this individual? Presumably this individual has a lot of experience and domain knowledge in testing self driving cars so replacing them with someone else may not be an improvement. If they did make a mistake that got someone killed that could change things but I would hope we wait to find out if the driver was actually at fault.

They released inward-facing video of the safety driver shortly after the incident. The driver was staring at her phone in her lap with only occasional glances at the road every 5 seconds or so. I would call that pretty negligent.

It was more likely the uber iPad data logging app, which they've allowed to be using whilst moving.

Can't think it would help with driver's night vision.

Re: Why collision avoidance is harder for an AI-based system

#39
post #2

> AI is not preprogrammed to monitor a known input from a sensor to take a predefined action. I guess one of my outstanding questions, which reading this only confirmed, is why this is the case? I mean, humans are pretty good examples of intelligence. And yet we still have and use these anti-collision systems. Because, in the end, when wrong decisions are made these systems save lives. Why would AI-driven vehicles no…

The false positive rate of this system could be very noisy. At what point should you brake, even if the AI says not to? When you might collide with a lidar point? You want to collide with lidar points all the time (the ground beneath the vehicle, for example, or any reflective noise). The AI is there to tell you what lidar points are actual obstacles that you can't collide with, otherwise it's not really meaningful.
Post reply on HN