Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Openpilot – open-source self-driving agent

github.com

Post reply on HN
Print view View original

81–90 of 114 posts

Re: Openpilot – open-source self-driving agent

#81

Post by jerkstate »

Bendingo wrote: post #51

This seems mind bogglingly dangerous. At least with commercial self-driving vehicles, there is a responsible legal entity (the car manufacturer). The thought of thousands of amateurs and enthusiasts releasing their hacked self-driving cars on the public roads is terrifying. The disclaimer "YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS" is nice, but I can't that stopping the mindset of "I've just t…

> At least with commercial self-driving vehicles, there is a responsible legal entity (the car manufacturer).

That’s not correct, the responsible legal entity in self-driving cases is likely to be the “backup” driver. For example, Uber was not found criminally liable for the Arizona accident a couple years back that killed a woman, but the driver could face charges of vehicular manslaughter: https://www.insurancejournal.com/news/west/2019/03/06/519721...

Uber has settled the civil case out of court with the woman’s family on the driver’s behalf, there is nothing they can do for the criminal case (besides provide representation for the driver)

Re: Openpilot – open-source self-driving agent

#82

Post by jerkstate »

taneq wrote: post #60
kyle_martin1 wrote: post #25

I work as a lead autonomous vehicle engineer. Here's a few thoughts: * This isn't really open-source. The most important part, the neural network perception pipeline, is closed sourced. Without the training data and the network architectures, the supporting code is not that useful. * This system is non-deterministic, which is a no-no in critical systems like autonomous vehicle. There's no guarantee that vehicle detec…

I'd love to try it out but I can't bring myself to trust (on any level) something that's essentially a mobile phone running a hacked-together mashup of C and Python. There's no redundancy, no independent components cross-checking each other, nothing as far as I'm aware to try and provide any given level of reliability. (Don't get me wrong, Python is great for research - I wouldn't regard it as highly reliable or good…

I’m not trying to convince you, but the a-ha moment for me was when I was watching George talk about reverse-engineering the firmware for part of the cruise control in his car, and after decompiling it, he found that there was no functionality for the manufacturer to verify that the firmware had been written properly when flashed by the manufacturer. So just because a big, expensive car company made something, doesn’t mean there is any redundancy or cross-checking.

By the way, the only cars you can install OpenPilot onto are cars that already have lane keeping assist (i.e. a camera that watches the lines) and adaptive cruise control (i.e. a millimeter wave radar that makes sure you don’t get too close to the car in front of you). So OpenPilot is just a vastly improved version of the software that those guys who didn’t write a firmware checksum routine wrote.

Re: Openpilot – open-source self-driving agent

#83

Post by TickleSteve »

a-dub wrote: post #45
a-dub wrote: post #44

Earlier quoted context omitted.

1) They actually open sourced the vision pipeline code a few months ago. The visiond binary has been replaced by source code. 2) Yeah. The proper way to do these things is with a real RTOS, but at the same time 20hz is pretty slow anyways.

I should also mention that the RT approach isn't the holy grail either for time sensitive robotic control systems. Read up on the Pathfinder/vxWorks priority inversion bug if you'd like to see how hard deadlines can sometimes cause more problems than they solve...

priority-inversion is well-known and easily coped with.

Hard real-time, deterministic code is simply a prerequisite for safety in these types of systems, you have to be able to reason about how your system will behave in worst-case situations.

Re: Openpilot – open-source self-driving agent

#84

Post by TickleSteve »

holoduke wrote: post #32
kyle_martin1 wrote: post #25

I work as a lead autonomous vehicle engineer. Here's a few thoughts: * This isn't really open-source. The most important part, the neural network perception pipeline, is closed sourced. Without the training data and the network architectures, the supporting code is not that useful. * This system is non-deterministic, which is a no-no in critical systems like autonomous vehicle. There's no guarantee that vehicle detec…

What do you mean it's non deterministic? Because it is using floating point operations? Can you elaborate?

Deterministic code would be running under a hard real-time kernel such as VxWorks or bare-metal and would be designed such that all timing has fixed upper-bounds.

This involves reasoning about the scheduling of tasks to ensure that in worst-case situations, your timing requirements are still being met.

All safety-critical systems would be deterministic, you simply would not get certified otherwise.

Note: Theres much more to safety-critical application design, but basically, you have to be able to prove the behaviour of you code under all circumstances, which typically means no garbage-collection, dynamic-allocation, etc.

Re: Openpilot – open-source self-driving agent

#85

Post by lordricky »

taneq wrote: post #60
kyle_martin1 wrote: post #25

I work as a lead autonomous vehicle engineer. Here's a few thoughts: * This isn't really open-source. The most important part, the neural network perception pipeline, is closed sourced. Without the training data and the network architectures, the supporting code is not that useful. * This system is non-deterministic, which is a no-no in critical systems like autonomous vehicle. There's no guarantee that vehicle detec…

I'd love to try it out but I can't bring myself to trust (on any level) something that's essentially a mobile phone running a hacked-together mashup of C and Python. There's no redundancy, no independent components cross-checking each other, nothing as far as I'm aware to try and provide any given level of reliability. (Don't get me wrong, Python is great for research - I wouldn't regard it as highly reliable or good…

With respect to Python and RTOS, this is a copy paste of what I posted few months ago on a similar thread, here on HN: ----

No safety relevant code is written in Python. All the safety relevant code runs real-time on a STM32 micro (inside the Panda), it's written in C and it's placed at the interface between the car and the EON. This code ensures the satisfaction of the 2 main safety principles that a Level 2 driver assistance system must have: 1- the driver needs to be able to easily disengage the system at any time; 2- the vehicle must not alter its trajectory too quickly for the driver to safely react. See https://github.com/commaai/openpilot/blob/devel/SAFETY.md

Among the processes that runs on the EON, you can find algorithms for perception, planning and controls. Most of it is actually autogenerated code in C++ (see model predictive controls). Python code is used mainly as a wrapper and for non-computational expensive parts. To use functional safety terminology, the EON functionality is considered QM (Quality Management). This means that any failure in delivering the desired output at the right time is perceived as bad quality and has no safety implications. So, how often those algorithms deliver the wrong output because some parts are written in Python? How often because RT isn't enforced? Negligible. Pretty much all the mistakes of a level 2 driver assistance system are due to the quality of the algorithms, the models, the policies etc… There is a long way to go before changing the coding language will be the lowest hanging fruit to improve the system. Until then, using the simplest and most agile coding language (given performance constraints) is probably the best way to maximize quality.

Re: Openpilot – open-source self-driving agent

#86

Post by taneq »

jerkstate wrote: post #82
taneq wrote: post #60

Earlier quoted context omitted.

I'd love to try it out but I can't bring myself to trust (on any level) something that's essentially a mobile phone running a hacked-together mashup of C and Python. There's no redundancy, no independent components cross-checking each other, nothing as far as I'm aware to try and provide any given level of reliability. (Don't get me wrong, Python is great for research - I wouldn't regard it as highly reliable or good…

I’m not trying to convince you, but the a-ha moment for me was when I was watching George talk about reverse-engineering the firmware for part of the cruise control in his car, and after decompiling it, he found that there was no functionality for the manufacturer to verify that the firmware had been written properly when flashed by the manufacturer. So just because a big, expensive car company made something, doesn’…

Oh, I'm not saying car companies are perfect and I'm well aware that the way the sausage gets made is nowhere as rigorous as we'd like (just take the Toyota unintended acceleration debacle!) I'm just saying I wouldn't add on top of that something that's essentially a phone app.

If nothing else, injecting stuff into your car's critical data bus seems like a bad idea. What if that less-than-carefully-written firmware you're talking about gets tripped up by a malformed CAN packet (which it would otherwise never have encountered) and fails in some fatal way?

Re: Openpilot – open-source self-driving agent

#87

Post by jensvdh »

cs02rm0 wrote: post #11
lukeqsee wrote: post #3

One side of me thinks this is really, really cool. The other side is running away as fast as possible. If anything shouldn't be developed by committee, it's something that controls a few thousand pounds hurtling down the highway at 70MPH with at best an observant human watching over it. Here's my reasoning: Mistakes in Django? Bad things happen. Mistakes in the Linux kernel? Potentially really bad things happen. Mist…

Anyone could anyway. I'd imagine this is safer than a solo effort. I think it's cool. Driving is risky anyway, this sort of thing could help improve that. I'm not sure it's any worse than a closed source alternative.

What if a bug in this software kills an innocent pedestrian?

Re: Openpilot – open-source self-driving agent

#88

Post by jensvdh »

aphextron wrote: post #76
mayniac wrote: post #74

Earlier quoted context omitted.

>Any sort of situation that could require you to take control is warned in advance. And what about false negatives, where something happens but it doesn't warn you? The fact that comma.ai pulled out of releasing the comma one when the NHTSA asked them some fairly basic questions is a massive red flag. They can't even guarantee that openpilot will detect low slung trailers or motorcycles [1]. If they want to develop a…

> If you're already forced to keep your eyes on the road, stop being lazy and do everyone around you a favour: grip the steering wheel, keep your feet on the pedals, and focus on the road. You do that, and I'll continue enjoying my self driving car.

Until it kills an innocent pedestrian. What are you gonna do then?

Re: Openpilot – open-source self-driving agent

#89

Post by dsl »

Bendingo wrote: post #51

This seems mind bogglingly dangerous. At least with commercial self-driving vehicles, there is a responsible legal entity (the car manufacturer). The thought of thousands of amateurs and enthusiasts releasing their hacked self-driving cars on the public roads is terrifying. The disclaimer "YOU ARE RESPONSIBLE FOR COMPLYING WITH LOCAL LAWS AND REGULATIONS" is nice, but I can't that stopping the mindset of "I've just t…

Some forward thinking car manufacturers are already trying to get ahead of this. My new Jeep for example has a physical box upstream from the ODB2 port that acts as a firewall blocking all non-read commands while in motion. (I of course disassembled my dash and installed a bypass so I could permanently disable the auto off at stop lights)

Re: Openpilot – open-source self-driving agent

#90

Post by jerkstate »

taneq wrote: post #86
jerkstate wrote: post #82

Earlier quoted context omitted.

I’m not trying to convince you, but the a-ha moment for me was when I was watching George talk about reverse-engineering the firmware for part of the cruise control in his car, and after decompiling it, he found that there was no functionality for the manufacturer to verify that the firmware had been written properly when flashed by the manufacturer. So just because a big, expensive car company made something, doesn’…

Oh, I'm not saying car companies are perfect and I'm well aware that the way the sausage gets made is nowhere as rigorous as we'd like (just take the Toyota unintended acceleration debacle!) I'm just saying I wouldn't add on top of that something that's essentially a phone app. If nothing else, injecting stuff into your car's critical data bus seems like a bad idea. What if that less-than-carefully-written firmware y…

On the other hand, maybe I lowered the chance of that happening by removing my stock LKAS system. I believe your implicit argument is that the manufacturer and the subcontractors and integrators who made the stock vehicle have a lower defect rate than the OpenPilot engineers, which I’m not sure if there’s any data to support.
Post reply on HN