Lane Following Autopilot with Keras and Tensorflow

This is an unbelievably wrong comment. All the Lyapunov and traditional Process control theory in the world won't help you solve autonomous driving. Also regarding "Guarantees and Safety" they don't magically appear out of thin air when you use traditional process control especially in noisy domains like autonomous driving. This comment is equivalent of "I can write code to solve Atari Pong in any programming language deterministicly so any post showing Deep Reinforcement Learning is stupid"...

While this example is simplified - and I wouldn't recommend it for a real-world full-size vehicle trial - it does implement everything (scaled down) described in NVidia's paper:

https://images.nvidia.com/content/tegra/automotive/images/20...

In short, the project uses OpenCV for the vision aspect, a small CNN for the model, uses "behavioral cloning" (where the driver drives the vehicle, taking images of the "road" and other sensor data like steering - as features and labels respectively - then trains on that data), and augmentation of the data to add more training examples, plus training data for "off course" correction examples...

If you read the NVidia paper, you'll find that's virtually all the same things they did, too! Now - they gathered a butt-ton (that's a technical measurement) more data, and their CNN was bigger and more complex (and probably couldn't be trained in reasonable time without a GPU), plus they used multiple cameras (to simulate the "off-lane" modes), and they gathered other label data (not just steering, but throttle, braking, and other bits)...but ultimately, the author of the smaller system captured everything.

Furthermore, NVidia's system was used on a real-world car, and performed quite well; there are videos out there of it in action.

This is virtually the same kind of example system that the "behavioral cloning" lab of Udacity's Self-Driving Car Engineer Nanodegree is using. We're free to select what and how to implement things, of course, but I am pretty certain we all understand that this form of system works fairly well in a real-world situation, and so most of us are going down the same route (ie, behavioral cloning, cnn, opencv, etc). Our "car" though is a simulation vehicle on a track, built using Unity3D.

> Of course, the CV part -- "seeing" the lines -- requires some form of ML to work in the real world.

Actually, it doesn't. The first lab we did in the Udacity course used OpenCV and Numpy exclusively to "find and highlight lane-lines" (key part was to convert the image from BGR to HSV, and mask using the hue). No ML was required.

That said - I wouldn't trust it for real-world vehicle driving use - but it possibly could be used as part of a system; however, as NVidia has shown, a CNN works much better, without needing to do any pre-processing with OpenCV to extract features of the image - the CNN learns to do this on its own.

A tree falls in an intersection because some carpenter ants chewed through trunk. Cars swerve to miss the tree and collide in an inelastic ball of nonlinearity, showering debris everywhere. You approach this at 65 mph and have 23 ft to decide what to do. Fear not, you have a list, a perfect list with coordinates, velocities, and material properties of every solid body in the area. Furthermore, without great intellectual effort, you can solve the millions of coupled differential equations that govern the dynamics of the entire system in near real time. Oh, and your list also has a measure of importance of each bit of mass, whether it is human, animal, or inert. And your list also accounts for the degrees of freedom introduced by every other car approaching the intersection, also using their own respective lists and perfect knowledge of the world to miss each other?

https://github.com/wroscoe/donkey