Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lane Following Autopilot with Keras and Tensorflow

wroscoe.github.io

Post reply on HN
Print view View original

51–60 of 78 posts

Re: Lane Following Autopilot with Keras and Tensorflow

#51

Post by billyzs »

Not to put down the OP's work (I think it's a great project), but I'm just wondering what advantages might an ML approach have over "traditional" CV algorithms. In a really well controlled environment lanes will be easy to detect, and computing the difference between the current heading and lane direction should be doable; maybe if we're talking about complex outdoor environments and poor sensors then ML would have an advantage? Or if we're teaching the robot what the concept of a lane is?

I think back to the days when I basically implemented lane following with an array of photo resistors, an Arduino, a shitty robot made from Vex parts and some c code. The problem is much simpler than the one presented in this article, but then the computational resource used was order of magnitudes less. At what point then, do you decide that "OK I think the complexity and nature of the problem warrants the use of ML" or "Hmmm I think neural network is an overkill here"?

Re: Lane Following Autopilot with Keras and Tensorflow

#52

Post by wroscoe »

nojvek wrote: post #45

What I would love to see is an end to end neural network soln. On one end camera input comes through, on the other outputs for speed and steering angle. But rather than a black box, it's explainable what the different layers are doing. If neural nets are turing machines then we should be able to compile some parts of the net from code. Then the net is a library of layers. Some Layers trained with back prop, some comp…

I'm working on adding the throttle. This is difficult because you need to drive the correct speed and stopping or running off course can mess up the training data. This project was inspired by Otavio's carputer which does predict throttle, steering angle, and odometer.

Re: Lane Following Autopilot with Keras and Tensorflow

#53

Post by wroscoe »

Cyph0n wrote: post #3

Nicely done! But I'm assuming that this is more of an exercise rather than a real-world application of ML? I say this because the task of keeping a car between two lines is trivially done using control algorithms. Of course, the CV part -- "seeing" the lines -- requires some form of ML to work in the real world.

Yep, this was an exercise to compete in the DIYRobocars race in West Oakland last weekend. There were 9ish cars with 7 running end to end Tensorflow autopilots and the others using OpenCV/line detection. Open CV one the race.

Re: Lane Following Autopilot with Keras and Tensorflow

#54

Post by seertaak »

Cyph0n wrote: post #10
seertaak wrote: post #6

Earlier quoted context omitted.

> trivially done using control algorithms Is it really trivial? Honest question... Which control algorithms are you speaking of?

As u/gumby said, I was thinking of a PID controller. Basically, the car would continuously measure how far away it is from the line and compare that with the "expected" (computed) value. Based on the error between the two values, the controller would adjust some variable (e.g., wheel angle). Computing how much of an adjustment is required is where the PID part comes in. The controller uses the D erivative (rate of ch…

Cool. I'd never heard of a PID controller... although it sounds vaguely related to a Kalman filter. Indeed: https://www.quora.com/Is-there-any-intrinsic-connection-betw...

Thanks also for the link to wikiwand!

Re: Lane Following Autopilot with Keras and Tensorflow

#55

Post by Cyph0n »

seertaak wrote: post #54
Cyph0n wrote: post #10

Earlier quoted context omitted.

As u/gumby said, I was thinking of a PID controller. Basically, the car would continuously measure how far away it is from the line and compare that with the "expected" (computed) value. Based on the error between the two values, the controller would adjust some variable (e.g., wheel angle). Computing how much of an adjustment is required is where the PID part comes in. The controller uses the D erivative (rate of ch…

Cool. I'd never heard of a PID controller... although it sounds vaguely related to a Kalman filter. Indeed: https://www.quora.com/Is-there-any-intrinsic-connection-betw... Thanks also for the link to wikiwand!

Huh, I'll need to read up on Kalman filters then!

Yeah, Wikiwand is amazing. Be sure to grab the browser plugin: it automatically redirects any Wikipedia links to Wikiwand.

Re: Lane Following Autopilot with Keras and Tensorflow

#56

Post by argonaut »

Two major errors: 1) This doesn't seem to be controlling overfitting on the right validation set. 2) There isn't a test set at all (separate from validation).

Using Keras' "validation_split" parameter will just randomly select a validation set. This is not the right thing to do when your data is image sequences, because you will get essentially identical data in training and validation.

Because of this, the numbers/plot here might as well be training accuracy numbers.

Re: Lane Following Autopilot with Keras and Tensorflow

#57

Post by sja »

wroscoe wrote: post #47
sja wrote: post #46

I might be missing it, but I don't see instructions for installing TensorFlow/Keras on the Raspberry Pi in the Donkey repo or in this blog post (needed to actually run the trained model, it looks like). For TensorFlow, there are pre-built binaries and instructions to build from source here: https://github.com/samjabrahams/tensorflow-on-raspberry-pi Note: I am the owner of this repo

Donkey runs a client on the Pi and a remote server that runs Keras/Tensorflow.

A ha! Very cool- apologies for not seeing how it worked at first; I assumed you used the server to control/collect data manually, and then loaded the model onto the device. Thanks for the demo!

Re: Lane Following Autopilot with Keras and Tensorflow

#58

Post by argonaut »

nojvek wrote: post #45

What I would love to see is an end to end neural network soln. On one end camera input comes through, on the other outputs for speed and steering angle. But rather than a black box, it's explainable what the different layers are doing. If neural nets are turing machines then we should be able to compile some parts of the net from code. Then the net is a library of layers. Some Layers trained with back prop, some comp…

Almost all neural nets are not Turing complete. Only very specific RNNs are; most RNNs aren't, including pretty much any RNN model used in the real world right now (https://uclmr.github.io/nampi/talk_slides/grefenstette-nampi...).

Also, this is a useless fact, because so many other random things are Turing complete.

Re: Lane Following Autopilot with Keras and Tensorflow

#59

Post by argonaut »

ogrisel wrote: post #44
webaba wrote: post #41

Earlier quoted context omitted.

Indeed, I was not really talking about the vision pipeline. But once you decouple the problem (use ML for vision, planning for the trajectory, controls for the rest), you'll get much more stability, guarantees and insight into how to improve your problem. These kinds of end-to-end approaches are very hard to evaluate, they have zero educational value, are not parsimonious and tend to reduce people's analytical skills…

But to be able to decouple the vision pipeline you need a lot of manual annotation work which is tedious.

Tedious, and also solved for a decade already. Also, it's much easier to just find lanes using traditional CV and simply using annotators to verify the lane labels.

Re: Lane Following Autopilot with Keras and Tensorflow

#60

Post by deepnotderp »

nojvek wrote: post #45

What I would love to see is an end to end neural network soln. On one end camera input comes through, on the other outputs for speed and steering angle. But rather than a black box, it's explainable what the different layers are doing. If neural nets are turing machines then we should be able to compile some parts of the net from code. Then the net is a library of layers. Some Layers trained with back prop, some comp…

The end to end approach of regressing steering wheel angle already exists, check nvidia's paper.
Post reply on HN