Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

End to End Learning for Self-Driving Cars [pdf]

images.nvidia.com

Post reply on HN
Print view View original

91–100 of 102 posts

Re: End to End Learning for Self-Driving Cars [pdf]

#91

Post by argonaut »

jkldotio wrote: post #86
euyyn wrote: post #39

One problem with training a neural network end-to-end this way is that the system is susceptible to unpredictable glitches: The same principle that lets people trick a NN into [thinking a panda is a vulture]( https://codewords.recurse.com/issues/five/why-do-neural-netw... ) can happen randomly just by differing lighting/shadow conditions, sun glare, or who knows. One can always train the network with more and more sc…

The examples where this happens have always seemed fairly weak to me. How many of the grave errors, not just where it's the wrong type of animal or container but actually thinking it's radically different, survive an application of Gaussian blur? Furthermore self-driving cars are a combination of signals; you are going to need to simultaneously fool both LIDAR and cameras. On top of that you are going need to fool th…

1. Gaussian blur is just a spatial convolution (recall from signal processing). If a network is susceptible to adversarial examples, it will still be susceptible after a Gaussian blur (assuming the adversary knows you're applying a Gaussian blur. If the adversary doesn't, that's just security by obscurity, and they'll find out eventually).

2. A sequence of frames does not solve the issue because you can have a sequence of adversarial examples (although it would certainly make the actual physical process of projecting onto the camera more difficult, but not really any more difficult than the original problem of projecting an image onto a camera).

3. Using something conventional like LIDAR as a backup is the right approach IMO, and I totally agree with you there. But Tesla and lots of other companies aren't doing that because it's too expensive.

Re: End to End Learning for Self-Driving Cars [pdf]

#92

Post by smackmybishop »

taf2 wrote: post #20
jonny_eh wrote: post #10

Earlier quoted context omitted.

Even if it was trained to drive like a human, it'd be better in many ways: It never gets drunk, doesn't get tired, doesn't get distracted, can drive old people (or anyone) around who shouldn't be driving anymore, etc.

This almost sounds like the line from terminator two. "It can't be bargained with. It can't be reasoned with It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ..."

That's the first Terminator. Terminator 2 would be:

It would never leave him, and it would never hurt him, never shout at him, or get drunk and hit him, or say it was too busy to spend time with him. It would always be there. And it would die, to protect him.

Re: End to End Learning for Self-Driving Cars [pdf]

#93

Post by jkldotio »

argonaut wrote: post #91
jkldotio wrote: post #86

Earlier quoted context omitted.

The examples where this happens have always seemed fairly weak to me. How many of the grave errors, not just where it's the wrong type of animal or container but actually thinking it's radically different, survive an application of Gaussian blur? Furthermore self-driving cars are a combination of signals; you are going to need to simultaneously fool both LIDAR and cameras. On top of that you are going need to fool th…

1. Gaussian blur is just a spatial convolution (recall from signal processing). If a network is susceptible to adversarial examples, it will still be susceptible after a Gaussian blur (assuming the adversary knows you're applying a Gaussian blur. If the adversary doesn't, that's just security by obscurity, and they'll find out eventually). 2. A sequence of frames does not solve the issue because you can have a sequen…

1. If that's the case perhaps another kind of blurring? "Intriguing properties of neural networks" (https://arxiv.org/pdf/1312.6199.pdf page 6) has examples where you get radically different classifications that I don't think would occur naturally or survive a blur with some random element, let alone two moving cameras and a sequence of images. As the title says it's an intriguing property, not necessarily a huge problem.

2. I honestly can't think of a situation where this could occur. It's the equivalent of kids shining lasers into the eyes of airline pilots, but the kids need a PhD in deep learning and specialised equipment to be able to do it. A hacker doing some update to the software via a network sounds much more plausible than attacking the system through its vision while it's traveling.

3. This is the real point in the end I guess, this Google presentation (https://www.youtube.com/watch?v=tiwVMrTLUWg) shows that the first autonomous cars to be sold will be very sophisticated with multiple systems and a lot of traditional software engineering. Hopefully LIDAR costs will come down.

Re: End to End Learning for Self-Driving Cars [pdf]

#94

Post by iamaaditya »

ksml wrote: post #73
EugeneOZ wrote: post #68

Earlier quoted context omitted.

Can be used as additional system for taking decisions.

This is a great point. Reminds me of how airplanes have redundant flight computers and compare the outputs of the computers to determine if one might be faulty. https://en.wikipedia.org/wiki/Fly-by-wire#Redundancy The output of different self driving models could be compared to handle more difficult driving situations -- I never thought of that.

This reminds me of ensemble. Most machine learning techniques benefit in accuracy and precision by ensembling different methods. More different the methods are (variance) better improvement you get.

Re: End to End Learning for Self-Driving Cars [pdf]

#95

Post by aab0 »

argonaut wrote: post #59
aab0 wrote: post #55

Earlier quoted context omitted.

The adversarial examples are so weak that they disappear if you give the CNN even some attention or foveation mechanisms (that is, they work only on a single pass). How much effect are they going to have on a CNN being used at 30FPS+ to do lane following under constantly varying lighting and appearances and position? None.

Are you referring to this foveation paper ( http://arxiv.org/abs/1511.06292 )? I'm quite skeptical of the claims in that paper; upon closer reading their experiments are problematic. Also, it appears the paper was rejected. I can elaborate if that is indeed the case.

Yes. And rejection means little. The point is that adversarial examples have to be fragiley constructed to fool on one single example for one forward-pass. There is no evidence that any adversarial examples exist which can fool an even slightly more sophisticated CNN, fool a simple CNN over many time-steps, fool a simple CNN for enough time-steps to lead to any noticeable differences in action, fool a simple CNN for enough time-steps to lead to a noticeable difference in action which could lead to an accident, or fool a simple CNN for enough time-steps to lead to a noticeable difference in action which leads to an accident frequently enough to noticeably reduce the safety advantages.

Re: End to End Learning for Self-Driving Cars [pdf]

#96

Post by webaba »

In aeronautics, we were able to increase safety to an insane level by understanding the physics of the environment, formally proving and certifying algorithms, using Robust control theories that allow to formally deal with uncertainty. The power of mathematical modeling together with robust software testing and a limited/controlled use of learning algorithms is - in my opinion - likelier to bring safety to such systems rather than such an opaque use of CNNs. And I'm not even talking about human machine interaction or liability issue that would emerge from such extreme approaches (which node of the net or which training sample will be blamed?). I guess CNNs allow a lot of wannabe engineers to play with real world problems and dream that their 10 lines python code would match semantic models if fed with more training data, but I'm pretty sure Aircraft/Car manufacturers will/should not replace formally certified controls algorithms and redundant architectures built on the top of hundreds years of analytical results with a rack of NVIDIA GPUs.

Re: End to End Learning for Self-Driving Cars [pdf]

#97

Post by argonaut »

jkldotio wrote: post #93
argonaut wrote: post #91

Earlier quoted context omitted.

1. Gaussian blur is just a spatial convolution (recall from signal processing). If a network is susceptible to adversarial examples, it will still be susceptible after a Gaussian blur (assuming the adversary knows you're applying a Gaussian blur. If the adversary doesn't, that's just security by obscurity, and they'll find out eventually). 2. A sequence of frames does not solve the issue because you can have a sequen…

1. If that's the case perhaps another kind of blurring? "Intriguing properties of neural networks" ( https://arxiv.org/pdf/1312.6199.pdf page 6) has examples where you get radically different classifications that I don't think would occur naturally or survive a blur with some random element, let alone two moving cameras and a sequence of images. As the title says it's an intriguing property, not necessarily a huge pr…

1. Those are examples for a network that does not use blurring. You have the be careful because, remember, the adversary can tailor their examples to whatever preprocessing you use. So the adversarial examples for a network with blurring would look completely different, but they would still exist. Randomness could just force the adversary to use a distribution over examples, and it could mean they are still able to fool you half the time instead of all the time. However, I wouldn't trust my intuition here: that is really a question for the machine learning theory researchers (whether there is some random scheme that is provably resilient or if they're all provably vulnerable, or proving some error bounds on resilience, etc.).

2. The problem of projecting an image onto a car's camera already implies you'd be able to do it for a few seconds.

Re: End to End Learning for Self-Driving Cars [pdf]

#98

Post by zardo »

monk_e_boy wrote: post #83
Joeri wrote: post #79

Earlier quoted context omitted.

You use a test set of scenario's on which you don't train but only measure effectiveness. When accuracy on the test set exceeds your chosen threshold, that is good enough. What is the accuracy of the human brain in recognizing traffic situations? It is probably not that hard to get a NN to do better, even if periodically it still causes an accident. This is the uncanny valley effect for self-driving cars. It's not en…

The constant question for self driving cars is "How will we know when they are good enough?" Is there any reason they couldn't just put a driving test examiner in the car and test it like you would a human? Just ask the thing to drive around town, emergency stop, park, navigate a roundabout etc.

Yeah, the driving test assumes you have human level cognitive function and can apply the demonstrated skills in a much much wider variety of situations than those that occur during the test.

Re: End to End Learning for Self-Driving Cars [pdf]

#99

Post by argonaut »

aab0 wrote: post #95
argonaut wrote: post #59

Earlier quoted context omitted.

Are you referring to this foveation paper ( http://arxiv.org/abs/1511.06292 )? I'm quite skeptical of the claims in that paper; upon closer reading their experiments are problematic. Also, it appears the paper was rejected. I can elaborate if that is indeed the case.

Yes. And rejection means little. The point is that adversarial examples have to be fragiley constructed to fool on one single example for one forward-pass. There is no evidence that any adversarial examples exist which can fool an even slightly more sophisticated CNN, fool a simple CNN over many time-steps, fool a simple CNN for enough time-steps to lead to any noticeable differences in action, fool a simple CNN for…

The paper was rejected (you can read the ICLR comments) because the experiments did not really support their point. And I agree. The gist of the experiments they ran to support their thesis was to take a CNN and construct adversarial examples that sucessfully fooled it. They then applied foveation, and showed that the CNN was no longer fooled. Which is obvious! It's kind of obvious to me that adding preprocessing that the attacker is unaware of would be able to beat the attacker. What they didn't do is regenerate the adversarial examples assuming the attacker has knowledge that the target was using foveation.

There are no experiments that support your statements, unfortunately.

Re: End to End Learning for Self-Driving Cars [pdf]

#100

Post by euyyn »

RobertoG wrote: post #77
euyyn wrote: post #39

One problem with training a neural network end-to-end this way is that the system is susceptible to unpredictable glitches: The same principle that lets people trick a NN into [thinking a panda is a vulture]( https://codewords.recurse.com/issues/five/why-do-neural-netw... ) can happen randomly just by differing lighting/shadow conditions, sun glare, or who knows. One can always train the network with more and more sc…

Is not this the problem of induction in Philosophy? Are not all the minds subjected to the same limitation? ( https://en.wikipedia.org/wiki/Problem_of_induction )

In theory, yes. In practice, we've built our roads, signals, and car interiors, tailored to our specific minds and concepts.

So when we see someone that just started driving perform well under some circumstances, we can good performance under circumstances that are similar to the human mind. The problem that the "fooling neural networks" experiments show is that two things that are similar for humans can be wildly different for a NN that's been trained to recognize them.

Post reply on HN