Seems very similar to the GeoHot self driving car experiment.
End to End Learning for Self-Driving Cars [pdf]
81–90 of 102 posts
Re: End to End Learning for Self-Driving Cars [pdf]
#82Earlier quoted context omitted.
For the "thinking a panda is a vulture" problem, don't humans fail in similar ways? The analogous examples for us are camouflage, optical illusions, logical fallacies, etc. It doesn't really have to be perfect as long as it doesn't fail in common scenarios.
For the "thinking a panda is a vulture" problem, don't humans fail in similar ways? This is a good question. My impression is that humans fail and artificial neural networks fail but we don't know enough about the brain to say artificial neural networks fail in the same way as humans. As another poster notes, humans accept human error more than computer error and I think that's because humans have an internal model o…
Is this a good time to ask whether the dress is blue/black or white/gold? ;)
Re: End to End Learning for Self-Driving Cars [pdf]
#83One problem with training a neural network end-to-end this way is that the system is susceptible to unpredictable glitches: The same principle that lets people trick a NN into [thinking a panda is a vulture]( https://codewords.recurse.com/issues/five/why-do-neural-netw... ) can happen randomly just by differing lighting/shadow conditions, sun glare, or who knows. One can always train the network with more and more sc…
You use a test set of scenario's on which you don't train but only measure effectiveness. When accuracy on the test set exceeds your chosen threshold, that is good enough. What is the accuracy of the human brain in recognizing traffic situations? It is probably not that hard to get a NN to do better, even if periodically it still causes an accident. This is the uncanny valley effect for self-driving cars. It's not en…
The constant question for self driving cars is "How will we know when they are good enough?"
Is there any reason they couldn't just put a driving test examiner in the car and test it like you would a human? Just ask the thing to drive around town, emergency stop, park, navigate a roundabout etc.
Re: End to End Learning for Self-Driving Cars [pdf]
#84Earlier quoted context omitted.
For the "thinking a panda is a vulture" problem, don't humans fail in similar ways? The analogous examples for us are camouflage, optical illusions, logical fallacies, etc. It doesn't really have to be perfect as long as it doesn't fail in common scenarios.
It should never fail since any failure could potentially create a fatal scenario. People usually accept fatalities because of human error but they won't accept death because of algorithmic failure.
Not every car crash ends in death. But the AI will learn a lot from each crash. I think mistakes and 'bugs' in the system will get ironed out at low speed crashes and in high speed crashes on test circuits...
Have you seen the AI Formula 1 called roborace? Once those cars get good enough to beat Lewis Hamilton or Seb Vettel I'll trust it with me and my family.
Re: End to End Learning for Self-Driving Cars [pdf]
#85In case you missed it, the ninth reference is a good video of the system in action:
https://drive.google.com/file/d/0B9raQzOpizn1TkRIa241ZnBEcjQ...
Re: End to End Learning for Self-Driving Cars [pdf]
#86One problem with training a neural network end-to-end this way is that the system is susceptible to unpredictable glitches: The same principle that lets people trick a NN into [thinking a panda is a vulture]( https://codewords.recurse.com/issues/five/why-do-neural-netw... ) can happen randomly just by differing lighting/shadow conditions, sun glare, or who knows. One can always train the network with more and more sc…
The examples where this happens have always seemed fairly weak to me. How many of the grave errors, not just where it's the wrong type of animal or container but actually thinking it's radically different, survive an application of Gaussian blur? Furthermore self-driving cars are a combination of signals; you are going to need to simultaneously fool both LIDAR and cameras.
On top of that you are going need to fool them over multiple frames, while the sensors get a different angle on the subject as the car moves. For example in the first Deep Q-learning paper, "Playing Atari with Deep Reinforcement Learning"[0], they use four frames in sequence. That was at the end of 2013.
I don't think anyone will be able to come up with a serious example that fools multiple sensors over multiple frame as the sensors are moving. Even if they do then inducing an unnecessary emergency stopping situation is still not the same as getting the car to drive into a group of people. Even if fooled in some circumstances the cars will still be safer than most human drivers and still have a massive utilitarian moral case in relation to human deaths, on top of the economic case, to be used.
The fooling of networks is still an interesting thing, but it's been overplayed to my mind and is not particularly more interesting than someone being fooled for a split second into thinking a hat stand with a coat and hat on it is a person when they first see it out of the corner of their eye.
[0]http://arxiv.org/pdf/1312.5602.pdf page 5
Re: End to End Learning for Self-Driving Cars [pdf]
#87Earlier quoted context omitted.
I feel like people just assume that they are annoying to drive around, but very few people actually have experience driving around google's cars. The times that I'm around them (a few mornings a week), they are never in any way weird or annoying. In fact, they are extremely predictable, and therefor, if anything, less annoying to drive around.
I think the fear is that they'll drive the speed limit on the freeway, even when no one else is.
People are afraid the cars will follow traffic laws, and drive legally, even when no one else is? And, I find it hard to believe that everyone disobeys the posted speed limits on freeways. For example, trucks with speed-limiters, people who don't want to break the law, buses, etc.
Re: End to End Learning for Self-Driving Cars [pdf]
#88Earlier quoted context omitted.
For the "thinking a panda is a vulture" problem, don't humans fail in similar ways? The analogous examples for us are camouflage, optical illusions, logical fallacies, etc. It doesn't really have to be perfect as long as it doesn't fail in common scenarios.
It should never fail since any failure could potentially create a fatal scenario. People usually accept fatalities because of human error but they won't accept death because of algorithmic failure.
Do people accept death due to autopilot error in aeroplanes? It's the same thing. There has been no demands for autopilot to be removed from planes or mass refusal to fly. The reason is that most people can see that autopilot is an overall safety gain compared with getting a human to concentrate on the same thing for long periods of time.
Re: End to End Learning for Self-Driving Cars [pdf]
#89One problem with training a neural network end-to-end this way is that the system is susceptible to unpredictable glitches: The same principle that lets people trick a NN into [thinking a panda is a vulture]( https://codewords.recurse.com/issues/five/why-do-neural-netw... ) can happen randomly just by differing lighting/shadow conditions, sun glare, or who knows. One can always train the network with more and more sc…
The examples where this happens have always seemed fairly weak to me. How many of the grave errors, not just where it's the wrong type of animal or container but actually thinking it's radically different, survive an application of Gaussian blur? Furthermore self-driving cars are a combination of signals; you are going to need to simultaneously fool both LIDAR and cameras. On top of that you are going need to fool th…
"It unlikely to happen" is not a good strategy to rely on with systems operating at scale. There are about a billion cars on earth traveling trillions of miles every year, many of which will eventually be self-driving. At that scale, you don't need a malicious actor working to fool these systems, you just need to encounter the wrong environment. And even if the system is perfect on the day it's released, that doesn't mean that it will remain so indefinitely (even with proper maintenance).
Studying induced failure in neural networks may help us understand the failure modes and mechanisms of these systems.
Re: End to End Learning for Self-Driving Cars [pdf]
#90One problem with training a neural network end-to-end this way is that the system is susceptible to unpredictable glitches: The same principle that lets people trick a NN into [thinking a panda is a vulture]( https://codewords.recurse.com/issues/five/why-do-neural-netw... ) can happen randomly just by differing lighting/shadow conditions, sun glare, or who knows. One can always train the network with more and more sc…
For the "thinking a panda is a vulture" problem, don't humans fail in similar ways? The analogous examples for us are camouflage, optical illusions, logical fallacies, etc. It doesn't really have to be perfect as long as it doesn't fail in common scenarios.
Humans don't appear to fail in the same ways - camouflage and optical illusions are very different to the specific imperceptible-to-humans changes that trick neural networks. Then again, there's no way to test the method on humans because you need to know the neural network weights and that is tricky for people!
In practice it probably doesn't matter anyway - the chance of the exact required perturbation of the input happening by chance are infinitesimal, due to the high dimensionality of the input. And even if it was a problem there are ways around it.