An Unintuitive Take on Data Augmentation for Self-Driving Cars
towardsdatascience.com
An Unintuitive Take on Data Augmentation for Self-Driving Cars
1–10 of 38 posts
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#2Forgive me hijackjng a self-driving ML discussion to ask a novice question on self-driving — “stateful” vs “stateless”, and how can state be used safely?
Specifically I mean pre-determined saved/downloaded knowledge about the current route being driven, or your specific instantaneous location. Anything from annotated maps, to LIDAR scans.
It seems like the paradox is that stateful algorithms can dramatically improve performance by factoring in data that real drivers benefit from tremendously, namely, familiarity with the road.
However detecting when known state is violated due to some temporal change (road work, accident, natural disaster), and being able to shift back into “general flight” rules, seems in some ways to be even trickier than not using historical state in the first place.
So you need an algorithm that can learn to drive better than a human, without having basically any contextual knowledge of the road it’s driving on. In other words the algorithm needs to be a better driver the first time it ever goes down a road than a human who has driven the road for years.
As a human, familiarity with the road makes a tremendous difference for how I drive it. I generally drive the same roads 80% of the time (I.e. commuting) and knowing what to expect around each bend absolutely changes how I drive the road, even down to where the speed traps will be.
Any ML driving solution that depends on super-fancy stateful pre-scans of the environment seem fundamentally flawed. If you can’t drive a road safely that isn’t pre-scanned in hidef LIDAR for instance, I don’t suppose you can safely drive that road on an arbitrary Monday. Maybe solutions like this were never even attempted, but certainly some amount of statefulness is inherent in some of the commercial solutions out there (Supercruise?)
So what kind of state can you use safely? First thought was basics like speed limit, number of lanes, type of road surface, and your algorithm may have predictions of upcoming changes being constantly weighted against the current assessment of the present state.
So maybe the fundamental rule is no hard-coded state that can’t be reliably detected in real-time to the point where at some point a real-time signal is able to over-ride the programmed state?
But then it seems that inevitably you get to the point where your real-time classifiers are basically running the show anyway so are you back to - what good is a Map anyway?
Would love interested but accessible readings on the subject, or maybe it’s so off base that it’s not really part of the discussion?
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#3Forgive me hijackjng a self-driving ML discussion to ask a novice question on self-driving — “stateful” vs “stateless”, and how can state be used safely? Specifically I mean pre-determined saved/downloaded knowledge about the current route being driven, or your specific instantaneous location. Anything from annotated maps, to LIDAR scans. It seems like the paradox is that stateful algorithms can dramatically improve…
"But then it seems that inevitably you get to the point where your real-time classifiers are basically running the show anyway so are you back to - what good is a Map anyway?"
Yep, pretty much. Robust real-time perception, for a wide variety of tasks that happen very rarely is one of the key bottlenecks.
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#4Forgive me hijackjng a self-driving ML discussion to ask a novice question on self-driving — “stateful” vs “stateless”, and how can state be used safely? Specifically I mean pre-determined saved/downloaded knowledge about the current route being driven, or your specific instantaneous location. Anything from annotated maps, to LIDAR scans. It seems like the paradox is that stateful algorithms can dramatically improve…
I wonder how humans can benefit from pre-knowledge and be able to respond to the unexpected.
Can this be modeled somehow?
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#5That's scary. Using deep learning to recognize "obstruction" using cameras.
Discriminate "car", "bike", "pedestrian", and "bus", sure. Recognize signs and traffic lights, fine. But "obstruction"? No. That's a geometry problem. If it's not flat road, it's an obstacle. Deep learning is for deciding what kind of obstacle. Because deep learning just isn't that good. It's going to be badly wrong a few percent of the time.
"The car will almost never be on the left side of the road, and the cameras will never flip angles, so training on flipped data forces the network to overgeneralize to situations it will never see." What could possibly go wrong?
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#6That's scary. Using deep learning to recognize "obstruction" using cameras. Discriminate "car", "bike", "pedestrian", and "bus", sure. Recognize signs and traffic lights, fine. But "obstruction"? No. That's a geometry problem. If it's not flat road, it's an obstacle. Deep learning is for deciding what kind of obstacle. Because deep learning just isn't that good. It's going to be badly wrong a few percent of the time.…
What aspect of "deep learning" are you concerned about? Would you prefer obstruction detection be done with shallow NNs? Or a linear classifier?
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#7Forgive me hijackjng a self-driving ML discussion to ask a novice question on self-driving — “stateful” vs “stateless”, and how can state be used safely? Specifically I mean pre-determined saved/downloaded knowledge about the current route being driven, or your specific instantaneous location. Anything from annotated maps, to LIDAR scans. It seems like the paradox is that stateful algorithms can dramatically improve…
The question of statefulness reminds me of some not-immediately-intuitive observations from game development. Say you're working on potentially-visible-set computation for a first-person shooter. Where do you spend your optimization efforts? Do you put a lot of work into taking advantage of object coherence between frames, for instance? Chances are good that something that's visible in one frame will remain visible in the next, right? So if you start with that assumption you might think you could save some time by not bothering to test any surfaces on that object for visibility until some nominally-unrelated condition is met, such as the player turning rapidly in place.
You can waste a lot of time thinking about optimizations like that, but at some point it'll occur to you that it's a giant waste of time to optimize for anything but the worst case, where visibility information computed during one frame is completely unusable in the next frame for whatever reason. Otherwise all that your clever frame-coherence hacks can ever do is speed up the rendering of the sorts of frames that weren't going to dominate the player's perception of the game's performance anyway. An engine that renders 95% of frames at 60 FPS and 5% at 43 FPS is going to look pretty terrible, so you're usually better off putting work into the slowest frames rather than wasting time looking for hacks and shortcuts that make the fast frames even faster.
Likewise, yes, you can assume that the car will almost never be traveling backwards on the left side of the road or whatever, so the temptation to take advantage of that is going to be high. But the cases where that assumption breaks down will hurt the user's experience badly, possibly disastrously. So you're better off without relying too much on assumptions that contain phrases like "hardly ever" or "most of the time" or "typically."
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#8That's scary. Using deep learning to recognize "obstruction" using cameras. Discriminate "car", "bike", "pedestrian", and "bus", sure. Recognize signs and traffic lights, fine. But "obstruction"? No. That's a geometry problem. If it's not flat road, it's an obstacle. Deep learning is for deciding what kind of obstacle. Because deep learning just isn't that good. It's going to be badly wrong a few percent of the time.…
What aspect of "deep learning" are you concerned about? Would you prefer obstruction detection be done with shallow NNs? Or a linear classifier?
With something that measures depth and can profile the ground. Probably a LIDAR.
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#9That's scary. Using deep learning to recognize "obstruction" using cameras. Discriminate "car", "bike", "pedestrian", and "bus", sure. Recognize signs and traffic lights, fine. But "obstruction"? No. That's a geometry problem. If it's not flat road, it's an obstacle. Deep learning is for deciding what kind of obstacle. Because deep learning just isn't that good. It's going to be badly wrong a few percent of the time.…
If DNNs are unreliable, why is it ok to use them to discriminate a car from a pedestrian? The planning that you do in order to avoid hitting a car is very different from what you need in order to avoid a pedestrian. If you erroneously treat a pedestrian as a car, you are very likely to cause a tragedy.
Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars
#10Forgive me hijackjng a self-driving ML discussion to ask a novice question on self-driving — “stateful” vs “stateless”, and how can state be used safely? Specifically I mean pre-determined saved/downloaded knowledge about the current route being driven, or your specific instantaneous location. Anything from annotated maps, to LIDAR scans. It seems like the paradox is that stateful algorithms can dramatically improve…
The question of statefulness reminds me of some not-immediately-intuitive observations from game development. Say you're working on potentially-visible-set computation for a first-person shooter. Where do you spend your optimization efforts? Do you put a lot of work into taking advantage of object coherence between frames, for instance? Chances are good that something that's visible in one frame will remain visible i…
[deleted]