21–30 of 38 posts
I find it interesting that the project team had been implementing several "improvements" that actually worsened performance, and that it took an intern to figure this out. Or are these augmentations default-on options in deep learning frameworks?
That's scary. Using deep learning to recognize "obstruction" using cameras. Discriminate "car", "bike", "pedestrian", and "bus", sure. Recognize signs and traffic lights, fine. But "obstruction"? No. That's a geometry problem. If it's not flat road, it's an obstacle. Deep learning is for deciding what kind of obstacle. Because deep learning just isn't that good. It's going to be badly wrong a few percent of the time.…
Forgive me hijackjng a self-driving ML discussion to ask a novice question on self-driving — “stateful” vs “stateless”, and how can state be used safely? Specifically I mean pre-determined saved/downloaded knowledge about the current route being driven, or your specific instantaneous location. Anything from annotated maps, to LIDAR scans. It seems like the paradox is that stateful algorithms can dramatically improve…
The question of statefulness reminds me of some not-immediately-intuitive observations from game development. Say you're working on potentially-visible-set computation for a first-person shooter. Where do you spend your optimization efforts? Do you put a lot of work into taking advantage of object coherence between frames, for instance? Chances are good that something that's visible in one frame will remain visible i…
That's scary. Using deep learning to recognize "obstruction" using cameras. Discriminate "car", "bike", "pedestrian", and "bus", sure. Recognize signs and traffic lights, fine. But "obstruction"? No. That's a geometry problem. If it's not flat road, it's an obstacle. Deep learning is for deciding what kind of obstacle. Because deep learning just isn't that good. It's going to be badly wrong a few percent of the time.…
Earlier quoted context omitted.
The question of statefulness reminds me of some not-immediately-intuitive observations from game development. Say you're working on potentially-visible-set computation for a first-person shooter. Where do you spend your optimization efforts? Do you put a lot of work into taking advantage of object coherence between frames, for instance? Chances are good that something that's visible in one frame will remain visible i…
The problem is a nigh-unbounded problem space: so many variables that there are myriads of corner cases.
My favorite example is a humorous image that went around several years ago, a photo that depicted someone driving in a Miata or similar convertible with the top down. The convertible was driving behind a sewage truck, the kind that has a large tank with a hose attachment to clean out septic tanks.
The sewage truck, in turn, was heading straight for an overpass that obviously had nowhere near enough clearance. An alert human driver would have no problem anticipating what was about to happen, but the oblivious one in the Miata was clearly about to find out the hard way.
Every time I find myself idly wondering if would be fun to work on self-driving cars, I flash back to that image. Then I get back to work on whatever I'm actually supposed to be doing.
Earlier quoted context omitted.
If DNNs are unreliable, why is it ok to use them to discriminate a car from a pedestrian? The planning that you do in order to avoid hitting a car is very different from what you need in order to avoid a pedestrian. If you erroneously treat a pedestrian as a car, you are very likely to cause a tragedy.
As a non-driver, I'm curious, how different is the planning? Isn't it just a matter of size?
That's scary. Using deep learning to recognize "obstruction" using cameras. Discriminate "car", "bike", "pedestrian", and "bus", sure. Recognize signs and traffic lights, fine. But "obstruction"? No. That's a geometry problem. If it's not flat road, it's an obstacle. Deep learning is for deciding what kind of obstacle. Because deep learning just isn't that good. It's going to be badly wrong a few percent of the time.…
> "The car will almost never be on the left side of the road, and the cameras will never flip angles, so training on flipped data forces the network to overgeneralize to situations it will never see." What could possibly go wrong? That quote jumped off the page for me as well. This sounds like the start of a new blog post: Falsehoods Programmers Believe About Driving You can throw a few more in there. - "Pedestrians…
Earlier quoted context omitted.
> "The car will almost never be on the left side of the road, and the cameras will never flip angles, so training on flipped data forces the network to overgeneralize to situations it will never see." What could possibly go wrong? That quote jumped off the page for me as well. This sounds like the start of a new blog post: Falsehoods Programmers Believe About Driving You can throw a few more in there. - "Pedestrians…
Structured, scrutible systems perform really badly on this task and I don’t think anyone knows how to do much better. The current state of the art is an extremely unstructured and inscrutable system with no design behind it whatsoever, and that system gets a lot of people killed. The moment anything can do better than that, we should push it hard even if it has stupid failures.
Earlier quoted context omitted.
Structured, scrutible systems perform really badly on this task and I don’t think anyone knows how to do much better. The current state of the art is an extremely unstructured and inscrutable system with no design behind it whatsoever, and that system gets a lot of people killed. The moment anything can do better than that, we should push it hard even if it has stupid failures.
With the obvious assumption that we do have something better. So far...nope.
I find it interesting that the project team had been implementing several "improvements" that actually worsened performance, and that it took an intern to figure this out. Or are these augmentations default-on options in deep learning frameworks?
nobody’s quite sure why anything works, though. hence comparisons to alchemy — we have lots of cool empirical results, and a highly elaborated set of theories and ideas trying to explain what works and what doesn’t, but in fact these theories often fail to make correct predictions. So the only way to make progress is constant blind experimentation.