Earlier quoted context omitted.
Humans don't use accurate depth maps to drive. Humans are very good at estimating distance from a combination parallax visual cues and experience. We don't need to have seen a specific model of car before to judge how far away it is with a high level of accuracy.
Humans can drive well with vision in only one eye, so it seems like even binocular vision isn't necessarily as important as some of those cues and experience.
Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
151–160 of 174 posts
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#152Earlier quoted context omitted.
> People drive reasonably well using vision primarily This is not accurate however. Other important senses in use include proprioceptive, hearing and tactile feedback from wheels. In addition to vision and the improved dynamic range of eyes, there is the important fact that human vision integrates a world model into expectations. Human vision also models time and motion which help manage where to focus attention. Hum…
I commend you. Very few people actually try feeding images into these things to show how bad they are. For a real shock at how poor performance is, try feeding frames of video in. (Video is different because it generally doesn't have carefully stereotypically framed, exposed, in-focus content). Edit: examples of ResNet applied to video by a colleague http://blog.piekniewski.info/2016/08/12/how-close-are-we-to-...
So where do the reports of superhuman abilities come from? Well since there are many breeds of dogs in the ImageNet, an average human (like me) will not be able to distinguish half of them (say Staffordshire bullterrier from Irish terrier or English foxhound - yes there are real categories in ImageNet, believe it or not). The network which was "trained to death" on this dataset will obviously be better at that aspect. In all practical aspects an average human (even a child) is orders of magnitude better at understanding/describing scenes than the best deep nets (as of late 2015) trained on ImageNet.
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#153Earlier quoted context omitted.
Classification is not the right metric to use here. Lidar doesn't classify the objects it's looking it, it just tells you the direction and distance. Cameras can also gauge distance pretty effectively from parallax. Either using multiple cameras, or from the motion of the vehicle itself, or both. From this it should be possible to gauge where obstacles are and drive safely. But NNs give the possibility of gathering m…
> But NNs give the possibility of gathering much more information from recognizing objects. Information that Lidar systems don't have. Wouldn't you feed both the depth map from the lidar and imagery from the cameras into the neural network? I imagine that a variety of different sensors as input would make it easier to do classification. As an analogy, someone who has lost their sense of smell might have a harder time…
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#154Earlier quoted context omitted.
Wait, who cares about the power consumption in a car? perf/W is a useful number if you are trying to stack a data center full of these things and electricity (incl for cooling) is essentially your only cost. But if a car is using 300Wh/mile or, in an ICE, generating 100kW in excess heat it is an entirely pointless metric. (Just for clarification: no one is using the Drive PX2 in a data center. Just look at these conn…
Informal anecdotal knowledge: GM wants 10-20 watts for the entire self-driving system, sensors and all.
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#155Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#156Earlier quoted context omitted.
A huge number of traffic deaths are due to alcohol. An autonomous system that's as safe as a sober human would improve safety by a factor of 2 or 3. Many of the other deaths are due to distraction, inattention, or slow reaction times. Get rid of those and you can probably see an order of magnitude improvement with something that is nominally "no better than a human driver."
Would you (while sober) get into a car driven by an autonomous system that was demonstrably more likely to get into a crash than the average sober, awake, healthy driver, but less likely to get into a crash than the average driver? Honest question. I don't think I would.
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#157Earlier quoted context omitted.
Interesting question. As long as you still have the manual option, it doesn't really matter. You can just get in and drive if you want to. As the option becomes more common, obviously impaired driving becomes less common.
It matters if, for example, the car is an Uber and you aren't allowed to drive it.
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#158I'm still not happy with self-driving on vision alone, or vision augmented with radar. There are too many hard cases for vision. Everybody who has good self-driving right now - Google, Otto, Volvo, GM - uses LIDAR. Self-driving is coming to the first end users in 2017, in Volvo's test of 100 vehicles. Volvo has multiple LIDARs, multiple radars, multiple cameras, redundant computers, and redundant actuators. They're b…
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#159Earlier quoted context omitted.
Driving has almost nothing to do with image classification. * Humans implicitly perform SLAM (simulataneous localization and mapping). What do I mean? Look around your room. Close your eyes. Visualize the room. As a human, you've built a rough 3D model of the room. And if you keep your eyes open and walk through the room, that map is pretty fine-grained/detailed too and humans can keep track of where they are in the…
Let me make sure I get this: When a Tesla (or other non-LIDAR) vehicle is driving, it is not continuously building a 3D model of its environment. Instead, it is matching patterns on the road, and "understanding" based off what it sees in an otherwise flat image. Whereas LIDAR vehicles use the LIDAR technology to develop a map of the world around them, for additional understanding?
Re: Nvidia on new self-driving system: “basically 5 years ahead and coming in 2017”
#160Earlier quoted context omitted.
At a certain point it's just about recognising an object which shares broad characteristics with a car rather than aesthetics. Eg it moves at the speed a car moves at, it's in the road, it's overtaking on the right hand lane. I would expect any autonomous car to be able to fail over to "this object is likely a vehicle I haven't seen before" given a strange car-like object being detected.
Great. Now the problem you've posed is no longer image classification. It's more like video classification or zero-shot classification! (neither of which are close to solved)
> What if every time Ford/GM/Toyota substantially changes the look of their cars, your classifier no longer recognizes them
My answer was probably incomplete, but I took the above to mean that cosmetic changes to vehicles mean that classifiers no longer identify them as cars, and this detrimentally modifies the behaviour of the car.
Whilst it's trivial to envisage a scenario where your problem is solved systemically (sufficient training data for a new chassis released in advance or something), it seems like it would be possible to train based on "things we expect to see from any car".
As far as I know, that's how all of the existing methods operate. They seem to have a hierarchy for decision-making:
0. Is there an object around me which I need to consider? If not, continue to monitor for one whilst operating the vehicle within the parameters of road signs and conditions.
1. Is this an object which has a predictable path based on either its signals or the expected behaviour of a car on this part of the road / operating within the parameters set by the road signs I can see?
2. Is this an object which is operating safely despite not falling into category 1?
3. Is this an object which I need to take action to avoid?
Which is to say that it ought to be possible to "fool" a Tesla with a non-car object behaving in a similar fashion to a car. The Tesla sees an object, not a car.