Inside Waymo's Secret World for Training Self-Driving Cars • Hacker News Board

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#111

Post by ghshephard » Thu, Aug 24, 2017, 2:37 PM UTC

skywhopper wrote: post #97

I'm disappointed the article wasn't a bit more skeptical of some of the claims. Certainly the simulation-based testing is a good thing, but stats about how many billions of simulated miles have been driven can create a self-reinforcing delusion if everyone involved isn't careful to remember that the simulations can only work with well-known and expected situations. It sounds like Waymo realizes this and is building a…

"One million simulated miles are not worth a hundred real miles in terms of confidence in the system"

I'm wondering why you say that? Wouldn't the more relevant factor be the type of miles, regardless of whether they are simulated or real? For example, 1 mile of simulated "Interesting Scenarios" (Duck on Street, Bicyclist going wrong way, pedestrian running onto road) is likely worth 100,000 miles of normal Freeway driving.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#112

Post by ghaff » Thu, Aug 24, 2017, 2:45 PM UTC

dxbydt wrote: post #106

Animats wrote: post #46

There's a lot of misunderstanding about self-driving. Mostly because nobody is publishing much. If you want to do it right, you start with geometry. The first step is capturing range imagery and grinding it down to a 3D model of the world. This tells you where you physically can go. That's where we were at the DARPA Grand Challenge over a decade ago. Then comes moving object popout. What out there isn't a stationary…

> If you want to do it wrong, you start out with lane keeping...recognition of common objects...some machine learning....hope for the best Thank you! Can I ask why the self-driving car programs like the 3-semester Udacity one deliberately focus on going about it the wrong way ? Why spend 1 whole semester on lane detection using trivia like OpenCV, 1 whole semester on object classification using CNNs with Keras, spend…

Isn't the practical reality that any course like that will address the theory and essentially deal with more or less toy problems? It would be hard to get into cutting edge research in an online course.

I'm not sure learning about natural language processing or image classification would be all that different--although those problems are arguably better understood and more bounded.

Even a site reliability engineering course isn't really going to give you deep insights into what, say, Google does on a day-to-day basis.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#113

Post by notahacker » Thu, Aug 24, 2017, 2:55 PM UTC

Retric wrote: post #107

notahacker wrote: post #84

Earlier quoted context omitted.

Novel scenarios might not be common compared with miles of traffic-following drudgery, but even really bad human drivers deal with novel scenarios on the road more often than they have accidents. Stopping might be a sensible safety protocol in some situations, but it isn't in others (not to mention the situations where the car may stop too late because it doesn't actually recognise that a novel scenario is about to o…

Except self driving cars are paying full attention all the time and can react significantly faster. This causes the difference in stopping distance to be dramatic. Remember, humans are basically going to do the same thing for the first 0.25 seconds in any emergency situation and that's the best case. So, self driving cars can simply be very cautious without seeming to.

I agree that self driving cars' response time can sometimes compensate for lack of general intelligence to anticipate a visible roadside activity developing into a hazard or non-routine situations in which another driver might cut into their lane. But lightning reflexes aren't going to eliminate situations in which buggy, late or nonexistent responses to things an AI hasn't been trained to deal with endanger other road users, especially when said other road users don't have lightning reflexes themselves.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#114

Post by timcamber » Thu, Aug 24, 2017, 2:59 PM UTC

dxbydt wrote: post #106

Animats wrote: post #46

There's a lot of misunderstanding about self-driving. Mostly because nobody is publishing much. If you want to do it right, you start with geometry. The first step is capturing range imagery and grinding it down to a 3D model of the world. This tells you where you physically can go. That's where we were at the DARPA Grand Challenge over a decade ago. Then comes moving object popout. What out there isn't a stationary…

> If you want to do it wrong, you start out with lane keeping...recognition of common objects...some machine learning....hope for the best Thank you! Can I ask why the self-driving car programs like the 3-semester Udacity one deliberately focus on going about it the wrong way ? Why spend 1 whole semester on lane detection using trivia like OpenCV, 1 whole semester on object classification using CNNs with Keras, spend…

As a Udactiy SDC student finishing up the program without much real insight into the industry, the program seems very much to me like an entry-level overview of topics you're likely to encounter in this field: computer vision, machine/deep learning, path-finding, sensors, and some basic system integration.

By the end, you'll have a few toy projects and a very basic ability to converse about or study more deeply these topics. Maybe they should have spent a little more time on maps and SLAM instead of other stuff, but hey, it's their first version anyway; I'm sure they'll iterate (as they already have somewhat in response to feedback). They did kind of admit in the CV introduction (lane finding) that this was kind of a super simple, satisfying project to start with. I feel like they often touch on topics acknowledging they have more of a historical relevance and are no longer state-of-the-art tools.

(Edit: I should also note: while I have criticisms of the program, overall I'm very happy that I took it. I'm sure some people are better at seeking out specifically what information is relevant to dig deeper into, but I really appreciate the broad overview kind of pointing me in the right direction. Since starting the program I've amassed quite a list of resources to dig deeper into, and hope to eventually find work in this or related field)

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#115

Post by Zigurd » Thu, Aug 24, 2017, 3:00 PM UTC

joering2 wrote: post #75

Great to see such progress made by Waymo and Co. But as the Murphy's law says, if something might break, it will eventually break. With possibly millions of self driving cars and trillions of unique situations, its inevitable that someone will get hurt. So my question is: what kind of progress is being made to draft a legal framework for situation in which I rode my bike in bike lane and for whatever reason self-driv…

Globally, the death toll on roads is about 1 million per year. If autonomous vehicles make a significant cut in that number, people will start to see human drivers as the more-difficult liability issue.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#116

Post by Retric » Thu, Aug 24, 2017, 3:04 PM UTC

notahacker wrote: post #113

Retric wrote: post #107

Earlier quoted context omitted.

Except self driving cars are paying full attention all the time and can react significantly faster. This causes the difference in stopping distance to be dramatic. Remember, humans are basically going to do the same thing for the first 0.25 seconds in any emergency situation and that's the best case. So, self driving cars can simply be very cautious without seeming to.

I agree that self driving cars' response time can sometimes compensate for lack of general intelligence to anticipate a visible roadside activity developing into a hazard or non-routine situations in which another driver might cut into their lane. But lightning reflexes aren't going to eliminate situations in which buggy, late or nonexistent responses to things an AI hasn't been trained to deal with endanger other ro…

Can you give an actual example? Because, not being able to identify something is not necessarily an issue as long as the car notices something is there and it should not hit it.

EX: I am sure the car had no idea what this was: https://youtu.be/Uj-rK8V-rik?t=26m11s but as long as it can tell it's bigger than a bread box and so it should not to hit it that's enough.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#117

Post by adrianpike » Thu, Aug 24, 2017, 3:19 PM UTC

bit_logic wrote: post #48

Self driving cars is based on machine learning which is basically processing massive amounts of past data. This is great for routine situations and it seems Waymo is making good progress covering most of these. However the key weakness is the lack of true intelligence. When anything unusual or unexpected happens, the best it can do is simply safely shutdown and wait for a human to intervene. And the car can't really…

> Data gathering driver.

Anecdotally, Uber has been gathering lots of road quality data from its drivers for years. Not sure if it still holds, but they were recording and uploading accelerometer data for a while.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#118

Post by ghaff » Thu, Aug 24, 2017, 3:19 PM UTC

Animats wrote: post #46

There's a lot of misunderstanding about self-driving. Mostly because nobody is publishing much. If you want to do it right, you start with geometry. The first step is capturing range imagery and grinding it down to a 3D model of the world. This tells you where you physically can go. That's where we were at the DARPA Grand Challenge over a decade ago. Then comes moving object popout. What out there isn't a stationary…

One of the things that seems to be going on is that, after relatively limited visible progress for a long time, the ability to collect and analyze large datasets suddenly produced fairly striking results. And this in turn has led to a lot of thinking along the lines of "We just need to collect more data and crunch the numbers a bit better."

As you suggest, for at least some classes of problems, we'll find that this approach will asymptotically approach some level of "good" that turns out to not actually be good enough. And the ways to get beyond that will be hard and may not be obvious.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#119

Post by joshuamorton » Thu, Aug 24, 2017, 3:20 PM UTC

ghaff wrote: post #112

dxbydt wrote: post #106

Earlier quoted context omitted.

> If you want to do it wrong, you start out with lane keeping...recognition of common objects...some machine learning....hope for the best Thank you! Can I ask why the self-driving car programs like the 3-semester Udacity one deliberately focus on going about it the wrong way ? Why spend 1 whole semester on lane detection using trivia like OpenCV, 1 whole semester on object classification using CNNs with Keras, spend…

Isn't the practical reality that any course like that will address the theory and essentially deal with more or less toy problems? It would be hard to get into cutting edge research in an online course. I'm not sure learning about natural language processing or image classification would be all that different--although those problems are arguably better understood and more bounded. Even a site reliability engineering…

>Even a site reliability engineering course isn't really going to give you deep insights into what, say, Google does on a day-to-day basis.

I think this really gets to the core of the issue. There aren't many courses that will accurately reflect the actual state of the art in any field. Sometimes, graduate courses with a small scope and a relevant professor taught in person can partially address the state of the art, but an online course about "how to self driving car" is not that.

Re: Inside Waymo's Secret World for Training Self-Driving Cars

#120

Post by organsnyder » Thu, Aug 24, 2017, 3:33 PM UTC

dsfyu404ed wrote: post #93

kylegordon wrote: post #70

Earlier quoted context omitted.

I wonder how many disengagements per unit of distance your average human has...

If you ignore low speed, low visibility parking lot stuff many people will go their whole lives without being in an accident that they are at least partially responsible for.

A "disengagement" isn't defined as an at-fault accident, though—it's a situation where the AI is unsure what to do, so it reverts to human control.

I think the equivalent for a human driver would be a situation where they're unsure what to do, so they freeze up, or proceed extremely cautiously. I'm not prone to freezing up, but I've definitely encountered situations where I'm uncertain to the point that I proceed with the utmost of caution.