Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Unintuitive Take on Data Augmentation for Self-Driving Cars

towardsdatascience.com

Post reply on HN
Print view View original

31–38 of 38 posts

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#31

Post by Piskvorrr »

mikeash wrote: post #29
Piskvorrr wrote: post #28

Earlier quoted context omitted.

With the obvious assumption that we do have something better. So far...nope.

With the assumption that we will, not that we currently do. That’s not a given, but it seems likely. The bar to clear is not high.

And what was an assumption, again becomes a circular axiom by the end of the line: "[it will be easy to do, because] the bar to clear is not high, [because it's easy]" Do you have any data to support that? (Or more precisely, are we talking about the 80/20 Pareto bar? ("it is okay, as long as it kills fewer people per million miles, on average"))

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#32

Post by mikeash »

Piskvorrr wrote: post #31
mikeash wrote: post #29

Earlier quoted context omitted.

With the assumption that we will, not that we currently do. That’s not a given, but it seems likely. The bar to clear is not high.

And what was an assumption, again becomes a circular axiom by the end of the line: "[it will be easy to do, because] the bar to clear is not high, [because it's easy]" Do you have any data to support that? (Or more precisely, are we talking about the 80/20 Pareto bar? ("it is okay, as long as it kills fewer people per million miles, on average "))

It’s not circular. I’m referring to specific objections people raise, like misclassifying objects or failing to react or whatever. The current state of the art often spends several seconds at a time with its cameras pointed at a screen instead of at the road.

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#33

Post by function_seven »

mikeash wrote: post #32
Piskvorrr wrote: post #31

Earlier quoted context omitted.

And what was an assumption, again becomes a circular axiom by the end of the line: "[it will be easy to do, because] the bar to clear is not high, [because it's easy]" Do you have any data to support that? (Or more precisely, are we talking about the 80/20 Pareto bar? ("it is okay, as long as it kills fewer people per million miles, on average "))

It’s not circular. I’m referring to specific objections people raise, like misclassifying objects or failing to react or whatever. The current state of the art often spends several seconds at a time with its cameras pointed at a screen instead of at the road.

I appreciate you continuing to refer to sacks of meat as "the current state of the art". It does drive the point home that we don't need to outrun the bear, just the other camper :)

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#34

Post by oldgradstudent »

wcoenen wrote: post #17

I find it interesting that the project team had been implementing several "improvements" that actually worsened performance, and that it took an intern to figure this out. Or are these augmentations default-on options in deep learning frameworks?

You're right to put the "improvements" in scare quotes.

He improved the performance in normal conditions, but he might have made the performance outside normal catastrophic.

When you can't kill anyone more often than once in 100 million miles, this might not be the right tradeoff.

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#35

Post by Piskvorrr »

mikeash wrote: post #32
Piskvorrr wrote: post #31

Earlier quoted context omitted.

And what was an assumption, again becomes a circular axiom by the end of the line: "[it will be easy to do, because] the bar to clear is not high, [because it's easy]" Do you have any data to support that? (Or more precisely, are we talking about the 80/20 Pareto bar? ("it is okay, as long as it kills fewer people per million miles, on average "))

It’s not circular. I’m referring to specific objections people raise, like misclassifying objects or failing to react or whatever. The current state of the art often spends several seconds at a time with its cameras pointed at a screen instead of at the road.

Okay, "baseless" then. "It's easy [that's the baseless claim], we just haven't gotten around to it [which tends to point to the task not actually being easy]." In my opinion, this is the same class of "easy" that AI researchers have been chasing for half a century now, and the end goal always seems to be juuust out of reach - in other words, seems easy but isn't.

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#36

Post by Piskvorrr »

function_seven wrote: post #33
mikeash wrote: post #32

Earlier quoted context omitted.

It’s not circular. I’m referring to specific objections people raise, like misclassifying objects or failing to react or whatever. The current state of the art often spends several seconds at a time with its cameras pointed at a screen instead of at the road.

I appreciate you continuing to refer to sacks of meat as "the current state of the art". It does drive the point home that we don't need to outrun the bear, just the other camper :)

I suppose that's a part of the issue: the SDV camp has been overenthusiastic in flying their "Mission Accomplished" banners, and hitting (pun not intended) yet another unexpected problem right afterwards. In other words, we're not at that point yet - in fact, we might not even have the complete toolset to measure this.

The field is in flux - alchemic approaches "what if we try something unrelated" might work for unrelated reasons etc.; practical applications that don't spontaneously combust are still some way out there.

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#37

Post by mikeash »

Piskvorrr wrote: post #35
mikeash wrote: post #32

Earlier quoted context omitted.

It’s not circular. I’m referring to specific objections people raise, like misclassifying objects or failing to react or whatever. The current state of the art often spends several seconds at a time with its cameras pointed at a screen instead of at the road.

Okay, "baseless" then. "It's easy [that's the baseless claim], we just haven't gotten around to it [which tends to point to the task not actually being easy]." In my opinion, this is the same class of "easy" that AI researchers have been chasing for half a century now, and the end goal always seems to be juuust out of reach - in other words, seems easy but isn't.

I didn’t say it’s easy. My point is merely that current systems are horribly flawed and the needs to be kept in mind when evaluating the flaws of the new systems. The fact that neural net systems are hard to understand and have no systematic design is not necessarily a blocker.

Re: An Unintuitive Take on Data Augmentation for Self-Driving Cars

#38

Post by Piskvorrr »

mikeash wrote: post #37
Piskvorrr wrote: post #35

Earlier quoted context omitted.

Okay, "baseless" then. "It's easy [that's the baseless claim], we just haven't gotten around to it [which tends to point to the task not actually being easy]." In my opinion, this is the same class of "easy" that AI researchers have been chasing for half a century now, and the end goal always seems to be juuust out of reach - in other words, seems easy but isn't.

I didn’t say it’s easy. My point is merely that current systems are horribly flawed and the needs to be kept in mind when evaluating the flaws of the new systems. The fact that neural net systems are hard to understand and have no systematic design is not necessarily a blocker.

Not a blocker, but an eventual solution needs to come up with a meaningful comparison metric - the ones referred to now are meaningless ("no crashes except for things that are not counted as crashes under our definition, which doesn't look suspiciously circular").
Post reply on HN