Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

youtube.com

Post reply on HN
Print view View original

121–130 of 214 posts

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#121

Post by stefan_ »

kortex wrote: post #105
bobsomers wrote: post #59

Earlier quoted context omitted.

This framing is a common error in the debate. It's not cameras or lidar, it's cameras or cameras + lidar + radar. Nobody is driving on lidar alone. Many others actually have more cameras and are doing substantially more vision than Tesla is, they're just fusing lidar and radar perception with their vision pipeline. It gives you a more robust view of the world than using a single sensor modality.

If you have one piece of rotten meat in a perfect stew, you still have a disgusting dish. Good sensor fused with garbage in is still garbage in. That was one of the major points of the talk - the vision-only system is more accurate than the one with other modalities thrown in, even though the latter has more data. We intuit that the fusion network should just learn to ignore the bad sensor when it's unreliable, but t…

You have fallen victim to the trap that this video so perfectly laid out for you. Tesla used the mmWave radar that has been in cars forever since it's a good way to do emergency braking and things like adaptive cruise control, particularly when you are a new company and you need these capabilities on your luxury sedan from day one. Now that they are much further along in their FSD efforts, they realize this mmWave radar isn't very helpful anymore. Cool, but nobody else was using it to begin with. LIDAR is totally and utterly different sensor technology.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#122

Post by aaaxyz »

kortex wrote: post #105
bobsomers wrote: post #59

Earlier quoted context omitted.

This framing is a common error in the debate. It's not cameras or lidar, it's cameras or cameras + lidar + radar. Nobody is driving on lidar alone. Many others actually have more cameras and are doing substantially more vision than Tesla is, they're just fusing lidar and radar perception with their vision pipeline. It gives you a more robust view of the world than using a single sensor modality.

If you have one piece of rotten meat in a perfect stew, you still have a disgusting dish. Good sensor fused with garbage in is still garbage in. That was one of the major points of the talk - the vision-only system is more accurate than the one with other modalities thrown in, even though the latter has more data. We intuit that the fusion network should just learn to ignore the bad sensor when it's unreliable, but t…

>We intuit that the fusion network should just learn to ignore the bad sensor when it's unreliable, but this rarely happens in practice.

That sounds like a problem with the network's architecture and not the data itself

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#123

Post by kortex »

aimkey wrote: post #102

What an annoying charlatan. Karpathy is a brilliant computer vision engineer, but he has let his expertise in that subfield cloud his judgement on achieving the overall goal of autonomous driving. Musk and Karpathy have been dead wrong about LIDAR for years. Remember Musk making the absurd claim of a million Tesla robotaxis by 2020? I think most hilarious is that both Karpathy and Musk claim the LIDAR systems are too…

Re: senses - have you ever played a driving sim? You can drive just fine with vision without tactile.

Sirens are primarily a means to get you to look in a direction. 360° cameras can notice the emergency vehicle as soon as it's visually relevant. And if they decide they need an audio siren detector, that's like, practically intern level signals detection at this point. Hardly a dealbreaker.

100% hands-down would pick autonomous car vs airplane. Flying a plane isn't just moving the aluminum bird through 3 space and periodically taking off or landing.

Autonomous aircraft don't exist because a huge amount of the ritual of flight is before and after the captain is even on the plane, let alone flying. There is a tremendous amount that the pilot and copilot go through, on the ground, before taxi, after liftoff. It's way, way more involved and way more generally intelligent. We can design AI to take off, path to a destination, and land. Those 3 things are the easiest parts of flying, yet do not comprise the act of flying a 2-seater, let alone an airliner.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#124

Post by nickik »

stefan_ wrote: post #30
aeternum wrote: post #14

Earlier quoted context omitted.

In the talk, he gave clear examples with detailed position + velocity graphs where the vision system detected obstacles sooner and with less jitter than the radar system. Specifically the overpass where radar triggers erroneous braking, and the pulled over truck where radar detects the obstacle significantly slower.

That's a strawman. No one is looking to build FSD with radar sensors that have shipped on cars for 20 years now for things like adaptive cruise control. LIDAR is what vision only is compared to.

Radar is the technology that is actually shipping in millions of cars. Lidar is future tech that is not practically to deploy in millions of cars as of now.

There is absolutely 0 chance they could just 'put in lidar' from now on so for what he has to do its not relevant.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#125

Post by aimkey »

kortex wrote: post #123
aimkey wrote: post #102

What an annoying charlatan. Karpathy is a brilliant computer vision engineer, but he has let his expertise in that subfield cloud his judgement on achieving the overall goal of autonomous driving. Musk and Karpathy have been dead wrong about LIDAR for years. Remember Musk making the absurd claim of a million Tesla robotaxis by 2020? I think most hilarious is that both Karpathy and Musk claim the LIDAR systems are too…

Re: senses - have you ever played a driving sim? You can drive just fine with vision without tactile. Sirens are primarily a means to get you to look in a direction. 360° cameras can notice the emergency vehicle as soon as it's visually relevant. And if they decide they need an audio siren detector, that's like, practically intern level signals detection at this point. Hardly a dealbreaker. 100% hands-down would pick…

Mentioning the senses wasn't meant to be an itemization of "senses" that a car or car operator needs. Of course you can still drive decently while deaf and without feeling the g-forces. The point is that vision is NOT the only input. In semi modern (non-self-driving) cars, the driver is still assisted by a flurry of additional vehicle sensors in addition to vision and the other human senses. So dismissing LIDAR as "not needed" is foolish.

I understand the complexities of modern commercial flight. I still consider it far less complex (computationally) than driving on today's public road. The fact that you have V2V communication out of the box (via transponders) is probably the biggest factor. I would not rule out autonomous driving WITH a standardized V2V / V2E system. Without one? I'm bearish.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#126

Post by nickik »

aiddun wrote: post #96

Per "it's unscalable to get HD 3D maps of all the roads on earth", it's interesting to consider that Google/Waymo has been growing this for years with street view and the sensors on each car. Curious to see how that plays out

HD 3D maps need to be way more accurate and be enriched with massive amounts of detail. Like where lights are, what lights are relevant for what lanes and so on.

You can't just pull out your street view footage from 3 years ago.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#127

Post by sumnuyungi »

aimkey wrote: post #93
01100011 wrote: post #49

Earlier quoted context omitted.

The fallacy here is that the scale of the neural network used by Tesla is sufficient to capture the problem of driving given enough training. There is no guarantee that a reasonably priced neural network can encompass the task of driving. Having training data beyond a certain point is overrated, and Tesla's advantage in gathering it is overstated. Other companies are capturing this data as well. Is there any indicati…

It seems as if the people gobbling up the "Tesla has the data! Autopilot will keep getting better!" line have never trained a neural network in their life. Models converge. Loss stops decreasing, regardless of more incoming data. Extreme manual data cleaning effort becomes required to prevent overfitting. Model architecture has to change and hyper parameters have to be tweaked. Then you're back at square one as far a…

You make it sound extremely manual and sequential when reality is anything but.

A team with funds like Tesla, Google, FAIR is going to be using NAS and have a continuous testing pipeline. Tesla has arguably the best environment for continuous testing which is the most difficult part of improving a model. Andrej even said in his talk that their supercomputer is in the top 5 for FLOPs.

SOTA on ImageNet for the past few years has been driven by pre-training on massive datasets. Vision transformers are increasingly more common and are extremely data-hungry.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#128

Post by gwern »

kortex wrote: post #115
aimkey wrote: post #93

Earlier quoted context omitted.

It seems as if the people gobbling up the "Tesla has the data! Autopilot will keep getting better!" line have never trained a neural network in their life. Models converge. Loss stops decreasing, regardless of more incoming data. Extreme manual data cleaning effort becomes required to prevent overfitting. Model architecture has to change and hyper parameters have to be tweaked. Then you're back at square one as far a…

> And, in fact, quite insulting to the intelligence of even the most casual ML engineers. Exactly, casual ML engineers. The issue of plateauing tends to occur because there is no more novelty to be had in the data. What mega-experiments like GPT and similar have shown us is that actually you can keep adding novel data and keep improving the model. Kinda inelegant, yet effective. The problem is, most institutions can'…

> Tesla merely has to open the money tap to get more of both compute and storage, and let the real-time data flow in.

And if you watch the other parts of the presentation, you'll see the bits about them buying clusters with 5k+ A100 GPUs. Presumably they intend to do something with those. Probably not streaming Fortnite concerts.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#129

Post by nickik »

aimkey wrote: post #102

What an annoying charlatan. Karpathy is a brilliant computer vision engineer, but he has let his expertise in that subfield cloud his judgement on achieving the overall goal of autonomous driving. Musk and Karpathy have been dead wrong about LIDAR for years. Remember Musk making the absurd claim of a million Tesla robotaxis by 2020? I think most hilarious is that both Karpathy and Musk claim the LIDAR systems are too…

> Musk and Karpathy have been dead wrong about LIDAR for years.

Right, because all those companies that use LIDAR are making billions driving people around. Oh, wait, actually they are burning 100 of millions every year.

> I think most hilarious is that both Karpathy and Musk claim the LIDAR systems are too expensive.

That's literally the opposite of what Musk says about Lidar. He LITERALLY said he wouldn't use them if they were free.

> yet stumbling towards autonomy on vision only is

Look up the Marginal Revolution from 1870.

> The entire ...

... The entire paragraph is an exercise in missing the point.

It seems really what you are saying is not that Tesla are charlatan but the whole industry is.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#130

Post by sumnuyungi »

stefan_ wrote: post #121
kortex wrote: post #105

Earlier quoted context omitted.

If you have one piece of rotten meat in a perfect stew, you still have a disgusting dish. Good sensor fused with garbage in is still garbage in. That was one of the major points of the talk - the vision-only system is more accurate than the one with other modalities thrown in, even though the latter has more data. We intuit that the fusion network should just learn to ignore the bad sensor when it's unreliable, but t…

You have fallen victim to the trap that this video so perfectly laid out for you. Tesla used the mmWave radar that has been in cars forever since it's a good way to do emergency braking and things like adaptive cruise control, particularly when you are a new company and you need these capabilities on your luxury sedan from day one. Now that they are much further along in their FSD efforts, they realize this mmWave ra…

Didn't Andrej show examples of emergency braking and how poorly radar performed v. vision?
Post reply on HN