Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

I resonantly had an argument on here where somebody insistent that breaking because of over-passes were issues with vision system. Seem pretty clear that it is the resolution of the radar, not the shadow of the bridge that causes the issue. Good to get some more insight into this.

This is the right thing to focus on, as it is by far the largest issue with Autopilot on the highway. Multiple people who do testing of these system that false positives on some highway overpasses are the biggest usability issue.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#6

Post by nickik » Mon, Jun 21, 2021, 8:44 PM UTC

sam_goody wrote: post #2

tldr: Tesla uses vision alone, and has dropped radar and the other sensor. He makes a very decent argument why. (Surprisingly, he basically ignores night driving.)

The list of triggers contains things like 'motorcycles at night', so it seems its all in that dataset.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#7

Post by ejdyksen » Mon, Jun 21, 2021, 9:13 PM UTC

dang wrote: post #4

childintime wrote: post #3

The video: https://youtu.be/NSDTZQdo6H8

Thanks! Maybe it's best if we change the URL to that from https://twitter.com/vpj/status/1407000737423368197 .

That video is a screen capture from another video (which was screen capped from a livestream), but the original has much better audio quality.

Here's a direct link:

https://www.youtube.com/watch?v=eOL_rCK59ZI&t=28293s

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#8

Post by andyxor » Mon, Jun 21, 2021, 9:18 PM UTC

I like Andrej from his PhD research days and awesome blog posts but this is a series of disasters in the making, that is until FTC steps in after more people die from “self-driving” accidents under interesting and unexpected circumstances.

The whole vision vs. LIDAR stuff is a distraction as long as Tesla “AI” doesn’t have common sense.

It literally doesn’t know what it’s doing, and the tail of edge cases to "fit" the models is infinitely long. ANNs are fundamentally backwards looking and cannot adapt to unforeseen or even slightly unusual combination of circumstances. It will go fine for n miles and will dramatically fail at mile n+1 where a new situation requires understanding of ones surroundings, and n is arbitrary number.

It would be more honest to show the cases where it missed, thankfully there is no lack of them in “FSD beta“ videos on YouTube.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#9

Post by nightski » Mon, Jun 21, 2021, 9:24 PM UTC

It's interesting that an academic conference now feels like a marketing op for industrial research labs more than anything. His claims about how accurate their vision system is and how it is exceeding other sensors is not verifiable in any way to the public. Given how well qualified he is I am sure he is not wrong! Andrej is brilliant. But this is an academic conference right? This isn't open science, it's a discussion about an engineered system. I'm afraid this is the future of ML research (which CV is so heavily dependent on now). Long gone are the days of reading a paper and understanding the approach. Now you need the data and model which may not even be computationally feasible without millions of dollars in hardware. This isn't Tesla's fault or anything, it just makes me sad.

Re: Andrej Karpathy (Tesla): CVPR 2021 Workshop on Autonomous Vehicles [video]

#10

Post by option » Mon, Jun 21, 2021, 9:26 PM UTC

andyxor wrote: post #8

I like Andrej from his PhD research days and awesome blog posts but this is a series of disasters in the making, that is until FTC steps in after more people die from “self-driving” accidents under interesting and unexpected circumstances. The whole vision vs. LIDAR stuff is a distraction as long as Tesla “AI” doesn’t have common sense. It literally doesn’t know what it’s doing, and the tail of edge cases to "fit" th…

There are two fundamental reasons, in principle, vision alone can do it: 1) Humans do it with vision alone 2) You can actually predict lidar’s output with vision alone. So many systems out there actually use lidar to generate more labeled data to make lidar unnecessary